Deploy a TRE Layer
This page contains:
- a summary of what is involved when deploying a Five Safes TES TRE Layer
- a Sample Deployment Guide
- with recommendations for deploying in Production
Component Summary
Here is a summary of the required components which make up the TRE Layer:
| Component | Notes |
|---|---|
| KeyCloak | The TRE Agent and Egress apps each require a Keycloak realm containing users and clients authorised to access them |
| Minio | An Amazon S3 compatible TRE Storage service |
| RabbitMQ | A message broker used for queueing |
| PostgreSQL | A database for the Submission apps to keep local state |
| Hashicorp Vault | Used for accessing ephemeral credentials as secrets |
| OpenLDAP | Used to provide ephemeral credentials for Trino (if in use as a datasource) |
| Camunda Connectors | A REST API allowing other services (e.g. TRE Agent) to integrate with Camunda |
| Camunda Orchestration | A consolidated service that combines Camunda functionality via Zeebe + Operate + Tasklist |
| ElasticSearch | Used for Camunda’s local datastore such as workflow state |
| TRE-Camunda | A service defining the handlers for ephemeral credentials management. Registers the handlers with Camunda, via Zeebe |
| TRE Agent API | A REST API for TRE Agent functionality |
| TRE Agent GUI | A Web Frontend for TRE Admins to log into and interact with the TRE Agent |
| Egress API | A REST API for Egress Portal functionality |
| Egress GUI | A Web Frontend for Egress Officers to log into and interact with the Egress Portal |
| TES Backend | A standard GA4GH TES implementation to execute the analysis |
You may choose how to distribute your own deployment of these components, as long as they are able to communicate with each other over the network.
For some components, such as KeyCloak or MinIO, you may wish to use your own existing deployments.
User accessible components
A subset of the components will need to be accessible by users, outside of the TRE Layer environent:
| Component | Reason |
|---|---|
| KeyCloak | So that users can authenticate and admins can manage the TRE Agent and Egress KeyCloak Realms |
| TRE Agent GUI | So that TRE Admins can log into and interact with the TRE Agent |
| TRE Agent API | So that TRE Agent GUI browser functionality, and optionally other services, can interact with the TRE Agent REST API |
| Egress GUI | So that Egress Officers can log into and interact with the Egress Portal |
| Egress API | Optional. So that other services can integrate with the Egress REST API |
Sample Deployment Guide
Note that this guide is a sample. It can be modified to reflect the your own infrastructure and configuration choices.
This guide deploys all the components of the TRE Layer (except the TES Backend) on a single host (e.g. a Virtual Machine) using Docker Compose.
All the components are deployed on suitable Docker networks and can communicate with each other as required.
It also automatically configures the necessary KeyCloak realms, ready for following our other guides.
Because the TES Backend is not part of this deployment, it will need to be deployed separately and configured to communicate with the required components.
We also provide guidance for installing Funnel as a TES Backend.
User accessible components
Components which may need to be accessed from outside the host are forwarded to the host on specific ports
| Component | Ports |
|---|---|
| KeyCloak | HTTP: 8085 |
| TRE Agent GUI | HTTP: 8989 |
| TRE Agent API | HTTP: 8072 |
| Egress GUI | HTTP: 8100 |
| Egress API | HTTP: 8101 |
In this sample, where the TES Backend is deployed elsewhere, the TES Backend will also need access to some components:
| Component | Ports |
|---|---|
| MinIO | API HTTP: 9002 |
| TRE Agent API | HTTP: 8072 |
Remember the TES Backend’s environment will also need network access to project data sources, in order for analysis to run against them.
This sample also includes some additional components useful for diagnostic purposes:
| Component | Notes | Ports |
|---|---|---|
| Seq | Logs aggregator and web-based viewer | HTTP: 5341 |
| Adminer | Web-based PostgreSQL client | HTTP: 8087 |
Production Recommendations
To use this sample as a starting point for a production deployment, some general recommendations should be considered.
The ports specified above can be used for configuring a reverse proxy for access to the necessary components.
Deployment steps
To deploy an instance of the TRE Agent and data Egress
Go to the TRE Layer directory
Within the 5S-TES-deployment directory the TRE Layer docker-compose and configuration files are in the TRE directory:
cd TREConfigure the .env file
Unless you are running it locally, change localhost to the machine’s host.
# Set the following env vars to point to your machine's host:
KeycloakHostName=http://localhost:8085
MinioBrowser=http://localhost:9003
EgressKeyCloakClientUIRedirectURL=https//localhost:8100/
EgressKeyCloakTokenExpiredAddressUI=http://localhost:8100/Account/LoginAfterTokenExpired
# For example if running on a VM then change the hostname:
KeycloakHostName=http://<VMHost>:8085
MinioBrowser=http://<VMHost>:9003
EgressKeyCloakClientUIRedirectURL=https//<VMHost>:8100/
EgressKeyCloakTokenExpiredAddressUI=http://<VMHost>:8100/Account/LoginAfterTokenExpired
# Set SubmissionAPIAddressURL to the URL where the Submission API is hosted
# This is assuming you have deployed an instance of the Submission Layer, or that there is an existing Submission Layer you can connect the TRE Agent to.
# If you are using the default docker compose then the port is 5034
# So only the hostname needs to change
SubmissionAPIAddressURL=http://localhost:5034
# Set TesAPIUrl to the URL where the TES API executing agent is hosted. e.g Funnel or TES-K
TesAPIUrl=http://localhost:8000/v1/tasks
# Set output bucket prefix for the TES executing agent to write results to default is:
TesOutputBucketPrefix=s3://
# Optional: The TRE Agent syncs with the Submission Layer Projects every 10 minutes by default, adjust if needed:
syncSchedule=10
Instructions on how to set up a Funnel TES API executing agent can be found here
Run docker-compose
docker-compose up -dTRE Agent
The TRE Agent UI is hosted on port 8989
Enter Submission Credentials
The TRE Agent UI will prompt you to provide the submission credentials in order to authenticate against the Submission Layer API.
By default these are:
Username: accessfromtretosubmission
Password: password123These are the TRE Agent user credentials, created in the Submission Layer Keycloak realm.
Enter Egress Credentials
Next, the TRE Agent UI will prompt you to provide the egress credentials in order to authenticate against the Egress API.
By default these are:
Username: accessfromtretoegress
Password: password123These are the TRE Agent user credentials, created in the Egress Layer Keycloak realm.
Egress
The Egress UI is hosted on port 8100