Five Safes TES All-In-One Quickstart

⚠️

This is not intended to be a production deployment. To run these applications in a deployed environment, see their respective deployment documentation

All-In-One is a demonstration instance of the complete Five Safes TES Weave, intended to be run locally.

The purpose of this Quickstart is allowing you to run all the components of Five Safes TES in a single install, and guide you through sending your first “Hello World” submission to the Five Safes TES. This then will help you to explore how this weave can be useful for your use case, or as part of a development pathway.

The quickstart is expected to take approximately 1 hour to complete.

Prerequisites

To run the quickstart, you will need Docker, which can be installed and running on your machine, following these instructions. Any issues can be raised here.
The Docker Compose files of Five Safes TES Weave need to be downloaded or cloned from the GitHub repository. You can do this by running the following command:

git clone https://github.com/SwanseaUniversityMedical/5S-TES-deployment.git

You do not need a GitHub account to access these, but you may wish to create one in order to keep your configuration script in version control. If you do this or wish to clone the repository, you will also need Git installed on your machine.

Funnel is chosen as the TES executor for the All-In-One quickstart and it needs to be installed beforehand. Instructions can be found here.

Sending Hello World submission

Change to directory containing Docker Compose configuration file

Within the 5S-TES-deployment directory, change to the AllInOne directory where the docker compose file of this stack is located:

cd AllInOne

Start the Docker Service with `DemoMode = true`

In the same directory as the docker-compose.yml file run:

docker compose up -d

By default, the docker compose file is configured to run in DemoMode = true. This will start the stack with the seeding of some demo data and instances. For example, Testing project will be created, TRE named DEMO will be deployed, and users credentials used to communicate between apps will be seeded.

You can check the status of the services by running docker stats in the terminal.

💡

Once the keycloak service has started, the configuration files for three realms DARE-Control, DARE-TRE and Data-Egress will be imported and applied automatically. As a result, the admin user globaladminuser will be created in each realm.

Username: globaladminuser
Password: password123

Set up Funnel with Minio

While the All-in-One components are running, Funnel needs to be configured to use the TRE MinIO instance minioTRE. Follow this guide to set up Funnel with MinIO.

Approve the Project access to the TRE

Follow this guide to approve the project access to the TRE.

To make sure that the project has been approved and Submission Layer is in sync with TRE Agent, choose the project in Submission layer and navigate to the TREs section. You should see Approved status for the TRE.

Prepare to send the submission

To send a real submission, some settings need to be adjusted in the .env file inside the AllInOne directory as follow.

.env

// Turn off Demo mode to execute analysis:
DemoMode = false
 
// Allow Keycloak to not require https:
KeyCloakDemoMode = true
 
// Set UseTESK to true:
UseTESK = true
 
// Set TesAPIUrl to the URL where the TES API executing agent is hosted. e.g Funnel or TES-K:
TesAPIUrl=http://<HostName>:8000/v1/tasks
 
// Output bucket prefix for the TES executing agent to write outputs to, default is s3 bucket prefix:
TesOutputBucketPrefix=s3://

If using a system-wide installation of funnel as the TES API executing agent then the HostName should be host.docker.internal.

You need to restart the containers to apply the new configuration. Use the following command to restart the containers: docker compose up -d

Send the Hello World submission using Swagger API

The Hello World submission is a simple submission that will be used to demo the AllInOne stack.

To send this submission using Submission Layer Swagger API hosted at http://localhost:5034/swagger, follow these steps:

Authorise the request with the Access Token with the Authorise button on the top right. The access token is obtained by this guide.
Click on the POST /v1/tasks endpoint, under the TaskServiceApi section. Then click on the Try it out button.
Enter the following payload, then click on the Execute button:

{
  "state": 0,
  "name": "Hello World",
  "inputs": [],
  "outputs": [
    {
      "name": "Stdout",
      "description": "Stdout results",
      "url": "s3://",
      "path": "/outputs",
      "type": "DIRECTORY"
    }
  ],
  "executors": [
    {
      "image": "ubuntu",
      "command": ["echo", "Hello World"],
      "workdir": "/outputs",
      "stdout": "/outputs/stdout"
    }
  ],
  "volumes": null,
  "tags": {
    "project": "Testing", // Project that was seeded in the Demo mode
    "tres": "DEMO" // TRE that was seeded in the Demo mode
  },
  "logs": null,
  "creation_time": null
}

Tracking and approving the submission results

Navigate back to Submission Layer UI, and click onto the submission you just sent.

You should be able to see the status of the submission. Once the submission has been processed, the status will be set to: Requested Egress. This means that the results are waiting for egress approval. Follow this guide to approve the submission results.

View the submission results

Once the Egress request has been approved, you can view the results in the Submission Layer’s output bucket.

The final successful process should be like this:

Next steps

Now you have sent your first submission, you can try to send a different submission with a different project and TRE using this guide.

Core components Install Funnel TES