Use the Workbench to submit an analysis

Initialise the Workbench

Import the Workbench into the notebook and initialise a Workbench instance by running the cell afterwards.

from five_safes_tes_workbench.workbench import Workbench
 
# Initialise the Workbench
wb = Workbench()

Choose how to validate and pass Workbench configuration

Configuration parameters

The configuration parameters are required to establish a connection to the TES endpoint, Submission layer storage and define which TREs the task will be submitted to.

Validating the configuration

Depend on how you choose to authenticate yourself to the Submission layer API and pass the configuration values for validation, the first step of using Workbench will be different.

Authentication methods

• A. Using your own credentials (recommended, because you don’t need to worry about token expiration)
• B. Using an access token (fast, and less parameters to pass, but token may expire before you retrieve the results)

Passing configuration values methods

• I. Using a YAML config file (you can reuse the config file for future submissions)
• II. Using direct parameters (you and others can see the parameters you are passing in your notebook)

Choose the tab below that corresponds to your chosen authentication and passing configuration values methods, and validate steps will be shown accordingly.

config:
  project: "your-project-name"
  tes_base_url: "http://localhost:5034"
  tres:
    - "Your-TRE1"
    - "Add more TREs as needed"
 
auth:
  client_id: "your-keycloak-client-id"
  client_secret: "your-keycloak-client-secret"
  username: "your-keycloak-username"
  password: "your-keycloak-password"
  keycloak_url: "http://localhost:your-keycloak-port/"

  wb.validate(config_path="path/to/config.yml")

 wb.validate(
   project="your-project-name",
   tes_base_url="http://your-tes-endpoint:5034",
   tres=["Your-TRE1"],
   client_id="your-client-id",
   client_secret="your-client-secret",
   username="your-username",
   password="your-password",
   keycloak_url="http://your-keycloak-endpoint/",
 )

config:
  project: "your-project-name"
  tes_base_url: "http://localhost:5034"
  tres:
    - "Your-TRE1"
    - "Add more TREs as needed"
 
auth:
  access_token: "your-access-token-here"

  wb.validate(config_path="path/to/config.yml")

 wb.validate(
   project="your-project-name",
   tes_base_url="http://your-tes-endpoint:5034",
   tres=["Your-TRE1"],
   access_token="your-access-token"
 )

Build the TES task

Once the Workbench has been validated, the next step is to build the TES task in a new cell.

The Workbench provides a template-based interface via wb.build_tes.<template>(...) which constructs the TES message that will be submitted to the endpoint.

The available templates are:

• Hello World
• Simple SQL
• Bunny
• Custom

Choose the tab below that corresponds to the template you want to use.

wb.build_tes.hello_world()

query = (
    'WITH user_query AS ('
    'SELECT value_as_number FROM "NottinghamDemo".measurement '
    'WHERE measurement_concept_id = 3000905 '
    'AND value_as_number IS NOT NULL'
    ') SELECT COUNT(*) AS n, SUM(value_as_number) AS total FROM user_query;'
)
 
wb.build_tes.simple_sql(
    name="Simple SQL Task",
    query=query,
)

wb.build_tes.bunny(
    name="Bunny Task",
    command=[
        "--body-json",
        '{"code":"GENERIC","analysis":"DISTRIBUTION","uuid":"123","collection":"test","owner":"me"}',
        "--output",
        "/outputs/output.json",
        "--no-encode",
    ],
)
 

wb.build_tes.custom(
    name="Test Custom mode from Workbench",
    description="custom analysis",
    executors=[
        {
            "image": "ubuntu",
            "command": ["echo", "Hello World"],
            "workdir": "/outputs",
            "stdout": "/outputs/stdout"
        }
    ],
    outputs=[
        {
            "name": "Stdout",
            "description": "Stdout results",
            "url": "s3://",
            "path": "/outputs",
            "type": "DIRECTORY"
        }
    ],
)
 

Submit the TES task

Once the TES task has been built, the next step is to submit it to the TES endpoint.

wb.submit()

Fetch the results of the TES task

You can download the output files from Submission layer storage using the fetch methods in a new cell.

Usage

The wb.fetch_outputs() method retrieve results from the TRE for the submissions made in the previous steps. This method works based on the parameters tre,task_id, and output_dir. Depends on if you provide these parameters, the method will behave differently.

Example Implementation:

# Fetch results for all TREs (uses last submitted task ID)
wb.fetch_outputs()
 
# Fetch results for a specific TRE
wb.fetch_outputs(tre="Nottingham TRE 01")
 
# Fetch with an explicit task ID (e.g. from a previous submission)
wb.fetch_outputs(task_id=945)
 
# Fetch for a specific TRE with an explicit task ID
wb.fetch_outputs(task_id=945, tre="Nottingham TRE 01")

How it works

Before downloading any files, the Workbench first queries the submission layer to check the current status of each child task (the per-TRE sub-task created when you called wb.submit()). Only tasks that have reached Completed status will have their files fetched from Submission layer storage. Tasks that are still running or have terminated with an error are skipped.

You can also check the submission layer UI to see the progress of the submission.

Fetching behavior by state:

<output_dir>/
└── <tre_name>/
    └── <child_task_id>/
        ├── <file_name_1>
        └── <file_name_2>
        └── ...

Logs example: single TRE, completed task:

INFO | Child task info: 945, status: Completed
INFO | Fetching token from keycloak...
INFO | Keycloak token fetched successfully
INFO | Exchanging bearer token for MinIO credentials via STS
INFO | MinIO client initialized
INFO | Found 2 result object(s) for task 945
INFO | Downloading result object: 945/acro_output_20260501_085731.zip
INFO | Downloaded -> output/Nottingham TRE 01/945/acro_output_20260501_085731.zip
INFO | Downloading result object: 945/output.json
INFO | Downloaded -> output/Nottingham TRE 01/945/output.json