Credentials setup and Workflow Execution

As a end-user in eFlows4HPC, the process of executing workflows involves defining inputs and triggering the workflow execution via the HPCWaaS API. This section provides guidance for end-users to perform these tasks effectively.

Download required tools

Visit the HPCWaaS API release page on GitHub to download a binary version of the waas Command Line Interface (CLI) that is compatible with your computer.

Alternatively, a Docker image (ghcr.io/eflows4hpc/hpcwaas-api:main-cli) containing the CLI can also be obtained. To utilize the Docker image, the following command can be executed in the terminal:

docker run -ti --rm ghcr.io/eflows4hpc/hpcwaas-api:main-cli help

This is equivalent to executing:

./waas help

SSO authentication

The waas CLI uses Unity for authentication. For each request, the user needs to provide a valid access token. This token can be retrieved by logging in (in a browser) to the /auth/login endpoint of your HPCWaaS server. You can then use this token to launch waas commands. For example:

./waas -t=<access_token> workfows list

It is also possible to pass this token by setting a HW_ACCESS_TOKEN environment variable or by adding access_token: <access_token> to your waas config file.

For more information on how to generate and use access tokens, please refer to the HPCWaaS documentation.

Setup your credentials

To enable secure data transfer and execution of workflows, it is necessary to generate a pair of private and public SSH keys using the CLI. The system generates the key pair and securely stores it in a Vault, with the private key being kept confidential and not accessible.

The public key and key ID are returned upon successful key pair generation and should be carefully recorded as they cannot be retrieved later

$ ./waas --api_url <api_url> -t=<access_token> ssh_keys key-gen
INFO: Below is your newly generated SSH public key.
INFO: Take note of it as you will not see it again.
INFO: You are responsible for adding it to the authorized_keys file on the systems you want to run your workflows.

INFO: SSH key ID: 31...3f
INFO: SSH Public key: ssh-rsa AAA...mH

To grant access to the designated HPC clusters, the SSH Public Key generated during the key pair generation process must be copied to the authorized_keys file located in the .ssh directory of the user’s home directory (${HOME}/.ssh/authorized_keys).

List available workflows

./waas --api_url <api_url> -t=<access_token> workflows list

The above command lists the workflows accessible to you. Take note of the workflow ID of the desired workflow for the next step.

Trigger a workflow execution

To initiate the execution of a workflow, you must first determine the inputs required for the workflow from the Workflow developer. Then, execute the following command to trigger the workflow execution:

./waas --api_url <api_url> -t=<access_token> workflows trigger -f \
    -i input1Name=input1Value -i input2Name=input2Value \
    <workflow_id>

Monitor a workflow execution

In order to monitor a workflow execution, one can use the -f flag on the trigger command. This flag enables the continuous retrieval of the execution status from the HPCWaaS API.

Alternatively, the execution status can be obtained using the execution status command along with the Execution ID, which is returned by the trigger command. The syntax for this command is as follows:

./waas --api_url <api_url> -t=<access_token> executions status <Execution_ID>

It is to be noted that the execution status command also has its own -f flag, which can be used for continuously monitoring the execution status.

Cancel a workflow execution

You may cancel a workflow execution that is currently in progress by utilizing the executions cancel command.

./waas --api_url <api_url> -t=<access_token> executions cancel <Execution_ID>