How to link multiple Substra backends¶
This guide shows you how to enable communication between two Substra backends to allow them to exchange Algorithms and Models. This can be achieved either at deployment time or when the backend is already deployed.
Updating your Helm values¶
This guide assume that you have two organization in your network, one named MyOrg1 and the other MyOrg2. The process would be the same if you have more organizations as we have to pair them. For example with three orgs you would repeat it for (org1, org2), (org1, org3) and (org2, org3).
In this setup we want MyOrg1 to exchange assets with MyOrg2.
We will assume that you have two values files with the configuration for your existing deployment, one for each organization named: backend-1-values.yaml
, backend-2-values.yaml
.
Configure matching values for your 2 Organizations:
Create an account for MyOrg2 on MyOrg1. In the
backend-1-values.yaml
file, add the following content:addAccountOperator: incomingOrganizations: - name: ORG2NAME secret: SECRET_ORG1_ORG2 channel: CHANNEL
ReplaceORG2NAME
by the name of the organization as defined with the valueorganizationName
in the second organization values file. In our example it would beMyOrg2
.ReplaceSECRET_ORG1_ORG2
with the password that MyOrg2 will need to use to download assets from MyOrg1.ReplaceCHANNEL
with the name of a Channel MyOrg1 and MyOrg2 are part of. This needs to be one of the channels defined in the Backend channel configuration.Create an account for MyOrg1 on MyOrg2. In the
backend-2-values.yaml
file add the following content:addAccountOperator: incomingOrganizations: - name: ORG1NAME secret: SECRET_ORG2_ORG1 channel: CHANNEL
ReplaceORG1NAME
by the name of the organization as defined with the valueorganizationName
in the first organization values file. In our example it would beMyOrg1
.ReplaceSECRET_ORG2_ORG1
with the password that MyOrg1 will need to use to download assets from MyOrg2.ReplaceCHANNEL
with the name of a Channel MyOrg1 and MyOrg2 are part of. This needs to be one of the channels defined in the Backend channel configuration.Configure MyOrg1 to use the right password when connecting to MyOrg2. In the
backend-1-values.yaml
file add the following content under theaddAccountOperator
key:outgoingOrganizations: - name: ORG2NAME secret: SECRET_ORG2_ORG1
ReplaceORG2NAME
with the name of the organization. In our example it would beMyOrg2
.ReplaceSECRET_ORG2_ORG1
with the password defined for MyOrg1 inbackend-2-values.yaml
.Configure MyOrg2 to use the right password when connecting to MyOrg1. In the
backend-2-values.yaml
file add the following content under theaddAccountOperator
key:outgoingOrganizations: - name: ORG1NAME secret: SECRET_ORG1_ORG2
ReplaceORG1NAME
with the name of the organization. In our example it would beMyOrg1
.ReplaceSECRET_ORG1_ORG2
with the password defined for MyOrg2 inbackend-1-values.yaml
.
In the end your configuration files should have a section looking like this:
addAccountOperator:
users: [...]
incomingOrganizations:
- name: ORG2NAME
secret: SECRET_ORG1_ORG2
channel: CHANNEL
outgoingOrganizations:
- name: ORG2NAME
secret: SECRET_ORG2_ORG1
For the backend-1-values.yaml
file.
addAccountOperator:
users: [...]
incomingOrganizations:
- name: ORG1NAME
secret: SECRET_ORG2_ORG1
channel: CHANNEL
outgoingOrganizations:
- name: ORG1NAME
secret: SECRET_ORG1_ORG2
For the backend-2-values.yaml
file.
Deploy the updated chart¶
Now that you have updated your values you can either continue your deployment or update a deployed app. u To update a deployed Substra application run:
helm upgrade RELEASE-NAME substra/substra-backend --version VERSION --values VALUES-FILE
RELEASE-NAME
with the name of your Substra backend release.
You can retrieve it with helm list
.VERSION
with the version of the Substra backend helm chart you want to deploy.VALUES-FILE
with the values file. In our example, backend-1-values.yaml
for the first backend and backend-2-values.yaml
for the second one.This will update the kubernetes resources to reflect your changes.
Validate that organizations are connected¶
We provide a small utility on the Substra backend server to test which organizations are accessible from the current organization. To use this utility follow these steps:
Connect to the Substra backend pod:
kubectl exec -it $(kubectl get pod -l "app.kubernetes.io/name=substra-backend-server" -o name) -- /bin/bash
This will open a shell on the backend server pod.
List all organizations defined in the outgoing list and their status:
./manage.py get_outgoing_organization
The output should look like this:
| org_id | org_address | http_status | | MyOrg2 | http://api.org-2.com:80 | 200 |
If there is an error while trying to connect to the node it will appear in the
http_status
column.