TNO provides a set of Data Apps that can be deployed on your Connector. Deployment is done by extending the Helm chart of your TNO Security Gateway deployment. TNO currently offers the following Data Apps:
In the near future, more data apps will be made open source. Including:
The Broker Data App is used to create a Metadata Broker, as specified in the International Data Spaces Reference Architecture Model. The Metadata Broker consists of a TSG Connector and a Broker Data App. The broker can be queried to discover services and other Participants in the data space. The Self Descriptions of the Connectors in the Data Space are stored in the Metadata Broker, specifically in the Fuseki triple store. The broker can be queried via SPARQL queries. The metadata broker of the TSG also comes with a User Interface , which can be used to view the Participants and Connectors that are registered in the broker. For each Connector, you can also view the Self Description and therefore also the available endpoints and resources.
The deployment of a TSG Metadata Broker is similar to the deployment of a TSG Connector. You need a file like the one in the example below. Replace every value with # in front with your own values.
host: # Your broker URL
coreContainer:
image: docker.nexus.dataspac.es/core-container:feature-remote-attestation-aks-amd-sev
ingress:
path: /(infrastructure|participants|broker)
rewriteTarget: /$1
clusterIssuer: letsencrypt
adminUi:
image: docker.nexus.dataspac.es/ui/core-container-ui:feature-open-sourcing
ingress:
path: /ui(/|$)(.*)
rewriteTarget: /$2
clusterIssuer: letsencrypt
ids:
info:
idsid: &idsid urn:ids:connectors:Broker
curator: urn:ids:participants:Test
maintainer: urn:ids:participants:Test
titles:
- Metadata Broker
descriptions:
- Metadata Broker
accessUrl:
- "https://{{ .Values.host }}/broker"
- "https://{{ .Values.host }}/infrastructure"
- "https://{{ .Values.host }}/participants"
daps:
url: # URL to your DAPS
broker:
id: *idsid
address: https://{{ .Values.host }}/infrastructure
autoRegister: false
routes:
ingress:
http:
- endpoint: infrastructure
dataApp: http://{{ template "tsg-connector.fullname" $ }}-broker-data-app-http:8080/router
- endpoint: participants
dataApp: http://{{ template "tsg-connector.fullname" $ }}-broker-data-app-http:8080/participants
security:
apiKeys:
- id: default
key: # Your API key here. IMPORTANT -- must start with APIKEY- so e.g. APIKEY-B2345sdl2skla23
roles:
- DATA_APP
- PEF_MANAGER
# API User Configuration
users:
- # -- User identifier (also used as username for the user)
id: admin
# -- BCrypt encoded password
password: # your bcrypt encoded password
# -- User role assignments
roles:
- ADMIN
containers:
- type: data-app
image: docker.nexus.dataspac.es/broker/data-app:4.2.0
name: broker-data-app
apiKey: # Your API key here. IMPORTANT -- must start with APIKEY- so e.g. APIKEY-B2345sdl2skla23
config:
connectorSparqlUrl: http://fuseki-fuseki:3030/connectorData
participantSparqlUrl: http://fuseki-fuseki:3030/participantData
users:
admin: # your bcrypt encoded password
- type: helper
name: broker-ui
image: docker.nexus.dataspac.es/broker/generic-ui:master
services:
- port: 80
name: http
ingress:
path: /(.*)
rewriteTarget: /$1
clusterIssuer: letsencrypt
environment:
- name: BROKER_ACCESSURL
value: http://{{ template "tsg-connector.fullname" $ }}-broker-data-app-http:8080/api/
configMaps:
- name: broker-ui-config
mountPath: /app/config/
binaryData:
favicon.ico: # OPTIONAL: add binary for favicon here.
data:
# OPTIONAL: Add config here
config.json: |
{}
The following commands can be used to install Fuseki and the TSG Metadata Broker:
helm upgrade --install -n NAMESPACE fuseki tsg/fuseki --version 1.0.2-master
helm upgrade --install -n NAMESPACE broker tsg/tsg-connector --version 3.2.1-master -f values.broker.yaml
The source code of the OpenAPI data app can be found on Gitlab.
The OpenAPI Data App is a Data App that is capable of exposing a back end API service to an IDS based Data Space using the TNO Security Gateway. By deploying the OpenAPI Data App you can easily expose any existing API service using the API’s openAPI spec.
The generic information flow of the OpenAPI data app is as follows:
As client, the backend system calls the OpenAPI Data App of its own connector. By providing the Forward-ID and Forward-Sender headers, for respectively the Agent ID of the intended receiver and the Agent ID of the sender of the message.
The endpoint of the OpenAPI Data App for forwarding the requests depends on the deployment. In general, the /openapi
endpoint is available on port 8080
. In case of Ingress configuration it will be something like https:///data-app/openapi
. The endpoint requires as first path variable the version of the OpenAPI you want to access, e.g. 1.0.2
for the SCSN API, and this is the base URL of the request, so specific OpenAPI endpoints should be appended to this URL.
In a Sequence Diagram, this looks as follows:
In the Connector Helm chart an example is provided that configures the core container with the OpenAPI data app, which can be found in the examples
folder.
The source code of the Federated Learning data app can be found on Gitlab, just like the source code of the helper.
The Federated Learning Data App is a Data App that is used for orchestrating Federated Learning over IDS communication. Federated learning is a type of machine learning that preserves privacy. The developed solution is suited for Horizontal Federated Learning which uses an Algorithm to Data approach. This means that the data does not need to transfer towards an external instution, but the model travels across the different data sets and averages the results.
The data app is meant to be used in a generic way, meaning any Keras model can be uploaded to it and no changes should be made to the data apps. This means the AI researcher can focus on creating the model and does not need to spend time orchestrating their solution.
The structure of the solution is that there is one data app that is tightly integrated with the TSG, like other data apps, which handles the IDS communication. Together with this data app, a helper is deployed to handle the workload of the training itself. This helper is also coupled to the User Interface. The Data App is written in Kotlin, the helper is written in Python and uses the Tensorflow library to facilitate the use of neural networks. All models that are uploaded in the UI should be Keras models.