The default deployment strategy of the TSG components uses Kubernetes as container orchestration solution. Partly, because of the feature set of Kubernetes that makes managing administrating larger deployments easier, but also because Kubernetes provides a layer of abstraction for the underlying (cloud) infrastructure. This allows the deployments to be highly portable and deployable on a wide variety of ecosystems.
Another aspect of using Kubernetes is the capability of the core container to communicate with the Kubernetes API to orchestrate data apps.
Non-kubernetes deployments are possible, since the deployments rely solely on Docker containers. However, we highly discourage non-Kubernetes deployments, due to the added complexity during the life-time of the deployment, while Kubernetes primarily adds complexity on the deploy-time of deployments but improves the maintainability over time.
All of the provided components can be deployed via Helm. Helm Charts provide a package of resources that can be easily configured and deployed to Kubernetes. These Charts make it easy to deploy several resources in Kubernetes that have to be linked together (e.g. Deployments & Services). Helm uses a template enginge, based on Golang templates, so that with configuration properties the relevant Kubernetes resources are generated.
This configuration allows the TSG components to be deployed on almost all of the Kubernetes environments, by providing for instance the possibility to expose services via
Ingress resources (preferred) or with so-called
An example of the configurabilty can be seen in the main Helm chart, with snippets for both
Ingress-based configuration as
NodePort service-based configuration:
coreContainer: # Ingress Service configuration ingress: path: /(.*) rewriteTarget: /$1 clusterIssuer: letsencrypt
Ingress-based core container deployment
coreContainer: # NodePort Service configuration nodePort: api: 31000 camel: 31001
NodePortservice-based core container deployment
See the Connector Helm chart for the complete overview of configuration options.
Another powerful aspect of using Helm charts is the capability of including Helm charts as dependency for Helm chart. This allows you to include, for instance, a database solution next to your application. Or create a full TSG environment with one chart that includes the relevant sub-charts.
The available Helm charts that can be used are:
connector: The main Helm chart for deploying the core container and relevant deploy-time data apps. This chart can be used standalone, or as a dependency to create a full local environment for testing purposes.
daps: The Helm chart for deploying a Dynamic Attribute Provisioning Service (DAPS) for creating a new identity provider. This chart will in most scenarios be deployed for a test environment, or for a newly created dataspace.
example-deployment: The Helm chart for deploying a full ecosystem with a DAPS, a Broker and two connectors. This chart is only intended for test environments and is not suited for use in production. The accompanying repository also contains scripts for setting up a Certificate Authority (CA) structure. This deployment has no dependencies to external components and can, therefore, be used as an isolated test environment.
The Connector Helm Chart is the primary starting point for TSG Connector deployments on Kubernetes clusters. The sources of the Helm Chart can be found on Gitlab.
The basic steps to install the Connector Helm are as follows:
kubectl create secret generic \ -n NAMESPACE \ ids-identity-secret \ --from-file=ids.crt=./component.crt \ --from-file=ids.key=./component.key \ --from-file=ca.crt=./cachain.crt helm repo add tsg https://nexus.dataspac.es/repository/tsg-helm helm install --create-namespace -n NAMESPACE DEPLOYMENT_NAME tsg/connector --version 3.0.0 -f values.yaml
More information on the connector Helm chart can be found in the README of the chart and the configuration properties of the core container can be found in the Configuration section.
The DAPS chart will be made open source soon.
For the TLS encryption between two connectors, the most used structure is to use Automatic Certificate Management Environment (ACME). For example, by using by using the free Certificate Authority Let’s Encrypt, since the trust aspect of IDS does not originate from the TLS encryption but from the Identity certificates. Which makes that Let’s Encrypt is a simple, automated, and free solution for encrypting the network traffic.
ACME is, however, not a requirement. If you already have a system or process in place for acquiring and managing TLS certificates, it is adviced to use that same system also for the TSG Connector deployments. In the examples we do assume the usage of Let’s Encrypt, but this can be easily adopted with your own certificates, especially for Kubernetes deployments with Ingresses enabled you’d only need to provide the certificate as a secret in the cluster and point the ingress to that.
Most of the Kotlin- or Java-based applications are using Spring Boot. This also means that all of the configuration for these components follow the Spring configuration, which allows the configuration to be manipulated in a couple of different ways. Most relevant:
application.yamlfile: primary configuration file for the components, is created by the Helm charts. This can be modified (e.g. for the core container everything in the
idskey will be placed in this file), for the addition of configuration properties required in certain scenarios (e.g. to modify the allowed HTTP post sizes to allow large files to be transfered).
Environment variables: configured environment variables override default properties and properties set in the
application.yaml file. Spring uses relaxed binding for properties, which means that properties will be matched case-insensitive and ignoring dots, dashes, and underscores. For example, the
security.users.password property can be provided by the
SECURITY_USERS_0_PASSWORD environment variable. Which in turn can be configured with an Kubernetes secret, for example the secret
ids-cc-secret (in the same namespace as the deployment) with key
coreContainer: environment: - name: SECURITY_USERS_0_PASSWORD valueFrom: secretKeyRef: name: ids-cc-secret key: password
The documentation of the core container and data apps provide all of the possible configuration properties that are used by the components.