This version is under construction, please use an official release version

Application Definition

An ApplicationDefinition represents a single Application and contains all its versions. It holds the necessary information to install an application. Two types of information are required to install an application:

How to download the application’s source (i.e Kubernetes manifest, helm chart…). We refer to this as source.
How to render (i.e. templating) the application’s source to install it into user-cluster. We refer to this astemplating method.

Each version can have a different source (.spec.version[].template.source) but share the same templating method (.spec.method). Here is the minimal example of ApplicationDefinition. More advanced configurations are described in subsequent paragraphs.

apiVersion: apps.kubermatic.k8c.io/v1
kind: ApplicationDefinition
metadata:
  name: apache
spec:
  description: Apache HTTP Server is an open-source HTTP server for modern operating systems
  method: helm
  versions:
    - template:
        source:
          helm:
            chartName: apache
            chartVersion: 9.2.9
            url: https://charts.bitnami.com/bitnami
      version: 9.2.9
    - template:
        source:
          git:
            path: bitnami/apache
            ref:
              branch: main
            remote: https://github.com/bitnami/charts.git
      version: 2.4.55-git

In this example, the ApplicationDefinition allows the installation of two versions of apache using the helm method. Notice that one source originates from a Helm repository and the other from a git repository

Templating Method

Templating Method describes how the Kubernetes manifests are being packaged and rendered.

Helm Method

This method use Helm to install, upgrade and uninstall the application into the user-cluster.

Templating Source

Helm Source

The Helm Source allows downloading the application’s source from a Helm HTTP repository or an OCI repository. The following parameters are required:

url -> URL to a helm chart repository; If you are unsure about a server, you can always double-check if “<your-url>/index.yaml” returns a valid index
chartName -> Name of the chart within the repository
chartVersion -> Version of the chart; corresponds to the chartVersion field

Example of Helm source with HTTP repository:

- template:
    source:
      helm:
        chartName: prometheus
        chartVersion: 15.10.4
        url: https://prometheus-community.github.io/helm-charts

Example of Helm source with OCI repository:

- template:
    source:
      helm:
        chartName: cilium
        chartVersion: 1.13.0-rc5
        url: oci://quay.io/kubermatic/helm-charts

For private git repositories, please check the working with private registries section.

Currently, the best way to obtain chartName and chartVersion for an HTTP repository is to make use of helm search:

# initial preparation
helm repo add <repo-name> <repo-url>
helm repo update

# listing the names of all charts in a repository
helm search repo <repo-name>

# listing versions of a chart
helm search repo <repo-name>/<chart-name> --versions

# you can also filter for versions. For example, if you want to list all Prometheus helm charts with a chart version of 15 or greater, you can run
helm search repo prometheus-community/prometheus --versions --version ">=15"

For OCI repositories, there is currently no native helm search. Instead, you have to rely on the capabilities of your OCI registry. For example, harbor supports searching for helm-charts directly in their UI.

Git Source

The Git source allows you to download the application’s source from a Git repository.

Example of Git Source:

- template:
    source:
      git:
      path: bitnami/apache
      ref:
        branch: main
      remote: https://github.com/bitnami/charts.git

path -> path where all manifests are stored; In case of the helm templating method, each chart should be inside its own subdirectory inside path
remote -> url to the repository
ref
- branch -> branch from which the chart should be pulled
- tag -> git tag from which the chart should be pulled; Can not be used in conjunction with commit or branch
- commit -> sha of a commit from which the chart should be pulled; Must be used in conjunction with a branch to ensure shallow cloning

For private git repositories, please check the working with private registries section.

Working With Private Registries

For private registries, the Applications Feature supports storing credentials in Kubernetes secrets in the KKP master and referencing the secrets in your ApplicationDefinitions. A KKP controller will ensure that the required secrets are synced to your seed clusters.

In order for the controller to sync your secrets, they must be annotated with apps.kubermatic.k8c.io/secret-type and be created in the namespace that KKP is installed in (unless changed, this defaults to kubermatic).

Git Repositories

KKP supports three types of authentication for git repositories:

password: authenticate with a username and password.
Token: authenticate with a Bearer token
SSH-Key: authenticate with an ssh private key.

Their setup is comparable:

Create a secret containing our credentials

# inside KKP master

# user-pass
kubectl create secret -n <kkp-install-namespace> generic --from-literal=pass=<password> --from-literal=user=<username> <secret-name>

# token
kubectl create secret -n <kkp-install-namespace> generic --from-literal=token=<token> <secret-name>

# ssh-key
kubectl create secret -n <kkp-install-namespace> generic --from-literal=sshKey=<private-ssh-key> <secret-name>

# after creation, annotate
kubectl annotate secret <secret-name> apps.kubermatic.k8c.io/secret-type="git"

Reference the secret in the ApplicationDefinition

spec:
  versions:
    - template:
        source:
          git:
            path: <path-inside-git-repo>
            ref:
              branch: <branch>
            remote: <server-url> # for ssh-key, an ssh url must be chosen (e.g. git@example.com/repo.git)
            credentials:
              method: <password || token || ssh-key>
              # user-pass
              username:
                key: user
                name: <secret-name>
              password:
                key: pass
                name: <secret-name>
              # token
              token:
                key: token
                name: <secret-name>
              # ssh-key
              sshKey:
                key: sshKey
                name: <secret-name>

Compatibility Warning

Be aware that all authentication methods may be available on your git server. More and more servers disable the authentication with username and password. More over on some providers like GitHub, to authenticate with an access token, you must use password method instead of token.

Example of secret to authenticate with GitHub access token:

kubectl create secret -n <kkp-install-namespace> generic --from-literal=pass=<github-access-token> --from-literal=user=<username> <secret-name>

For other providers, please refer to their official documentation.

Helm OCI Registries

Helm OCI registries are being accessed using a JSON configuration similar to the ~/.docker/config.json on the local machine. It should be noted, that all OCI server urls need to be prefixed with oci://.

Create a secret containing our credentials

# inside KKP master
kubectl create secret -n <kkp-install-namespace> docker-registry  --docker-server=<server> --docker-username=<user> --docker-password=<password> <secret-name>
kubectl annotate secret <secret-name> apps.kubermatic.k8c.io/secret-type="helm"

# example
kubectl create secret -n kubermatic docker-registry --docker-server=harbor.example.com/my-project --docker-username=someuser --docker-password=somepaswword oci-cred
kubectl annotate secret oci-cred apps.kubermatic.k8c.io/secret-type="helm"

Reference the secret in the ApplicationDefinition

spec:
  versions:
    - template:
        source:
          helm:
            chartName: examplechart
            chartVersion: 0.1.0
            credentials:
              registryConfigFile:
                key: .dockerconfigjson # `kubectl create secret docker-registry` stores by default the creds under this key
                name: <secret-name>
            url: <server>

Helm Userpass Registries

To use KKP Applications with a helm userpass auth registry, you can configure the following:

Create a secret containing our credentials

# inside KKP master
kubectl create secret -n <kkp-install-namespace> generic --from-literal=pass=<password> --from-literal=user=<username> <secret-name>
kubectl annotate secret <secret-name> apps.kubermatic.k8c.io/secret-type="helm"

Reference the secret in the ApplicationDefinition

spec:
  versions:
    - template:
        source:
          helm:
            chartName: examplechart
            chartVersion: 0.1.0
            credentials:
              password:
                key: pass
                name: <secret-name>
              username:
                key: user
                name: <secret-name>
            url: <server>

Templating Credentials

There is a particular case where credentials may be needed at the templating stage to render the manifests. For example, if the template method is helm and the source is git. To install the chart into the user cluster, we have to build the chart dependencies. These dependencies may be hosted on a private registry requiring authentication.

You can specify the templating credentials by settings .spec.version[].template.templateCredentials. It works the same way as source credentials.

Example of template credentials:

spec:
  versions:
    - template:
        source:
          git:
            path: <path-inside-git-repo>
            ref:
              branch: <branch>
            remote: <server-url> # for ssh-key, an ssh url must be chosen (e.g. git@example.com/repo.git)
        templateCredentials:
          helmCredentials: # this the same struct as .spec.versions[].template.source.helm.credentials
            password:
              key: pass
              name: <secret-name>
            username:
              key: user
              name: <secret-name>

Advanced Configuration

Default Values

The .spec.defaultValuesBlock field describes overrides for manifest-rendering in UI when creating an application. For example if the method is Helm, then this field contains the Helm values.

Example for helm values

spec:
  defaultValuesBlock: |
    key1:
      nestedkey: value 1
    # comments are persisted and will be displayed in the UI as well
    key2: value 2

Customize Deployment

You can tune how the application will be installed by setting .spec.defaultDeployOptions. The options depend on the template method (i.e. .spec.method).

note: defaultDeployOptions can be overridden at ApplicationInstallation level by settings .spec.deployOptions

Customize Deployment For Helm Method

You may tune how Helm deploys the application with the following options:

atomic: corresponds to the --atomic flag on Helm CLI. If set, the installation process deletes the installation on failure; the upgrade process rolls back changes made in case of failed upgrade.
wait: corresponds to the --wait flag on Helm CLI. If set, will wait until all Pods, PVCs, Services, and minimum number of Pods of a Deployment, StatefulSet, or ReplicaSet are in a ready state before marking the release as successful. It will wait for as long as --timeout
timeout: corresponds to the --timeout flag on Helm CLI. It’s time to wait for any individual Kubernetes operation.
enableDNS: corresponds to the -enable-dns flag on Helm CLI. It enables DNS lookups when rendering templates. if you enable this flag, you have to verify that helm template function ‘getHostByName’ is not being used in a chart to disclose any information you do not want to be passed to DNS servers.(c.f. CVE-2023-25165)

Example:

apiVersion: apps.kubermatic.k8c.io/v1
kind: ApplicationDefinition
metadata:
  name: apache
spec:
  defaultDeployOptions:
    helm:
      atomic: true
      wait: true
      timeout: "5m"

note: if atomic is true, then wait must be true. If wait is true then timeout must be defined.

ApplicationDefinition Reference

The following is an example of ApplicationDefinition, showing all the possible options.

apiVersion: apps.kubermatic.k8c.io/v1
kind: ApplicationDefinitions
metadata:
  name: <<appdef-name>>
spec:
  # Description of the application. what is its purpose
  description: ""
  # Method used to install the application
  method: helm
  # Available version for this application
  versions:
    - # Template defines how application is installed (source provenance, Method...)
      template:
        # Defined how the source of the application (e.g Helm chart) is retrieved.
        # Exactly one type of source must be defined.
        source:
          # Install application from a Git repository
          git:
            # Credentials are optional and holds the git credentials
            credentials:
              # Authentication method. Either password or token or ssh-key.
              # if method is password then username and password must be defined.
              # if method is token then token must be defined.
              # if method is ssh-key then ssh-key must be defined.
              method: password ||token || ssh-key
              # Password holds the ref and key in the secret for the Password credential.
              # The Secret must exist in the namespace where KKP is installed (default is "kubermatic").
              # The Secret must be annotated with `apps.kubermatic.k8c.io/secret-type:` set to helm or git
              password:
                # The key of the secret to select from. Must be a valid secret key.
                key: pass
                # Name of the referent.
                # More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
                name: <<secret-name>>
                # Specify whether the Secret or its key must be defined
                optional: false
              # SSHKey holds the ref and key in the secret for the SshKey credential.
              # The Secret must exist in the namespace where KKP is installed (default is "kubermatic").
              # The Secret must be annotated with `apps.kubermatic.k8c.io/secret-type:` set to helm or git
              sshKey:
                # The key of the secret to select from. Must be a valid secret key.
                key: private-key
                # Name of the referent.
                # More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
                name: <<secret-name>>
                # Specify whether the Secret or its key must be defined
                optional: false
              # Token holds the ref and key in the secret for the token credential.
              # The Secret must exist in the namespace where KKP is installed (default is "kubermatic").
              # The Secret must be annotated with `apps.kubermatic.k8c.io/secret-type:` set to helm or git
              token:
                # The key of the secret to select from. Must be a valid secret key.
                key: token
                # Name of the referent.
                # More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
                name: <<secret-name>>
                # Specify whether the Secret or its key must be defined
                optional: false
              # Username holds the ref and key in the secret for the username credential.
              # The Secret must exist in the namespace where KKP is installed (default is "kubermatic").
              # The Secret must be annotated with `apps.kubermatic.k8c.io/secret-type:` set to helm or git
              username:
                # The key of the secret to select from. Must be a valid secret key.
                key: user
                # Name of the referent.
                # More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
                name: <<secret-name>>
                # Specify whether the Secret or its key must be defined
                optional: false
            # Path of the "source" in the repository. default is repository root
            path: charts/apache
            # Git reference to checkout.
            # For large repositories, we recommend to either use Tag, Branch or Branch+Commit. This allows a shallow clone, which dramatically speeds up performance
            ref:
              # Branch to checkout. Only the last commit of the branch will be checkout in order to reduce the amount of data to download.
              branch: main
              # Commit SHA in a Branch to checkout.

              # It must be used in conjunction with branch field.
              commit: 8061ceb738db42fe82b4c305b7aa5459d926d03e
              # Tag to check out.
              # It can not be used in conjunction with commit or branch.
              tag: v1.2.3
            # URL to the repository. Can be HTTP(s) (e.g. https://example.com/myrepo) or SSH (e.g. git://example.com[:port]/path/to/repo.git/)
            remote: https://git.example.com/repo || git@example.com/repo
          # Install Application from a Helm repository
          helm:
            # Name of the Chart.
            chartName: my-app
            # Version of the Chart.
            chartVersion: v13.9.0
            # Credentials are optional and hold the ref to the secret with helm credentials.
            # Either username / Password or registryConfigFile can be defined.
            credentials:
              # Password holds the ref and key in the secret for the Password credential.
              # The Secret must exist in the namespace where KKP is installed (default is "kubermatic").
              # The Secret must be annotated with `apps.kubermatic.k8c.io/secret-type:` set to helm or git
              password:
                # The key of the secret to select from. Must be a valid secret key.
                key: pass
                # Name of the referent.
                # More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
                name: <<secret-name>>
                # Specify whether the Secret or its key must be defined
                optional: false
              # RegistryConfigFile holds the ref and key in the secret for the registry credential file. The value is dockercfg
              # file that follows the same format rules as ~/.docker/config.json
              # The The Secret must exist in the namespace where KKP is installed (default is "kubermatic").
              # The Secret must be annotated with `apps.kubermatic.k8c.io/secret-type:` set to helm or git
              registryConfigFile:
                # The key of the secret to select from. Must be a valid secret key.
                key: .dockerconfigjson
                # Name of the referent.
                # More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
                name: <<secret-name>>
                # Specify whether the Secret or its key must be defined
                optional: false
              # Username holds the ref and key in the secret for the username credential.
              # The Secret must exist in the namespace where KKP is installed (default is "kubermatic").
              # The Secret must be annotated with `apps.kubermatic.k8c.io/secret-type:` set to helm or git
              username:
                # The key of the secret to select from. Must be a valid secret key.
                key: user
                # Name of the referent.
                # More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
                name: <<secret-name>>
                # Specify whether the Secret or its key must be defined
                optional: false
            # URl of the helm repository.
            # It can be an HTTP(s) repository (e.g. https://localhost/myrepo) or on OCI repository (e.g. oci://localhost:5000/myrepo).
            url: https://charts.example.com || oci://localhost:5000/myrepo
      # Version of the application (e.g. v1.2.3)
      version: v1.2.3