An ApplicationDefinition represents a single Application and contains all its versions. It holds the necessary information to install an application.
Two types of information are required to install an application:
source.templating method.Each version can have a different source (.spec.version[].template.source) but share the same templating method (.spec.method).
Here is the minimal example of ApplicationDefinition. More advanced configurations are described in subsequent paragraphs.
apiVersion: apps.kubermatic.k8c.io/v1
kind: ApplicationDefinition
metadata:
name: apache
spec:
description: Apache HTTP Server is an open-source HTTP server for modern operating systems
method: helm
versions:
- template:
source:
helm:
chartName: apache
chartVersion: 9.2.9
url: https://charts.bitnami.com/bitnami
version: 9.2.9
- template:
source:
git:
path: bitnami/apache
ref:
branch: main
remote: https://github.com/bitnami/charts.git
version: 2.4.55-git
In this example, the ApplicationDefinition allows the installation of two versions of apache using the helm method. Notice that one source originates from a Helm repository and the other from a git repository
Templating Method describes how the Kubernetes manifests are being packaged and rendered.
This method use Helm to install, upgrade and uninstall the application into the user-cluster.
The Helm Source allows downloading the application’s source from a Helm HTTP repository or an OCI repository. The following parameters are required:
url -> URL to a helm chart repository; If you are unsure about a server, you can always double-check if “<your-url>/index.yaml” returns a valid indexchartName -> Name of the chart within the repositorychartVersion -> Version of the chart; corresponds to the chartVersion fieldExample of Helm source with HTTP repository:
- template:
source:
helm:
chartName: prometheus
chartVersion: 15.10.4
url: https://prometheus-community.github.io/helm-charts
Example of Helm source with OCI repository:
- template:
source:
helm:
chartName: cilium
chartVersion: 1.13.0-rc5
url: oci://quay.io/kubermatic/helm-charts
For private git repositories, please check the working with private registries section.
Currently, the best way to obtain chartName and chartVersion for an HTTP repository is to make use of helm search:
# initial preparation
helm repo add <repo-name> <repo-url>
helm repo update
# listing the names of all charts in a repository
helm search repo <repo-name>
# listing versions of a chart
helm search repo <repo-name>/<chart-name> --versions
# you can also filter for versions. For example, if you want to list all Prometheus helm charts with a chart version of 15 or greater, you can run
helm search repo prometheus-community/prometheus --versions --version ">=15"
For OCI repositories, there is currently no native helm search. Instead, you have to rely on the capabilities of your OCI registry. For example, harbor supports searching for helm-charts directly in their UI.
The Git source allows you to download the application’s source from a Git repository.
Example of Git Source:
- template:
source:
git:
path: bitnami/apache
ref:
branch: main
remote: https://github.com/bitnami/charts.git
path -> path where all manifests are stored; In case of the helm templating method, each chart should be inside its own subdirectory inside pathremote -> url to the repositoryrefbranch -> branch from which the chart should be pulledtag -> git tag from which the chart should be pulled; Can not be used in conjunction with commit or branchcommit -> sha of a commit from which the chart should be pulled; Must be used in conjunction with a branch to ensure shallow cloningFor private git repositories, please check the working with private registries section.
For private registries, the Applications Feature supports storing credentials in Kubernetes secrets in the KKP master and referencing the secrets in your ApplicationDefinitions. A KKP controller will ensure that the required secrets are synced to your seed clusters.
In order for the controller to sync your secrets, they must be annotated with apps.kubermatic.k8c.io/secret-type and be created in the namespace that KKP is installed in (unless changed, this defaults to kubermatic).
KKP supports three types of authentication for git repositories:
password: authenticate with a username and password.Token: authenticate with a Bearer tokenSSH-Key: authenticate with an ssh private key.Their setup is comparable:
# inside KKP master
# user-pass
kubectl create secret -n <kkp-install-namespace> generic --from-literal=pass=<password> --from-literal=user=<username> <secret-name>
# token
kubectl create secret -n <kkp-install-namespace> generic --from-literal=token=<token> <secret-name>
# ssh-key
kubectl create secret -n <kkp-install-namespace> generic --from-literal=sshKey=<private-ssh-key> <secret-name>
# after creation, annotate
kubectl annotate secret <secret-name> apps.kubermatic.k8c.io/secret-type="git"
spec:
versions:
- template:
source:
git:
path: <path-inside-git-repo>
ref:
branch: <branch>
remote: <server-url> # for ssh-key, an ssh url must be chosen (e.g. git@example.com/repo.git)
credentials:
method: <password || token || ssh-key>
# user-pass
username:
key: user
name: <secret-name>
password:
key: pass
name: <secret-name>
# token
token:
key: token
name: <secret-name>
# ssh-key
sshKey:
key: sshKey
name: <secret-name>
Be aware that all authentication methods may be available on your git server. More and more servers disable the authentication with username and password.
More over on some providers like GitHub, to authenticate with an access token, you must use password method instead of token.
Example of secret to authenticate with GitHub access token:
kubectl create secret -n <kkp-install-namespace> generic --from-literal=pass=<github-access-token> --from-literal=user=<username> <secret-name>
For other providers, please refer to their official documentation.
Helm OCI registries are being accessed using a JSON configuration similar to the ~/.docker/config.json on the local machine. It should be noted, that all OCI server urls need to be prefixed with oci://.
# inside KKP master
kubectl create secret -n <kkp-install-namespace> docker-registry --docker-server=<server> --docker-username=<user> --docker-password=<password> <secret-name>
kubectl annotate secret <secret-name> apps.kubermatic.k8c.io/secret-type="helm"
# example
kubectl create secret -n kubermatic docker-registry --docker-server=harbor.example.com/my-project --docker-username=someuser --docker-password=somepaswword oci-cred
kubectl annotate secret oci-cred apps.kubermatic.k8c.io/secret-type="helm"
spec:
versions:
- template:
source:
helm:
chartName: examplechart
chartVersion: 0.1.0
credentials:
registryConfigFile:
key: .dockerconfigjson # `kubectl create secret docker-registry` stores by default the creds under this key
name: <secret-name>
url: <server>
To use KKP Applications with a helm userpass auth registry, you can configure the following:
# inside KKP master
kubectl create secret -n <kkp-install-namespace> generic --from-literal=pass=<password> --from-literal=user=<username> <secret-name>
kubectl annotate secret <secret-name> apps.kubermatic.k8c.io/secret-type="helm"
spec:
versions:
- template:
source:
helm:
chartName: examplechart
chartVersion: 0.1.0
credentials:
password:
key: pass
name: <secret-name>
username:
key: user
name: <secret-name>
url: <server>
There is a particular case where credentials may be needed at the templating stage to render the manifests. For example, if the template method is helm and the source is git. To install the chart into the user cluster, we have to build the chart dependencies.
These dependencies may be hosted on a private registry requiring authentication.
You can specify the templating credentials by settings .spec.version[].template.templateCredentials. It works the same way as source credentials.
Example of template credentials:
spec:
versions:
- template:
source:
git:
path: <path-inside-git-repo>
ref:
branch: <branch>
remote: <server-url> # for ssh-key, an ssh url must be chosen (e.g. git@example.com/repo.git)
templateCredentials:
helmCredentials: # this the same struct as .spec.versions[].template.source.helm.credentials
password:
key: pass
name: <secret-name>
username:
key: user
name: <secret-name>
The .spec.defaultValuesBlock field describes overrides for manifest-rendering in UI when creating an application. For example if the method is Helm, then this field contains the Helm values.
Example for helm values
spec:
defaultValuesBlock: |
key1:
nestedkey: value 1
# comments are persisted and will be displayed in the UI as well
key2: value 2
You can tune how the application will be installed by setting .spec.defaultDeployOptions.
The options depend on the template method (i.e. .spec.method).
note: defaultDeployOptions can be overridden at ApplicationInstallation level by settings .spec.deployOptions
You may tune how Helm deploys the application with the following options:
atomic: corresponds to the --atomic flag on Helm CLI. If set, the installation process deletes the installation on failure; the upgrade process rolls back changes made in case of failed upgrade.wait: corresponds to the --wait flag on Helm CLI. If set, will wait until all Pods, PVCs, Services, and minimum number of Pods of a Deployment, StatefulSet, or ReplicaSet are in a ready state before marking the release as successful. It will wait for as long as --timeouttimeout: corresponds to the --timeout flag on Helm CLI. It’s time to wait for any individual Kubernetes operation.enableDNS: corresponds to the -enable-dns flag on Helm CLI. It enables DNS lookups when rendering templates. if you enable this flag, you have to verify that helm template function ‘getHostByName’ is not being used in a chart to disclose any information you do not want to be passed to DNS servers.(c.f. CVE-2023-25165)Example:
apiVersion: apps.kubermatic.k8c.io/v1
kind: ApplicationDefinition
metadata:
name: apache
spec:
defaultDeployOptions:
helm:
atomic: true
wait: true
timeout: "5m"
note: if atomic is true, then wait must be true. If wait is true then timeout must be defined.
The following is an example of ApplicationDefinition, showing all the possible options.
apiVersion: apps.kubermatic.k8c.io/v1
kind: ApplicationDefinitions
metadata:
name: <<appdef-name>>
spec:
# Description of the application. what is its purpose
description: ""
# Method used to install the application
method: helm
# Available version for this application
versions:
- # Template defines how application is installed (source provenance, Method...)
template:
# Defined how the source of the application (e.g Helm chart) is retrieved.
# Exactly one type of source must be defined.
source:
# Install application from a Git repository
git:
# Credentials are optional and holds the git credentials
credentials:
# Authentication method. Either password or token or ssh-key.
# If method is password then username and password must be defined.
# If method is token then token must be defined.
# If method is ssh-key then ssh-key must be defined.
method: password || token || ssh-key
# Password holds the ref and key in the secret for the Password credential.
# The Secret must exist in the namespace where KKP is installed (default is "kubermatic").
# The Secret must be annotated with `apps.kubermatic.k8c.io/secret-type:` set to "helm" or "git".
password:
# The key of the secret to select from. Must be a valid secret key.
key: pass
# Name of the referent.
# This field is effectively required, but due to backwards compatibility is
# allowed to be empty. Instances of this type with an empty value here are
# almost certainly wrong.
# More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
name: <<secret-name>>
# Specify whether the Secret or its key must be defined
optional: false
# SSHKey holds the ref and key in the secret for the SshKey credential.
# The Secret must exist in the namespace where KKP is installed (default is "kubermatic").
# The Secret must be annotated with `apps.kubermatic.k8c.io/secret-type:` set to "helm" or "git".
sshKey:
# The key of the secret to select from. Must be a valid secret key.
key: private-key
# Name of the referent.
# This field is effectively required, but due to backwards compatibility is
# allowed to be empty. Instances of this type with an empty value here are
# almost certainly wrong.
# More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
name: <<secret-name>>
# Specify whether the Secret or its key must be defined
optional: false
# Token holds the ref and key in the secret for the token credential.
# The Secret must exist in the namespace where KKP is installed (default is "kubermatic").
# The Secret must be annotated with `apps.kubermatic.k8c.io/secret-type:` set to "helm" or "git".
token:
# The key of the secret to select from. Must be a valid secret key.
key: token
# Name of the referent.
# This field is effectively required, but due to backwards compatibility is
# allowed to be empty. Instances of this type with an empty value here are
# almost certainly wrong.
# More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
name: <<secret-name>>
# Specify whether the Secret or its key must be defined
optional: false
# Username holds the ref and key in the secret for the username credential.
# The Secret must exist in the namespace where KKP is installed (default is "kubermatic").
# The Secret must be annotated with `apps.kubermatic.k8c.io/secret-type:` set to "helm" or "git".
username:
# The key of the secret to select from. Must be a valid secret key.
key: user
# Name of the referent.
# This field is effectively required, but due to backwards compatibility is
# allowed to be empty. Instances of this type with an empty value here are
# almost certainly wrong.
# More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
name: <<secret-name>>
# Specify whether the Secret or its key must be defined
optional: false
# Path of the "source" in the repository. default is repository root
path: charts/apache
# Git reference to checkout.
# For large repositories, we recommend to either use Tag, Branch or Branch+Commit.
# This allows a shallow clone, which dramatically speeds up performance
ref:
# Branch to checkout. Only the last commit of the branch will be checkout in order to reduce the amount of data to download.
branch: main
# Commit SHA in a Branch to checkout.
# It must be used in conjunction with branch field.
commit: 8061ceb738db42fe82b4c305b7aa5459d926d03e
# Tag to check out.
# It can not be used in conjunction with commit or branch.
tag: v1.2.3
# URL to the repository. Can be HTTP(s) (e.g. https://example.com/myrepo) or
# SSH (e.g. git://example.com[:port]/path/to/repo.git/).
remote: https://git.example.com/repo || git@example.com/repo
# Install Application from a Helm repository
helm:
# Name of the Chart.
chartName: my-app
# Version of the Chart.
chartVersion: v13.9.0
# Credentials are optional and hold the ref to the secret with Helm credentials.
# Either username / password or registryConfigFile can be defined.
credentials:
# Password holds the ref and key in the secret for the password credential.
# The Secret must exist in the namespace where KKP is installed (default is "kubermatic").
# The Secret must be annotated with `apps.kubermatic.k8c.io/secret-type:` set to "helm" or "git"
password:
# The key of the secret to select from. Must be a valid secret key.
key: pass
# Name of the referent.
# This field is effectively required, but due to backwards compatibility is
# allowed to be empty. Instances of this type with an empty value here are
# almost certainly wrong.
# More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
name: <<secret-name>>
# Specify whether the Secret or its key must be defined
optional: false
# RegistryConfigFile holds the ref and key in the secret for the registry credential file.
# The value is dockercfg file that follows the same format rules as ~/.docker/config.json.
# The Secret must exist in the namespace where KKP is installed (default is "kubermatic").
# The Secret must be annotated with `apps.kubermatic.k8c.io/secret-type:` set to "helm" or "git"
registryConfigFile:
# The key of the secret to select from. Must be a valid secret key.
key: .dockerconfigjson
# Name of the referent.
# This field is effectively required, but due to backwards compatibility is
# allowed to be empty. Instances of this type with an empty value here are
# almost certainly wrong.
# More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
name: <<secret-name>>
# Specify whether the Secret or its key must be defined
optional: false
# Username holds the ref and key in the secret for the username credential.
# The Secret must exist in the namespace where KKP is installed (default is "kubermatic").
# The Secret must be annotated with `apps.kubermatic.k8c.io/secret-type:` set to "helm" or "git"
username:
# The key of the secret to select from. Must be a valid secret key.
key: user
# Name of the referent.
# This field is effectively required, but due to backwards compatibility is
# allowed to be empty. Instances of this type with an empty value here are
# almost certainly wrong.
# More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
name: <<secret-name>>
# Specify whether the Secret or its key must be defined
optional: false
# URL of the Helm repository the following schemes are supported:
# * http://example.com/myrepo (HTTP)
# * https://example.com/myrepo (HTTPS)
# * oci://example.com:5000/myrepo (OCI, HTTPS by default, use plainHTTP to enable unencrypted HTTP)
url: https://charts.example.com || oci://localhost:5000/myrepo
# Version of the application (e.g. v1.2.3)
version: v1.2.3