Kubermatic Kubernetes Platform (KKP) Addons

KKP Addons

Addons are specific services and tools extending functionality of Kubernetes. In KKP we have a set of default addons installed on each user-cluster. The default addons are:

Canal: policy based networking for cloud native applications
Dashboard: General-purpose web UI for Kubernetes clusters
DNS: Kubernetes DNS service
kube-proxy: Kubernetes network proxy
rbac: Kubernetes Role-Based Access Control, needed for TLS node bootstrapping
OpenVPN client: virtual private network (VPN). Lets the control plan access the Pod & Service network. Required for functionality like kubectl proxy & kubectl port-forward.
node-exporter: Exports metrics from the node
default-storage-class: A cloud provider specific StorageClass
kubelet-configmap: A set of ConfigMaps used by kubeadm

Installation and configuration of these addons is done by 2 controllers which are part of the KKP controller-manager:

addon-installer-controller: Ensures a given set of addons will be installed in all clusters
addon-controller: Templates the addons & applies the manifests in the user clusters

Configuration

To configure which addons shall be installed in all user clusters, set the following settings in the values.yaml for the KKP chart:

kubermatic:
  controller:
    addons:
      kubernetes:
        defaultAddons:
        - canal
        - dashboard
        - dns
        - kube-proxy
        - openvpn
        - rbac
        - kubelet-configmap
        - default-storage-class
        - node-exporter
        image:
          repository: "quay.io/kubermatic/addons"
          tag: "v0.2.9"
          pullPolicy: "IfNotPresent"

To deploy the changes:

helm upgrade --install --wait --timeout 300 --values values.yaml --namespace kubermatic kubermatic charts/kubermatic

Setting a custom docker registry

In case you want to set a custom registry for all addons, you can specify the -overwriteRegistry flag on the kubermatic-controller-manager or via the helm setting kubermatic.controller.overwriteRegistry. It will set the specified registry on all control plane components & addons.

How to Add a Custom Addon

All manifests and config files for the default addons are stored in the quay.io/kubermatic/addons image. Use this image as a base image and copy configs and manifests for all custom addons to /addons folder.
Custom addon with manifest
```
.
├── Dockerfile
└── foo
    └── deployment.yaml
```
Dockerfile for custom addons:
```
FROM quay.io/kubermatic/addons:v0.0.1

ADD ./ /addons/
```
Release the image with custom addon
```
export TAG=v1.0
docker build -t customer/addons:${TAG} .
docker push customer/addons:${TAG}
```

Edit values.yaml you are using for the installation of KKP. Change the path to the addons repository

kubermatic:
  controller:
    addons:
      kubernetes:
        image:
          repository: "quay.io/customer/addons" # <-- add your repo here

Add your addon to the list of default addons in values.yaml:

kubermatic:
  controller:
    addons:
      kubernetes
        # list of addons to install into every user-cluster. All need to exist in the addons image
        defaultAddons:
        - foo # <-- add your addon here
        - canal
        - dashboard
        - dns
        - kube-proxy
        - openvpn
        - rbac

Update the installation of KKP

helm upgrade --install --wait --timeout 300 --values values.yaml --namespace kubermatic kubermatic charts/kubermatic

Template Variables

All cluster object variables can be used in all addon manifests. Specific template variables and functions used in default templates:

{{first .Cluster.Spec.ClusterNetwork.Pods.CIDRBlocks}}: will render an IP block of the cluster
{{.DNSClusterIP}}: will render the IP address of the DNS server
image: {{ Registry quay.io }}/some-org/some-app:v1.0: Will use quay.io as registry or the overwrite registry if specified