The Prometheus version included in Kubermatic Kubernetes Platform (KKP) 2.15 now enables WAL compression by default; our Helm chart follows this
recommendation. If the compression needs to stay disabled, the key
prometheus.tsdb.compressWAL can be set to
when upgrading the Helm chart.
In previous KKP releases, Helm was responsible for installing the CRDs for cert-manager and Velero. While this made the deployment rather simple, it lead to problems in keeping the CRDs up-to-date (as Helm never updates or deletes CRDs).
For this reason the CRD handling in KKP 2.15 was changed to require users to always manually install CRDs before installing/updating a Helm chart. This provides much greater control over the CRD lifecycle and eases integration with other deployment mechanisms.
Upgrading existing Helm releases in a cluster is simple, as Helm does not delete CRDs. To update cert-manager, simply install the CRDs and then run Helm as usual:
kubectl apply -f charts/cert-manager/crd/ helm --tiller-namespace kubermatic upgrade --install --values YOUR_VALUES_YAML_HERE --namespace cert-manager cert-manager charts/cert-manager/
Older versions of
kubectl can potentially have issues applying the CRD manifests. If you encounter problems, please update
kubectl to the latest stable version and refer to the
for more information.
Note that on the first
kubectl apply you will receive warnings because now “kubectl takes control over previously
Helm-owned resources”, which can be safely ignored.
Perform the same steps for Velero:
kubectl apply -f charts/backup/velero/crd/ helm --tiller-namespace kubermatic upgrade --install --values YOUR_VALUES_YAML_HERE --namespace velero velero charts/backup/velero/
The labelling for the Promtail DaemonSet has changed, requiring administrators to re-install the Helm chart. As a clean upgrade is not possible, we advise to delete and re-install the chart.
helm --tiller-namespace kubermatic delete promtail helm --tiller-namespace kubermatic upgrade --install --values YOUR_VALUES_YAML_HERE --namespace logging promtail charts/logging/promtail/
Promtail pods are stateless, so no data is lost during this migration.
To prevent insecure misconfigurations, the default credentials for Grafana and Minio have been removed. They must be
set explicitly when installing the charts. Additionally, the base64 encoding for Grafana credentials has been removed,
so the plaintext values are put into the Helm
When upgrading the charts, make sure your
values.yaml contains at least these keys:
grafana: # Remember to un-base64-encode the username if you have set a custom value. user: admin # generate random password, keep it plaintext as well password: ExamplePassword minio: credentials: accessKey: # generate a random, alphanumeric 32 byte secret secretKey: # generate a random, alphanumeric 64 byte secret
Previous KKP versions used Keycloak-Proxy for securing access to cluster services like Prometheus or Grafana. The project was then renamed to Louketo and then shortly thereafter deprecated and users are encouraged to move to OAuth2-Proxy.
KKP 2.15 therefore switches to OAuth2-Proxy, which covers most of Keycloak’s functionality but with a slightly different syntax. Please refer to the official documentation for the available settings, in addition to these changes:
iap.disocvery_urlin Helm values was renamed to
iap.oidc_issuer_url, pointing to the OIDC provider’s base URL (i.e. if you had this configured as
https://example.com/dex/.well-known/openid-configurationbefore, this must now be
passthroughoption for each IAP deployment has been removed. Instead paths that are not supposed to be secured by the proxy are now configured via
config.scopesoption for each IAP deployment is now
config.scope, a single string that must (for Dex) be space-separated.
config.resourcesmechanism for granting access based on user groups/roles has been removed. Instead the required organisations/teams are now configured via explicit config variables like
email_domainsmust be configured for each IAP deployment. In most cases it can be set to
A few examples can be found in the relevant code change in KKP.
To prevent issues with Helm re-using IAP deployment config values from a previous release, it can be helpful to purge and reinstall the chart:
helm --tiller-namespace kubermatic delete --purge iap helm --tiller-namespace kubermatic upgrade --install --values YOUR_VALUES_YAML_HERE --namespace iap iap charts/iap/
Be advised that during the re-installation the secured services (Prometheus, Grafana, …) will not be accessible.