Kubermatic Kubernetes Platform is transitioning from the nginx-ingress-controller to the Gateway API with Envoy Gateway. This document explains what this means for your deployment, why we are making this change, and how to migrate when you are ready.
This migration affects how traffic reaches the Kubermatic dashboard and API. The external behavior remains the same, but the underlying implementation changes to use modern Kubernetes standards.
The current implementation uses nginx-ingress-controller, which has been the standard way to expose HTTP services in Kubernetes for several years.
However, the Kubernetes project has announced the retirement of the community-maintained Ingress NGINX controller, with end of maintenance scheduled for March 2026.
When you migrate, instead of creating Ingress resources, Kubermatic will create Gateway and HTTPRoute resources. These are part of the official Gateway API specification from the Kubernetes community. Envoy Gateway serves as the implementation that processes these resources and actually handles the traffic.
From your perspective as a user, nothing changes in how you access Kubermatic. The same domain names continue to work and traffic flows to the same services. The difference is entirely in the cluster infrastructure.
BEFORE migration:
nginx-ingress-controller Helm release installednginx-ingress-controller → Kubermatic servicesAFTER migration (before uninstalling nginx):
envoy-gateway-controller Helm release installednginx-ingress-controller Helm release STILL INSTALLEDenvoy-gateway-controller → Kubermatic servicesAFTER migration (after uninstalling nginx):
envoy-gateway-controller Helm release installedenvoy-gateway-controller → Kubermatic servicesThe nginx-ingress-controller pods remain running after migration but have no effect because the Ingress resource is deleted.
Kubermatic is not going to delete Ingress LoadBalancer service from the cluster, instead it will only delete Ingress resources during migration.
The migration is designed to be explicit and safe. There is no automatic switching that could surprise you during an upgrade. Instead, you make a conscious decision to migrate by setting a flag during deployment.
The migration is controlled by two parameters: a Helm value called migrateGatewayAPI and the --migrate-gateway-api flag in kubermatic-installer.
By default, a Helm Chart configuration migrateGatewayAPI is set to false in KKP v2.30, meaning your cluster continues using nginx-ingress-controller as it always has.
When you are ready to migrate, you set this to true and redeploy Kubermatic via kubermatic-installer, by providing --migrate-gateway-api flag.
Note that if
kubermatic-installeris not being used for Kubermatic installation and upgrade, you need to manually provide-enable-gateway-apiflag tokubermatic-operator.
When you enable the migration, the following sequence occurs:
kubermatic-installer deploys the Envoy Gateway controller and its Custom Resource Definitions using Helm.
kubermatic-operator is deployed with the -enable-gateway-api startup flag, which allows kubermatic-operator to manage Gateway and HTTPRoute resources instead of Ingress resources.
kubermatic-operator creates Gateway and HTTPRoute resources. Envoy Proxy begins handling traffic.
kubermatic-installer verifies that the Gateway is fully operational:
Once the Gateway is ready, the kubermatic-installer removes Ingress resources to prevent conflicts between the old Ingress and the new Gateway resources:
kubermatic/kubermatic)dex/dex) if it existsIMPORTANT: The installer does NOT uninstall the nginx-ingress-controller Helm release.
After migration, both nginx-ingress-controller and envoy-gateway-controller will be running in your cluster.
You must manually uninstall nginx-ingress-controller when you are ready. Only Ingress resources (kubermatic/kubermatic and dex/dex) are deleted.
The kubermatic-operator runs in one of two modes depending on the startup flag.
If the flag (-enable-gateway-api) is not set, it creates and manages Ingress resources. If the flag is set, it creates and manages Gateway and HTTPRoute resources instead.
Importantly, the operator does not clean up resources from the other mode. This is a safety feature. If the operator automatically deleted Gateway resources when running in Ingress mode, or vice versa, it could break cluster access unexpectedly. Instead, we handle cleanup explicitly during the migration process.
Also, when kubermatic-operator starts with the Gateway API flag enabled, it checks that the Gateway API Custom Resource Definitions exist in the cluster. If they do not exist, the operator refuses to start and provides a clear error message. This fail-fast behavior prevents the operator from starting in a broken state where it cannot function properly.
When you migrate with --migrate-gateway-api, the kubermatic-installer deploys Envoy Gateway but does NOT remove nginx-ingress-controller. Both controllers will run in your cluster simultaneously.
This is intentional. The installer removes the Kubermatic and Dex Ingress resources to prevent routing conflicts. The nginx-ingress-controller pods continue running, but they have no Ingress resources to process (except possibly non-KKP Ingress resources you may have).
After you verify the migration is successful, you should manually uninstall nginx-ingress-controller to free up cluster resources:
helm uninstall nginx-ingress-controller -n nginx-ingress-controller
The kubermatic-installer handles cleanup during the migration process.
Please note that
kubermatic-operatordoes not automatically delete resources from the other mode at runtime. The deletion of old resources are being handled bykubermatic-installerat deployment time. If the installar is not used to manage Kubermatic operations, old resources need to be deleted manually.
By default, the installer performs the following cleanup steps:
Verifies Gateway readiness: When migrating to Gateway API, the installer waits up to 5 minutes for the Gateway to become fully operational before removing old resources. This verification includes:
Deletes Ingress resources: When migrating to Gateway API, the installer deletes the following Ingress resources:
kubermatic/kubermatic)dex/dex)This cleanup prevents routing conflicts between old and new resources.
You can control this behavior with the --skip-ingress-cleanup flag.
When set, the installer will not try to delete old resources, allowing you to manually verify the migration before cleanup.
Important: If you use --skip-ingress-cleanup, both Kubermatic and Dex Ingress resources will remain. You must manually delete them afterward:
kubectl delete ingress -n kubermatic kubermatic
kubectl delete ingress -n dex dex
To migrate from nginx-ingress-controller to Gateway API:
migrateGatewayAPI: true
# Dex configurations for Gateway API mode
# When migrateGatewayAPI is true, ingress must be disabled
# and httpRoute must be configured with your domain
dex:
ingress:
enabled: false # MUST be false when using Gateway API
httpRoute:
gatewayName: kubermatic
gatewayNamespace: kubermatic
domain: "kkp.example.com" # Replace with your domain
timeout: 3600s
# cert-manager helm chart needs to be configured to work
# with Gateway API resources.
# Note: this is only needed if cert-manager is being used.
cert-manager:
config:
apiVersion: controller.config.cert-manager.io/v1alpha1
kind: ControllerConfiguration
enableGatewayAPI: true
If cert-manager is being used to provision certificates, a new feature flag is needed to allow
cert-manager to provision certificates.
In KubermaticConfiguration, set HTTPRouteGatewaySync feature gate to true, as follows:
The following feature gate is only needed if cert-manager is in use in order to manage TLS certificates. Please check Automatic Certificate Provisioning section for details about cert-manager migration.
apiVersion: kubermatic.k8c.io/v1
kind: KubermaticConfiguration
metadata:
name: kubermatic
namespace: kubermatic
spec:
# simplified for brevity
#
# *add* the following feature gate to KubermaticConfiguration
featureGates:
HTTPRouteGatewaySync: true
In KubermaticConfiguration, under the ingress section, there is a gateway subsection where you can specify the GatewayClass to use.
The default value is kubermatic-envoy-gateway, which corresponds to the GatewayClass installed by the Envoy Gateway chart.
Most installations will not need to change this setting. However, if you have a custom GatewayClass or want to use a different implementation of Gateway API, you can specify it here.
kubermatic-installer deploy kubermatic-master --migrate-gateway-api [other options]
Conditional step, by default, kubermatic-installer automatically deletes the old Ingress resource after migration to prevent conflicts.
If you want to skip this automatic cleanup (for example, to manually verify before cleanup), use the --skip-ingress-cleanup flag:
kubermatic-installer deploy kubermatic-master --migrate-gateway-api --skip-ingress-cleanup [other options]
When cleanup is skipped, both Ingress and Gateway resources will coexist. You can manually delete the Ingress resources to prevent routing conflicts:
# beforehand, please verify the migration at Step 3
kubectl delete ingress -n kubermatic kubermatic
kubectl delete ingress -n dex dex
kubectl get gateway -n kubermatic -o yaml
kubectl get httproute -n kubermatic -o yaml
The Gateway resource should show the following status indicators:
true in status.conditions (Gateway is fully configured by Envoy)Programmed: true in its conditionsThe HTTPRoute should display:
true in status.parents[].conditions (route is accepted by the Gateway)These confirm that the Gateway is operational and actively routing traffic through the defined HTTPRoutes.
nginx-ingress-controller (optional but recommended)Please note that both nginx-ingress-controller and envoy-gateway-controller are running - so, if nginx-ingress-controller is not being used, consider uninstalling it.
helm uninstall nginx-ingress-controller -n nginx-ingress-controller
When using Gateway API with cert-manager, you can enable automatic certificate provisioning for KKP components such as Grafana, Dex, and Prometheus. This is handled by the HTTPRoute-Gateway sync controller, which watches HTTPRoutes and dynamically configures HTTPS listeners on the Gateway with explicit hostnames.
This controller is a workaround for a current limitation in cert-manager. cert-manager requires Gateway listeners to have explicit hostnames to automatically create Certificate resources. However, KKP uses a shared Gateway model where multiple HTTPRoutes from different namespaces attach to a single Gateway without explicit listener hostnames.
For more details on this constraint, see the upstream cert-manager issue: cert-manager/cert-manager#7473.
Once this issue is resolved by the cert-manager project, this controller will no longer be necessary; thus, this feature gate will be no-op once the upstream fix is released and stable.
cert-manager requires Gateway listeners to have explicit hostnames to automatically create Certificate resources. KKP uses a shared Gateway model where multiple HTTPRoutes from different namespaces attach to a single Gateway. The HTTPRoute-Gateway sync controller bridges this gap by:
monitoring, mla)This allows cert-manager to detect the listeners and automatically issue certificates for each hostname.
The controller uses a deterministic naming convention for certificates:
<namespace>-<httproute-name>grafana-iap in the mla namespace creates a secret named mla-grafana-iapWhen multiple HTTPRoutes share the same hostname, the first HTTPRoute (sorted by namespace, then name) determines the certificate name.
Dynamic HTTPS listeners are named based on the hostname:
grafana-lab-kubermatic-io for grafana.lab.kubermatic.io)w- (e.g., w-example-com for *.example.com)The controller only processes the Gateway if it contains cert-manager annotations.
metadata:
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
# or
cert-manager.io/issuer: my-issuer
http and https listeners)http and https listeners and only adds dynamic HTTPS listenersThe Envoy Gateway Helm chart includes several traffic policy resources that control how traffic is handled. These policies replace the nginx annotation-based configuration.
The ClientTrafficPolicy controls connection buffer limits. This is set to 256 kilobytes by default, which matches the previous nginx configuration and ensures large headers from LDAP or SAML authentication work properly.
The BackendTrafficPolicy controls request size limits. This is set to 100 megabytes by default. While nginx was configured with unlimited request size, Envoy Gateway requires a specific limit. The 100 megabyte value should be sufficient for most use cases.
If you need to adjust these values, you can modify them in the envoy-gateway-controller Helm chart values before installation.
The migration affects the following Ingress resources during migration:
kubermatic/kubermatic): Migrated to Gateway and HTTPRoute resourcesdex/dex): Deleted during migration to prevent conflictsComponents that use HTTPRoute resources (such as MLA monitoring components and IAP components) can leverage the HTTPRoute-Gateway sync controller for automatic certificate provisioning when the HTTPRouteGatewaySync feature gate is enabled.
These components create HTTPRoute resources that attach to the shared Gateway, and the sync controller ensures the Gateway has the necessary HTTPS listeners for cert-manager.
Note: Dex authentication is automatically migrated when you enable Gateway API mode.
The Dex Helm chart creates an HTTPRoute resource instead of an Ingress resource when migrateGatewayAPI: true is set in the Helm values.
This HTTPRoute references the Kubermatic Gateway, allowing Dex to remain accessible through the same domain.
Ensure your Dex deployment includes the following configuration:
migrateGatewayAPI: true in Dex Helm valuesingress.enabled: false (to avoid creating a conflicting Ingress)httpRoute.gatewayName and httpRoute.gatewayNamespace correctly reference the Kubermatic GatewayTo rollback to nginx-ingress-controller:
# kubermatic-operator/values.yaml
migrateGatewayAPI: false
# for dex, also enable ingress
dex:
ingress:
enabled: true
kubermatic-installer deploy kubermatic-master [other options]
This will deploy nginx-ingress-controller (if it was uninstalled) and the operator will recreate the Ingress resource.
Note: Uninstall envoy-gateway-controller if you no longer need it:
helm uninstall envoy-gateway-controller -n envoy-gateway-controller
If you encounter issues after migration, the first place to check is the Gateway resource itself. Use kubectl get gateway -n kubermatic to see the status.
A healthy Gateway should show as programmed and accepted.
If it shows as not accepted or has error conditions, check the Envoy Gateway logs for more details.
The HTTPRoute resource should also be checked to ensure it is properly attached to the Gateway.
Use kubectl get httproute -n kubermatic to verify its status.
If the Gateway shows as not ready, verify that the GatewayClass resource exists and that the Envoy Gateway controller pods are running.
The GatewayClass should be named kubermatic-envoy-gateway and should exist in the envoy-gateway-controller namespace.
If the migration times out or the Gateway appears stuck, verify each of the four readiness conditions individually:
kubectl get gateway -n kubermatic -o jsonpath='{.status.addresses}'
Should return at least one IP address or hostname. If empty, your LoadBalancer may not have been provisioned yet (check with your cloud provider).
kubectl get gateway -n kubermatic -o jsonpath='{.status.conditions[?(@.type=="Programmed")].status}'
Should return True. If False, review the Envoy Gateway controller logs for configuration errors.
kubectl get gateway -n kubermatic -o jsonpath='{.status.listeners[*].conditions[?(@.type=="Programmed")].status}'
All listeners should return True. Listeners may fail to program if there are port conflicts or TLS certificate issues.
kubectl get gateway -n kubermatic -o jsonpath='{.status.listeners[*].attachedRoutes}'
Each listener should show at least 1 attached route. If zero, verify that HTTPRoute resources exist and have correct parent references to the Gateway.
If you experience issues with large headers, such as those from LDAP or SAML authentication, verify that the ClientTrafficPolicy exists and has the correct buffer limit set. This policy should be in the envoy-gateway-controller namespace and should have a buffer limit of 256 kilobytes.
For routes that are not working as expected, verify that the Kubermatic namespace has the required label. The namespace should be labeled with kubermatic.io/gateway-access: true.
This label allows HTTPRoutes in that namespace to attach to the Gateway.
You should also verify that the backend services exist and are healthy. The kubermatic-api and kubermatic-dashboard services should both be present in the Kubermatic namespace.
If automatic certificate provisioning is not working, check the following:
kubectl get deployment -n kubermatic kubermatic-master-controller-manager -o jsonpath='{.spec.template.spec.containers[0].args}' | grep HTTPRouteGatewaySync
Should show --feature-gates=HTTPRouteGatewaySync=true.
kubectl get deployment -n kubermatic kubermatic-master-controller-manager -o jsonpath='{.spec.template.spec.containers[0].args}' | grep httproute-watch-namespaces
Verify that the namespaces containing your HTTPRoutes are in the list.
kubectl get gateway -n kubermatic -o jsonpath='{.metadata.annotations}'
Should show either cert-manager.io/cluster-issuer or cert-manager.io/issuer.
kubectl get httproute -n <namespace> <name> -o yaml
Verify that spec.parentRefs correctly references the Kubermatic Gateway.
kubectl logs -n kubermatic -l app.kubermatic.io/component=master-controller-manager | grep -i kkp-httproute-gateway-sync
After migration, you can use these commands to verify your installation. To check all Gateway API resources, run kubectl get gateway,httproute,gatewayclass -n kubermatic. This should show one Gateway, one HTTPRoute, and the GatewayClass.
To check the Envoy Gateway control plane, run kubectl get pods -n envoy-gateway-controller. This should show the Envoy Gateway controller pods running.
To check the Envoy data plane pods, run kubectl get pods -n envoy-gateway-controller -l app.kubernetes.io/name=envoy. These are the pods that actually handle the traffic.
To check the traffic policies, run kubectl get clienttrafficpolicy,backendtrafficpolicy -n envoy-gateway-controller. This should show the policies that control connection and request behavior.