Create a namespace kubelb for the CCM to be deployed in.
The agent expects a Secret with a kubeconf file named kubelb to access the management/load balancing cluster.
# Replace with the tenant name
TENANT_NAME=tenant-shroud
KUBELB_KUBECONFIG=$(kubectl get secret kubelb-ccm-kubeconfig -n $TENANT_NAME --template={{.data.kubelb}})
kubectl --namespace kubelb create secret generic kubelb-cluster --from-literal=kubelb="$(echo $KUBELB_KUBECONFIG | base64 -d)"
The name of secret can be overridden using .Values.kubelb.clusterSecretName, if required.
Update the tenantName in the values.yaml to a unique identifier for the tenant. This is used to identify the tenant in the manager cluster. Tenants are registered in the management cluster by the Platform Provider and the name is prefixed with tenant-. So for example, a tenant named my-tenant will be registered as tenant-my-tenant.
At this point a minimal values.yaml should look like this:
kubelb:
clusterSecretName: kubelb-cluster
tenantName: <unique-identifier-for-tenant>
Important configurations for private clusters!
If your cluster only uses internal IPs for nodes (check the ollowing example output) you would need to change the value kubelb.nodeAddressType to InternalIP
kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
node-x Ready control-plane 208d v1.29.9 10.66.99.222 <none> Ubuntu 5.15.0-121-generic containerd://1.6.33
Adjust values.yaml:
kubelb:
# -- Address type to use for routing traffic to node ports. Values are ExternalIP, InternalIP.
nodeAddressType: InternalIP
kubelb.enableGatewayAPI to true in the values.yaml. This is required otherwise due to missing CRDs, kubelb will not be able to start.At this point a minimal values.yaml should look like this:
imagePullSecrets:
- name: <imagePullSecretName>
kubelb:
clusterSecretName: kubelb-cluster
tenantName: <unique-identifier-for-tenant>
helm pull oci://quay.io/kubermatic/helm-charts/kubelb-ccm-ee --version=v1.1.2 --untardir "." --untar
## Create and update values.yaml with the required values.
helm upgrade --install kubelb-ccm kubelb-ccm-ee --namespace kubelb -f kubelb-ccm-ee/values.yaml
| Key | Type | Default | Description |
|---|---|---|---|
| affinity | object | {} | |
| autoscaling.enabled | bool | false | |
| autoscaling.maxReplicas | int | 10 | |
| autoscaling.minReplicas | int | 1 | |
| autoscaling.targetCPUUtilizationPercentage | int | 80 | |
| autoscaling.targetMemoryUtilizationPercentage | int | 80 | |
| extraVolumeMounts | list | [] | |
| extraVolumes | list | [] | |
| fullnameOverride | string | "" | |
| image.pullPolicy | string | "IfNotPresent" | |
| image.repository | string | "quay.io/kubermatic/kubelb-ccm-ee" | |
| image.tag | string | "v1.1.2" | |
| imagePullSecrets[0].name | string | "kubermatic-quay.io" | |
| kubelb.clusterSecretName | string | "kubelb-cluster" | Name of the secret that contains kubeconfig for the loadbalancer cluster |
| kubelb.disableGRPCRouteController | bool | false | disableGRPCRouteController specifies whether to disable the GRPCRoute Controller. |
| kubelb.enableGatewayAPI | bool | false | enableGatewayAPI specifies whether to enable the Gateway API and Gateway Controllers. By default Gateway API is disabled since without Gateway APIs installed the controller cannot start. |
| kubelb.disableGatewayController | bool | false | disableGatewayController specifies whether to disable the Gateway Controller. |
| kubelb.disableHTTPRouteController | bool | false | disableHTTPRouteController specifies whether to disable the HTTPRoute Controller. |
| kubelb.disableIngressController | bool | false | disableIngressController specifies whether to disable the Ingress Controller. |
| kubelb.disableTCPRouteController | bool | false | disableTCPRouteController specifies whether to disable the TCPRoute Controller. |
| kubelb.disableTLSRouteController | bool | false | disableTLSRouteController specifies whether to disable the TLSRoute Controller. |
| kubelb.disableUDPRouteController | bool | false | disableUDPRouteController specifies whether to disable the UDPRoute Controller. |
| kubelb.enableLeaderElection | bool | true | Enable the leader election. |
| kubelb.enableSecretSynchronizer | bool | false | Enable to automatically convert Secrets labelled with kubelb.k8c.io/managed-by: kubelb to Sync Secrets. This is used to sync secrets from tenants to the LB cluster in a controlled and secure way. |
| kubelb.nodeAddressType | string | "ExternalIP" | Address type to use for routing traffic to node ports. Values are ExternalIP, InternalIP. |
| kubelb.tenantName | string | nil | Name of the tenant, must be unique against a load balancer cluster. |
| kubelb.useGatewayClass | bool | true | useGatewayClass specifies whether to target resources with kubelb gateway class or all resources. |
| kubelb.useIngressClass | bool | true | useIngressClass specifies whether to target resources with kubelb ingress class or all resources. |
| kubelb.useLoadBalancerClass | bool | false | useLoadBalancerClass specifies whether to target services of type LoadBalancer with kubelb load balancer class or all services of type LoadBalancer. |
| nameOverride | string | "" | |
| nodeSelector | object | {} | |
| podAnnotations | object | {} | |
| podLabels | object | {} | |
| podSecurityContext.runAsNonRoot | bool | true | |
| podSecurityContext.seccompProfile.type | string | "RuntimeDefault" | |
| rbac.allowLeaderElectionRole | bool | true | |
| rbac.allowMetricsReaderRole | bool | true | |
| rbac.allowProxyRole | bool | true | |
| rbac.enabled | bool | true | |
| replicaCount | int | 1 | |
| resources.limits.cpu | string | "500m" | |
| resources.limits.memory | string | "512Mi" | |
| resources.requests.cpu | string | "100m" | |
| resources.requests.memory | string | "128Mi" | |
| securityContext.allowPrivilegeEscalation | bool | false | |
| securityContext.capabilities.drop[0] | string | "ALL" | |
| securityContext.runAsUser | int | 65532 | |
| service.port | int | 8443 | |
| service.protocol | string | "TCP" | |
| service.type | string | "ClusterIP" | |
| serviceAccount.annotations | object | {} | |
| serviceAccount.create | bool | true | |
| serviceAccount.name | string | "" | |
| serviceMonitor.enabled | bool | false | |
| tolerations | list | [] |
helm pull oci://quay.io/kubermatic/helm-charts/kubelb-ccm --version=v1.1.2 --untardir "." --untar
## Create and update values.yaml with the required values.
helm upgrade --install kubelb-ccm kubelb-ccm --namespace kubelb -f kubelb-ccm/values.yaml
| Key | Type | Default | Description |
|---|---|---|---|
| affinity | object | {} | |
| autoscaling.enabled | bool | false | |
| autoscaling.maxReplicas | int | 10 | |
| autoscaling.minReplicas | int | 1 | |
| autoscaling.targetCPUUtilizationPercentage | int | 80 | |
| autoscaling.targetMemoryUtilizationPercentage | int | 80 | |
| extraVolumeMounts | list | [] | |
| extraVolumes | list | [] | |
| fullnameOverride | string | "" | |
| image.pullPolicy | string | "IfNotPresent" | |
| image.repository | string | "quay.io/kubermatic/kubelb-ccm" | |
| image.tag | string | "v1.1.2" | |
| imagePullSecrets | list | [] | |
| kubelb.clusterSecretName | string | "kubelb-cluster" | Name of the secret that contains kubeconfig for the loadbalancer cluster |
| kubelb.disableGRPCRouteController | bool | false | disableGRPCRouteController specifies whether to disable the GRPCRoute Controller. |
| kubelb.enableGatewayAPI | bool | false | enableGatewayAPI specifies whether to enable the Gateway API and Gateway Controllers. By default Gateway API is disabled since without Gateway APIs installed the controller cannot start. |
| kubelb.disableGatewayController | bool | false | disableGatewayController specifies whether to disable the Gateway Controller. |
| kubelb.disableHTTPRouteController | bool | false | disableHTTPRouteController specifies whether to disable the HTTPRoute Controller. |
| kubelb.disableIngressController | bool | false | disableIngressController specifies whether to disable the Ingress Controller. |
| kubelb.enableLeaderElection | bool | true | Enable the leader election. |
| kubelb.enableSecretSynchronizer | bool | false | Enable to automatically convert Secrets labelled with kubelb.k8c.io/managed-by: kubelb to Sync Secrets. This is used to sync secrets from tenants to the LB cluster in a controlled and secure way. |
| kubelb.nodeAddressType | string | "ExternalIP" | Address type to use for routing traffic to node ports. Values are ExternalIP, InternalIP. |
| kubelb.tenantName | string | nil | Name of the tenant, must be unique against a load balancer cluster. |
| kubelb.useGatewayClass | bool | true | useGatewayClass specifies whether to target resources with kubelb gateway class or all resources. |
| kubelb.useIngressClass | bool | true | useIngressClass specifies whether to target resources with kubelb ingress class or all resources. |
| kubelb.useLoadBalancerClass | bool | false | useLoadBalancerClass specifies whether to target services of type LoadBalancer with kubelb load balancer class or all services of type LoadBalancer. |
| nameOverride | string | "" | |
| nodeSelector | object | {} | |
| podAnnotations | object | {} | |
| podLabels | object | {} | |
| podSecurityContext.runAsNonRoot | bool | true | |
| podSecurityContext.seccompProfile.type | string | "RuntimeDefault" | |
| rbac.allowLeaderElectionRole | bool | true | |
| rbac.allowMetricsReaderRole | bool | true | |
| rbac.allowProxyRole | bool | true | |
| rbac.enabled | bool | true | |
| replicaCount | int | 1 | |
| resources.limits.cpu | string | "500m" | |
| resources.limits.memory | string | "512Mi" | |
| resources.requests.cpu | string | "100m" | |
| resources.requests.memory | string | "128Mi" | |
| securityContext.allowPrivilegeEscalation | bool | false | |
| securityContext.capabilities.drop[0] | string | "ALL" | |
| securityContext.runAsUser | int | 65532 | |
| service.port | int | 8443 | |
| service.protocol | string | "TCP" | |
| service.type | string | "ClusterIP" | |
| serviceAccount.annotations | object | {} | |
| serviceAccount.create | bool | true | |
| serviceAccount.name | string | "" | |
| serviceMonitor.enabled | bool | false | |
| tolerations | list | [] |
At this point, the KubeLB CCM should be installed and running in the tenant cluster. Next steps are to install the Gateway API CRDs in the cluster. This is required to use the Gateway API resources in the tenant cluster.
Use the Experimental channel to install the CRDs:
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.1.0/experimental-install.yaml
For more details: Experimental Install
Use the Standard channel to install the CRDs:
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.1.0/standard-install.yaml
For more details: Standard Install
Due to the following reasons this has been left as a manual step and we haven’t added these CRDs to the KubeLB Manager chart, for automated installation: