Certificate Management

Enterprise Edition

Setup

Install Cert-Manager

Install cert-manager to manage certificates for your tenants.

These are minimal examples to get you started quickly. Please refer to the documentation of cert-manager for further details and configurations.

Update values.yaml for KubeLB manager chart to enable the cert-manager addon.

kubelb-addons:
  enabled: true
  cert-manager:
    enabled: true
    crds:
      enabled: true
    config:
      apiVersion: controller.config.cert-manager.io/v1alpha1
      kind: ControllerConfiguration
      enableGatewayAPI: true

Update values.yaml for KubeLB manager chart to enable the cert-manager addon.

kubelb-addons:
  enabled: true
  cert-manager:
    enabled: true
    crds:
      enabled: true
    config:
      apiVersion: controller.config.cert-manager.io/v1alpha1
      kind: ControllerConfiguration
      enableGatewayAPI: false

Configure Tenant

Certificate management can be enabled/disabled at global or tenant level. For automation purposes, you can configure allowed domains and default issuer for the certificates at the tenant level.

apiVersion: kubelb.k8c.io/v1alpha1
kind: Tenant
metadata:
  name: shroud
spec:
  # These domains are allowed to be used for Ingress, Gateway API, DNS, and certs.
  allowedDomains:
    - "kube.example.com"
    - "*.kube.example.com"
    - "*.shroud.example.com"
  certificates:
    # can also be configured in the `Config` resource at a global level.
    # Default issuer to use if `kubelb.k8c.io/manage-certificates` annotation is added to the cluster.
    defaultClusterIssuer: "letsencrypt-staging"
    # If not empty, only the domains specified here will have automation for Certificates. Everything else will be ignored.
    allowedDomains:
    - "*.shroud.example.com"

Users can then either use cert-manager annotations or the annotation kubelb.k8c.io/manage-certificates: true on their resources to automate certificate management.

Cluster Issuer example

Due to multi-tenancy, it’s recommended to use DNS challenge for certificate management. Gateway API has a limitation and doesn’t support wildcard domains with HTTP01 challenge. Similarly, for Ingress, unless you are using single ingress installation for all tenants, you will need to create a separate ClusterIssuer for each tenant. Same is the case for Gateway API since it needs the Gateway name to resolve the certificate challenges.

Example for DNS challenge with AWS Route53

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-production-dns
spec:
  acme:
    email: user@example.com
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-production-dns
    solvers:
      - dns01:
          route53:
            region: eu-central-1
            accessKeyIDSecretRef:
              name: route53-credentials
              key: access-key-id
            secretAccessKeySecretRef:
              name: route53-credentials
              key: secret-access-key

Example for HTTP01 challenge

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-production
spec:
  acme:
    email: user@example.com
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: example-issuer-account-key
    solvers:
      - http01:
          gatewayHTTPRoute:
            parentRefs:
              - kind: Gateway
                name: default
                namespace: tenant-default
                sectionName: http

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-production
spec:
  acme:
    # You must replace this email address with your own.
    # Let's Encrypt will use this to contact you about expiring
    # certificates, and issues related to your account.
    email: user@example.com
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      # Secret resource that will be used to store the account's private key.
      name: example-issuer-account-key
    # Add a single challenge solver, HTTP01 using nginx
    solvers:
    - http01:
        ingress:
          ingressClassName: nginx

The additional validation at the tenant level allows us to use a single instance of cert-manager for multiple tenants. Multiple cert-manager installations are not recommended and it’s better to have a single instance of cert-manager for all tenants but different ClusterIssuers/Issuers for different tenants, if required.

Usage

In tenant cluster, create the following resources. Based on your requirements:

  1. Use cert-manager with known issuer:
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: example
  annotations:
    cert-manager.io/issuer: foo
spec:
  gatewayClassName: kubelb
  listeners:
    - name: http
      hostname: example.com
      port: 443
      protocol: HTTPS
      allowedRoutes:
        namespaces:
          from: All
      tls:
        mode: Terminate
        certificateRefs:
          - name: example-com-tls
  1. Leave the issuer up to the management cluster:
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: example
  annotations:
    kubelb.k8c.io/manage-certificates: true
spec:
  gatewayClassName: kubelb
  listeners:
    - name: http
      hostname: example.com
      port: 443
      protocol: HTTPS
      allowedRoutes:
        namespaces:
          from: All
      tls:
        mode: Terminate
        certificateRefs:
          - name: example-com-tls
  1. Use custom certificates:
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: example
  namespace: default
spec:
  gatewayClassName: kubelb
  listeners:
    - name: http
      hostname: example.com
      port: 443
      protocol: HTTPS
      allowedRoutes:
        namespaces:
          from: All
      tls:
        mode: Terminate
        certificateRefs:
          - name: custom-certificate
---
kind: SyncSecret
apiVersion: kubelb.k8c.io/v1alpha1
data:
  tls.crt: ZnJhbmtsYW1wYXJkCg==
  tls.key: ZnJhbmtsYW1wYXJkCg==
metadata:
  annotations:
  name: custom-certificate
  namespace: default
type: kubernetes.io/tls
---

This will then sync the secret to the management cluster in a secure way. Refer to Bring your own Certificates for more details.

For more use cases, view cert-manager documentation