DNS Management

Enterprise Edition

Setup

Install external-dns

KubeLB uses external-dns to manage DNS records for the tenant clusters.

This is a minimal example. See the external-dns documentation for setup with other providers.

Update values.yaml for the KubeLB Manager chart to enable the external-dns addon.

kubelb-addons:
  enabled: true

  external-dns:
    enabled: true
    domainFilters:
      - example.com
    extraVolumes:
      - name: credentials
        secret:
          secretName: route53-credentials
    extraVolumeMounts:
      - name: credentials
        mountPath: /.aws
        readOnly: true
    env:
      - name: AWS_SHARED_CREDENTIALS_FILE
        value: /.aws/credentials
    txtOwnerId: kubelb-example-aws
    registry: txt
    provider: aws
    policy: sync
    sources:
      - service
      - ingress
      # Comment out the below resources if you are not using Gateway API.
      - gateway-httproute
      - gateway-grpcroute
      - gateway-tlsroute
      - gateway-tcproute
      - gateway-udproute

Credentials secret

apiVersion: v1
kind: Namespace
metadata:
  name: external-dns
---
apiVersion: v1
data:
  credentials: W2RlZmF1bHRdCmF3c19hY2Nlc3Nfa2V5X2lkID0gTk9UVEhBVERVTUIKYXdzX3NlY3JldF9hY2Nlc3Nfa2V5ID0gTUFZQkVJVFNBU0VDUkVU
kind: Secret
metadata:
  name: route53-credentials
  namespace: external-dns
type: Opaque

Enable DNS automation

DNS can be enabled/disabled at global or tenant level. For automation purposes, you can configure allowed domains for DNS per tenant.

apiVersion: kubelb.k8c.io/v1alpha1
kind: Tenant
metadata:
  name: shroud
spec:
  # These domains are allowed to be used for Ingress, Gateway API, DNS, and certs.
  allowedDomains:
    - "kube.example.com"
    - "*.kube.example.com"
    - "*.shroud.example.com"
  dns:
    # If not empty, only the domains specified here will have automation for DNS. Everything else will be ignored.
    allowedDomains:
    - "*.shroud.example.com"
    # The wildcard domain to use for auto-generated hostnames for Load balancers
    # In EE Edition, this is also use to generated dynamic hostnames for tunnels.
    wildcardDomain: "*.apps.example.com"
    # Allow tenants to specify explicit hostnames for Load balancers and tunnels(in EE Edition)
    allowExplicitHostnames: false
  gatewayAPI:
    class: "eg"
    defaultGateway:
      name: "default"
      namespace: "kubelb"

Use external-dns annotations or the annotation kubelb.k8c.io/manage-dns: true on the resources to automate DNS management.

The additional validation at the tenant level allows a single instance of external-dns to serve multiple tenants. If required, external-dns can also be installed per tenant.

Configure Gateway

The Gateway resource must be configured for this automation to work. With Gateway API, configure it as follows:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: default
  namespace: kubelb
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-production
spec:
  gatewayClassName: eg
  listeners:
    ## HTTP listener to solve DNS challenge for cert-manager
    - name: http
      protocol: HTTP
      port: 80
      allowedRoutes:
        namespaces:
          from: All
    - protocol: HTTPS
      port: 443
      name: https
      hostname: "*.apps.example.com"
      allowedRoutes:
        namespaces:
          from: All
      tls:
        mode: Terminate
        certificateRefs:
          - kind: Secret
            name: eg-https
    # Required in EE for tunneling
    - protocol: HTTPS
      port: 443
      name: https-connection-manager
      hostname: "connection-manager.example.com"
      allowedRoutes:
        namespaces:
          from: All
      tls:
        mode: Terminate
        certificateRefs:
          - kind: Secret
            name: eg-https-connection-manager

Usage

Using external-dns annotations:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: example
  annotations:
    external-dns.alpha.kubernetes.io/hostname: example.com
spec:
  gatewayClassName: kubelb
  listeners:
    - name: http
      hostname: example.com
      port: 443
      protocol: HTTPS
      allowedRoutes:
        namespaces:
          from: All

Delegate DNS management to KubeLB:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: example
  annotations:
    kubelb.k8c.io/manage-dns: true
spec:
  gatewayClassName: kubelb
  listeners:
    - name: http
      hostname: example.com
      port: 443
      protocol: HTTPS
      allowedRoutes:
        namespaces:
          from: All

Services can also be annotated to manage DNS:

apiVersion: v1
kind: Service
metadata:
  name: backend
  annotations:
    external-dns.alpha.kubernetes.io/hostname: backend.example.com
spec:
  ports:
    - name: http
      port: 3000
      targetPort: 3000
  selector:
    app: backend
  type: LoadBalancer