KubeV Configuration File Reference

This document provides a deep-dive reference for the KubeV cluster configuration file. The configuration file drives kubermatic-virtualization apply and controls every aspect of your Kubermatic Virtualization Platform deployment.

File Format

The configuration file is a YAML document conforming to the KubeV API schema:

apiVersion: virtualization.k8c.io/v1alpha1
kind: KubeVCluster

Both fields are required and must match exactly. Use kubermatic-virtualization config print to generate a starter file, or kubermatic-virtualization config print --full for an annotated version with inline guidance.

The installer checks the whole file when it loads it, and stops before it changes anything if a field is invalid. Every field it rejects is listed in one run.

Table of Contents

1. Nodes: Control Plane & Workers
2. Network Configuration
3. API Endpoint
4. Load Balancer
5. Storage
6. KubeVirt Configuration
7. Kubelet Configuration
8. Image Pull Secret
9. Offline / Air-Gapped Configuration
10. Identity Provider (IDP)
11. Dashboard
12. cert-manager
13. VM Migration from vSphere
14. Platform Feature Gates
15. KDP Integration
16. Complete Example

Nodes: Control Plane & Workers

KubeV manages two categories of nodes. All nodes share the same HostConfig shape and are provisioned over SSH using kubeadm.

Control Plane Nodes

Control plane nodes run the Kubernetes API server, scheduler, and controller manager. You must define at least one.

controlPlane:
  hosts:
    - address: "192.168.1.10"
      sshUsername: "ubuntu"
      sshPrivateKeyFile: "/home/ops/.ssh/id_rsa"
    - address: "192.168.1.11"
      sshUsername: "ubuntu"
      sshPrivateKeyFile: "/home/ops/.ssh/id_rsa"
    - address: "192.168.1.12"
      sshUsername: "ubuntu"
      sshPrivateKeyFile: "/home/ops/.ssh/id_rsa"

High-availability guidance: Always run an odd number of control plane nodes (1, 3, or 5). etcd, which runs on every control plane node, uses Raft consensus and requires a quorum — meaning more than half of members must be reachable to elect a leader and accept writes. A 3-node control plane tolerates one node failure; a 5-node control plane tolerates two. Spread nodes across different physical hosts or failure domains.

A single control plane node is valid and practical for development, lab, or edge deployments where HA is not a requirement.

Static Worker Nodes

Worker nodes run your virtual machine workloads via KubeVirt. The staticWorkers section is optional — a control plane node can also schedule workloads in single-node or resource-constrained environments.

staticWorkers:
  hosts:
    - address: "192.168.1.20"
      sshUsername: "ubuntu"
      sshPrivateKeyFile: "/home/ops/.ssh/id_rsa"
    - address: "192.168.1.21"
      sshUsername: "ubuntu"
      sshPrivateKeyFile: "/home/ops/.ssh/id_rsa"

HostConfig Fields

Every node entry (control plane and worker alike) supports the following fields:

SSH key requirements:

• The key file must exist on the machine running kubermatic-virtualization apply and be readable by the invoking user.
• The corresponding public key must be present in ~/.ssh/authorized_keys on each target node.
• Recommended permissions: chmod 600 /path/to/key.
• Passphrase-protected keys are not supported. Use ssh-keygen -p to strip a passphrase or generate a dedicated deployment key.

Applying labels and annotations:

Labels and annotations follow kubectl label / kubectl annotate semantics. To remove a key that was previously applied, suffix it with -:

labels:
  node-role: worker
  topology.kubernetes.io/zone: us-east-1a
  stale-label-: ""        # the trailing dash removes "stale-label" from the node

Network Configuration

networkConfiguration:
  dnsServerIP: "192.168.1.1"
  networkCIDR: "10.244.0.0/16"
  serviceCIDR: "10.96.0.0/12"
  gatewayIP: "10.244.0.1"
  tunnelInterface: "eth0"

About Kube-OVN

KubeV uses Kube-OVN as its CNI network plugin. Kube-OVN is a Kubernetes networking solution built on top of Open Virtual Network (OVN), which is the control plane layer of Open vSwitch (OVS).

All CIDR ranges are validated at load time and must not overlap with one another or with your physical network.

Fields

Fixed after installation: dnsServerIP and serviceCIDR can’t be changed once the cluster is installed. On an installed cluster, apply stops before it changes anything if serviceCIDR differs from the range the cluster runs with. Clusters installed before v1.3 run 10.96.0.0/12, whatever their file said, because older installers ignored the field: set 10.96.0.0/12 there, or leave the field out.

Tunnel Interface

The tunnelInterface field tells Kube-OVN which host NIC to use as the underlay for its overlay tunnels. Each node establishes Geneve tunnels to every other node using the IP of this interface. Choosing the wrong interface (e.g., a management NIC with limited throughput) will hurt east-west VM-to-VM performance.

If left unset, Kube-OVN selects the interface that holds the node’s default route. This works well on simple single-NIC servers but can produce incorrect results on servers with multiple NICs or bonded interfaces.

Three formats are accepted:

# Single interface name — all nodes must have an interface with exactly this name
tunnelInterface: "eth0"

# Comma-separated list — Kube-OVN picks the first name that exists on each node
tunnelInterface: "eth0,ens3,enp3s0"

# Regular expression — Kube-OVN uses the first interface whose name matches
tunnelInterface: "^eth[0-9]+$"

Per-node override: In mixed hardware environments where different servers use different NIC names, set a safe regex globally and override specific nodes that need a different interface:

networkConfiguration:
  tunnelInterface: "^ens[0-9]+$"     # matches ens3, ens4, etc. on most nodes

controlPlane:
  hosts:
    - address: "192.168.1.10"
      tunnelInterface: "bond0"        # this node uses a bonded interface

staticWorkers:
  hosts:
    - address: "192.168.1.20"
      tunnelInterface: "enp6s0f0"     # explicit NIC on this node

Validation rules:

• A per-node tunnelInterface must be a single valid Linux interface name: max 15 characters, alphanumeric plus _, ., -.
• The global tunnelInterface may be a single name, comma-separated list, or a regex pattern. If the value contains any regex metacharacter (^, $, *, ., +, ?, (, ), [, ], {, }, |, \), it is treated as a regex and is not name-validated.
• networkCIDR and serviceCIDR must not overlap.
• gatewayIP must be within networkCIDR.
• Every server in dnsServerIP must be a valid IP address.

API Endpoint

apiEndpoint:
  host: "api.example.com"
  alternativeNames:
    - "192.168.1.10"
    - "api-internal.example.com"

Load Balancer

Exactly one option must be set. If the section is omitted entirely, none is applied as the default.

MetalLB

MetalLB allocates external IPs for LoadBalancer-type services on bare-metal clusters. It operates in L2 mode using ARP/NDP announcements — no BGP router is required.

loadBalancer:
  metallb:
    ipRange: "192.168.1.100-192.168.1.150"

ipRange accepts two formats:

# CIDR block — all usable addresses in the block are available
ipRange: "192.168.10.0/24"

# Inclusive range — addresses from start to end (inclusive)
ipRange: "192.168.10.50-192.168.10.100"

Planning notes:

• The IP range must be routable on your LAN. Hosts outside the cluster must be able to send traffic to these IPs and have it arrive at the node that is currently announcing the address.
• IPs must not overlap with DHCP pools or existing static assignments.
• In L2 mode, one node at a time holds each address. If that node fails, MetalLB re-announces from a healthy node within a few seconds.

None

loadBalancer:
  none: {}

Disables external load balancing. LoadBalancer-type services will remain in <pending> state. Use this when your environment provides its own load balancer, or when only ClusterIP / NodePort services are needed.

Storage

Exactly one option must be set. If the section is omitted, none is applied as the default.

Longhorn

Longhorn provides distributed block storage backed by node-local disks. It is the default storage backend for VM persistent disks in KubeV.

storage:
  longhorn: {}

No additional fields are required. Longhorn discovers and uses available block devices on each node automatically. Each PersistentVolumeClaim is backed by replicated block volumes, with the replica count configurable via Longhorn’s own settings after deployment.

None

storage:
  none: {}

Disables managed storage. You must supply your own StorageClass or provision PersistentVolume objects manually.

KubeVirt Configuration

KubeV deploys KubeVirt to enable running virtual machines as Kubernetes workloads. KubeVirt extends Kubernetes with custom resources — VirtualMachine, VirtualMachineInstance, DataVolume, and others — and runs VM processes directly on the Kubernetes nodes using the host’s KVM/QEMU stack.

Alongside KubeVirt, KubeV also deploys the Containerized Data Importer (CDI), which handles importing VM disk images from HTTP sources, container registries, or object storage into PersistentVolumeClaims that VMs can boot from.

The kubevirt section in the configuration file is optional. When omitted, KubeVirt and CDI are deployed with their built-in defaults and no feature gates enabled.

kubevirt:
  developerConfiguration:
    featureGates:
      - "LiveMigration"
      - "Snapshot"
    useEmulation: false
  migrations:
    parallelMigrationsPerCluster: 5
    allowAutoConverge: true
  commonInstancetypesDeployment:
    enabled: true
  permittedHostDevices:
    pciHostDevices:
      - pciVendorSelector: "10DE:233B"
        resourceName: "nvidia.com/gpu"
  enforceRegionalScheduling: false

Every apply writes this section to the KubeVirt CR, so a change takes effect on the next run.

Developer Configuration

Feature Gates

Feature gates enable functionality in KubeVirt that is either experimental, requires extra cluster resources, or is disabled by default for safety. Gate names must start and end with an alphanumeric character and may contain alphanumerics, dots, or hyphens.

Example enabling live migration and snapshots:

kubevirt:
  developerConfiguration:
    featureGates:
      - "LiveMigration"
      - "Snapshot"

Software Emulation

kubevirt:
  developerConfiguration:
    useEmulation: true

When useEmulation: true, KubeVirt falls back to QEMU’s software TCG emulation if the node does not expose /dev/kvm. This is useful in:

• Nested virtualization environments (e.g., VMs running inside a cloud provider) where KVM may not be available or requires explicit opt-in.
• CI/CD pipelines that run tests against VM workloads on ephemeral runners without KVM.
• Development laptops running the cluster inside a VM.

Do not use useEmulation: true in production. Software emulation is typically 10–50× slower than KVM-accelerated execution. CPU-intensive or I/O-intensive workloads inside VMs will be severely degraded.

Only turn it on for nodes without hardware virtualization. While it is on, every new VM pod starts without /dev/kvm, so VMs that already run with KVM can no longer live-migrate, and node drains during an upgrade stall on them.

Live Migration

kubevirt.migrations tunes live migration of VMs between nodes, cluster-wide. Node drains during upgrades use it too. Not to be confused with VM Migration from vSphere.

PCI Passthrough

kubevirt.permittedHostDevices.pciHostDevices lists the PCI devices VMs may get, such as GPUs.

Regional Scheduling

With enforceRegionalScheduling: true, the installer deploys the Kyverno policy enforce-vm-regional-scheduling. When a VM is created with the annotation ovn.kubernetes.io/logical_switch on its template, the policy reads the topology.kubernetes.io/region label of that Kube-OVN subnet and adds it to the VM’s node selector, so the VM runs only on nodes of that region. VMs without the annotation, or on a subnet without the label, are scheduled freely.

Kubelet Configuration

kubeletConfig sets kubelet options on every node. A node’s kubelet field (see HostConfig Fields) overrides them for that node.

kubeletConfig:
  systemReserved:
    cpu: "500m"
    memory: "1Gi"
  kubeReserved:
    cpu: "500m"
    memory: "1Gi"
  evictionHard:
    memory.available: "500Mi"
  maxPods: 110

staticWorkers:
  hosts:
    - address: "192.168.1.20"
      kubelet:
        maxPods: 250

See Reserve Compute Resources for System Daemons for how the reservations add up.

Image Pull Secret

KubeV’s own images (the controller-manager, the API server and the dashboard) are in a private registry. The installer needs credentials for it, given in one of two ways:

Option 1 — imagePullSecret in the config file (preferred for automation):

imagePullSecret: |
  {"auths":{"quay.io":{"auth":"<base64 of username:password>"}}}

Generate the base64-encoded auth value:

echo -n "myuser:mypassword" | base64

Option 2 — Environment variables before running kubermatic-virtualization apply:

export KUBEV_USERNAME=myuser
export KUBEV_PASSWORD=mypassword
kubermatic-virtualization apply -f cluster.yaml

imagePullSecret is a top-level field and applies to every platform component. If it is set, the environment variables are ignored for these images. If neither is provided, kubermatic-virtualization apply fails the pre-flight check before any cluster changes are made. Air-gapped installations with offlineSettings.enabled: true are exempt: their images come from the internal registry.

dashboard.imagePullSecret overrides it for the API server and the dashboard only. Before v1.2, the credentials lived there; when upgrading such a configuration, move the value to the top level.

Offline / Air-Gapped Configuration

offlineSettings:
  enabled: true
  containerRegistry:
    address: "https://registry.internal.example.com:5000"
    username: "registry-user"
    password: "registry-password"
    insecure: false
  helmRegistry:
    address: "https://charts.internal.example.com:5000"
    username: "helm-user"
    password: "helm-password"
    insecure: false
  packageRepository: "https://packages.internal.example.com"
  packageRepositoryKeyFingerprint: "A1B2C3D4E5F60718293A4B5C6D7E8F9012345678"

When enabled: true, KubeV will not reach out to the public internet during installation or upgrades. Every container image, Helm chart, and OS package must be pre-loaded into the internal mirrors you configure here.

Fields

Preparing an Air-Gapped Environment

The release’s offline bundle provides all of the following on one jump host — see Offline Installation. To use your own registry and repository instead, the following must be in place before running kubermatic-virtualization apply in offline mode:

1. Mirror container images — use kubermatic-virtualization mirror-images to copy all required images to your internal registry. This includes KubeV’s own images and those of Kube-OVN, cert-manager, KubeVirt, CDI, Forklift, Longhorn, MetalLB, Kyverno, Multus, and others.
2. Prepare the package repository — sync the OS package repository used by your node OS so that kubeadm, kubelet, and kubectl packages are available.
3. Test connectivity — verify that all nodes can reach every internal mirror URL before applying.

In offline mode, KubeV’s own images come from containerRegistry too, so the Image Pull Secret is not required.

Identity Provider (IDP)

Exactly one option must be set. If the section is omitted, none is applied as the default.

Important: The idp section only controls whether KubeV deploys an identity provider alongside the cluster. It does not configure the dashboard’s authentication. Deploying Dex here and enabling OIDC on the dashboard are two separate steps — both are needed to achieve SSO login. See Using OIDC with in-cluster Dex for the end-to-end wiring.

When Should You Deploy an IDP?

If you already have an OIDC-compatible identity provider — Keycloak, Okta, Azure AD, Auth0, or any other — set idp: none: {} and point dashboard.auth.oidc.issuerURL directly at it. No in-cluster IDP is needed.

If you do not have an existing OIDC provider and want to get dashboard SSO running quickly without standing up external infrastructure, KubeV can deploy Dex as a lightweight, in-cluster OIDC provider. Dex acts as a federation hub: it speaks OIDC to the dashboard while connecting upstream to your actual user database — whether that is LDAP, GitHub, Google, or a local static password list.

Dex

Dex is a federated OpenID Connect provider. It does not store users itself; instead, it delegates authentication to upstream connectors (LDAP, GitHub, OIDC providers, SAML, etc.) and issues standard OIDC tokens that downstream applications — like the KubeV dashboard — can verify.

idp:
  dex:
    issuer: "https://dex.example.com"
    enablePasswordDB: false
    connectors:
      - type: oidc
        id: google
        name: Google
        config:
          issuer: https://accounts.google.com
          clientID: "your-google-client-id"
          clientSecret: "your-google-client-secret"
          redirectURI: "https://dex.example.com/callback"
    staticClients:
      - id: kubev-dashboard
        name: KubeV Dashboard
        secret: "a-strong-client-secret"
        redirectURIs:
          - "https://dashboard.example.com/auth/callback"

Dex Fields

Connectors

Connectors tell Dex where to authenticate users. Dex supports a wide range of upstream providers through its connector framework. All connector-specific fields go inside the config key, whose schema varies by connector type.

OIDC Connector

Use this to federate to any upstream OIDC provider: Google, Azure AD (via its OIDC endpoint), Okta, Auth0, another Dex instance, etc.

connectors:
  - type: oidc
    id: google
    name: Google
    config:
      issuer: https://accounts.google.com
      clientID: "your-google-client-id"
      clientSecret: "your-google-client-secret"
      redirectURI: "https://dex.example.com/callback"
      # Optional: request additional scopes from the upstream provider
      scopes:
        - openid
        - email
        - profile
      # Optional: claim to use as the user's email address
      userNameKey: email
      # Optional: do not verify the upstream provider's TLS certificate
      insecureSkipEmailVerified: false

LDAP Connector

Use this to authenticate against a corporate directory (Active Directory, OpenLDAP, FreeIPA, etc.).

connectors:
  - type: ldap
    id: ldap
    name: Corporate Directory
    config:
      host: "ldap.corp.example.com:389"
      # Use ldaps:// port 636 for TLS, or set startTLS: true for STARTTLS on port 389
      startTLS: true
      # Service account for binding to the directory
      bindDN: "cn=kubev-svc,ou=serviceaccounts,dc=corp,dc=example,dc=com"
      bindPW: "service-account-password"
      # Optional: skip TLS verification (only for internal/trusted directories)
      insecureSkipVerify: false

      # How to search for users
      userSearch:
        baseDN: "ou=people,dc=corp,dc=example,dc=com"
        filter: "(objectClass=person)"
        username: uid          # LDAP attribute mapped to the login field
        idAttr: uid            # Attribute used as the unique user identifier
        emailAttr: mail        # Attribute mapped to the user's email
        nameAttr: displayName  # Attribute mapped to the user's display name

      # How to search for groups (optional — needed for group-based access control)
      groupSearch:
        baseDN: "ou=groups,dc=corp,dc=example,dc=com"
        filter: "(objectClass=groupOfNames)"
        userAttr: DN           # User attribute to match against group membership
        groupAttr: member      # Group attribute holding member DNs
        nameAttr: cn           # Group attribute used as the group name in tokens

GitHub Connector

Authenticate users via their GitHub account. You need to create an OAuth App in your GitHub organization settings.

connectors:
  - type: github
    id: github
    name: GitHub
    config:
      clientID: "your-github-oauth-app-client-id"
      clientSecret: "your-github-oauth-app-client-secret"
      redirectURI: "https://dex.example.com/callback"
      # Optional: restrict login to members of specific organizations
      orgs:
        - name: my-github-org
          # Optional: restrict to members of specific teams within the org
          teams:
            - platform-team
            - sre-team
      # Optional: use GitHub Enterprise instead of github.com
      # hostName: "github.corp.example.com"

Microsoft / Azure AD Connector

Authenticate users via Microsoft Entra ID (formerly Azure Active Directory) or a personal Microsoft account.

connectors:
  - type: microsoft
    id: microsoft
    name: Microsoft
    config:
      clientID: "your-azure-app-client-id"
      clientSecret: "your-azure-app-client-secret"
      redirectURI: "https://dex.example.com/callback"
      # Optional: restrict to a specific Azure AD tenant
      tenant: "your-tenant-id-or-domain"
      # Optional: include group membership in tokens (requires Group.Read.All permission)
      groups:
        - "platform-team"

Local Password Database

When no upstream connector is available, or for quick bootstrapping, enable Dex’s built-in password store:

idp:
  dex:
    issuer: "https://dex.example.com"
    enablePasswordDB: true

With enablePasswordDB: true, Dex creates a Password custom resource in the dex namespace for each user. You can add users via the Dex API or directly via kubectl. This is suitable for development and small internal deployments. For any environment where more than a handful of people need access, use an upstream connector instead.

Static Clients

Static clients are OAuth2 applications pre-registered with Dex. Every application that wants to obtain tokens from Dex must be listed here.

staticClients:
  # Confidential client — requires a client secret (e.g., a web app with a backend)
  - id: kubev-dashboard
    name: KubeV Dashboard
    secret: "a-strong-random-secret"
    redirectURIs:
      - "https://dashboard.example.com/auth/callback"

  # Public client — no secret, suitable for CLI tools and native apps using PKCE
  - id: kubev-cli
    name: KubeV CLI
    public: true
    redirectURIs:
      - "http://localhost:8000/callback"
      - "urn:ietf:wg:oauth:2.0:oob"   # out-of-band flow for headless environments

Connector ID uniqueness: Each connector id must be unique within the connectors list. Duplicate IDs cause a validation error.

None

idp:
  none: {}

No identity provider is deployed. Use this when you already have an external OIDC provider or when the dashboard is not enabled.

Dashboard

dashboard:
  enabled: true
  dashboardURL: "https://dashboard.example.com"
  auth:
    oidc:
      issuerURL: "https://dex.example.com"
      clientID: "kubev-dashboard"
      clientSecret: "a-strong-client-secret"
      redirectURL: "https://dashboard.example.com/auth/callback"

Service and Scheduling

By default the dashboard runs one pod behind a ClusterIP Service on port 8080, and you publish it yourself, for example with an Ingress. These fields change the dashboard’s Helm chart values; unset fields keep the chart defaults.

dashboard:
  enabled: true
  service:
    type: LoadBalancer
    port: 8080
  replicaCount: 2
  resources:
    requests: {cpu: 100m, memory: 128Mi}
    limits: {cpu: 500m, memory: 256Mi}
  nodeSelector:
    node-role.kubernetes.io/control-plane: ""
  tolerations:
    - key: node-role.kubernetes.io/control-plane
      operator: Exists
      effect: NoSchedule

The installer warns when these fields are set but dashboard.enabled is false, and when a NodePort or LoadBalancer Service has no dashboardURL.

Authentication Modes

The dashboard supports three mutually exclusive authentication modes. Exactly one must be set; if dashboard.enabled is true and auth is omitted, none is applied as the default.

None Mode

dashboard:
  enabled: true
  auth:
    none: {}

The dashboard is publicly accessible without any login. Anyone who can reach dashboardURL has full access.

Use when: The cluster network is private and access-controlled at the infrastructure layer, or during initial setup before authentication is configured.

Basic Mode

Username/password authentication backed by a Kubernetes Secret. The simplest mode to get started — no OIDC provider required.

The minimal configuration is:

dashboard:
  enabled: true
  auth:
    basic: {}

That is all that is required. The installer generates a random credential pair, stores it in the default Secret, and prints the path to a credentials file in the post-apply output. Retrieve the credentials with:

cat <path printed by kubermatic-virtualization apply>

Advanced options — all fields are optional and have sensible defaults:

dashboard:
  enabled: true
  auth:
    basic:
      secretName: "kubev-basic-auth"           # default: kubev-basic-auth
      secretNamespace: "kubermatic-virtualization"  # default: kubermatic-virtualization
      sessionDuration: "24h"                   # default: 24h

Use when: You need a simple login gate without setting up an external identity provider. Suitable for small teams, internal tooling, and environments where SSO is not a requirement.

OIDC Mode

Users authenticate via an external OpenID Connect provider. After a successful login the provider redirects back to redirectURL with an authorization code; the dashboard’s API server exchanges it for ID and access tokens.

dashboard:
  enabled: true
  auth:
    oidc:
      issuerURL: "https://dex.example.com"
      clientID: "kubev-dashboard"
      clientSecret: "a-strong-client-secret"
      redirectURL: "https://dashboard.example.com/auth/callback"
      scopes:
        - "openid"
        - "email"
        - "profile"

Use when: You want SSO backed by a corporate identity provider (Keycloak, Azure AD, Okta, Google Workspace, etc.) or by the in-cluster Dex deployed via idp.dex.

Using OIDC with the In-Cluster Dex

When you configure idp.dex, KubeV deploys Dex but does not automatically connect it to the dashboard. You must wire them together manually:

1. Register the dashboard as a static client in idp.dex.staticClients.
2. Set dashboard.auth.oidc.issuerURL to exactly match idp.dex.issuer.
3. Set dashboard.auth.oidc.clientID and clientSecret to match the static client registration.
4. Set dashboard.auth.oidc.redirectURL to <dashboardURL>/auth/callback.

Complete example:

idp:
  dex:
    issuer: "https://dex.example.com"
    enablePasswordDB: true          # quick start: local users
    staticClients:
      - id: kubev-dashboard
        name: KubeV Dashboard
        secret: "a-strong-client-secret"
        redirectURIs:
          - "https://dashboard.example.com/auth/callback"

dashboard:
  enabled: true
  dashboardURL: "https://dashboard.example.com"
  auth:
    oidc:
      issuerURL: "https://dex.example.com"       # must match idp.dex.issuer exactly
      clientID: "kubev-dashboard"                # must match staticClients[].id
      clientSecret: "a-strong-client-secret"     # must match staticClients[].secret
      redirectURL: "https://dashboard.example.com/auth/callback"

cert-manager

KubeV always installs cert-manager.

certManager:
  gatewayAPI: true

VM Migration from vSphere

KubeV migrates virtual machines from VMware vSphere with Forklift, which the installer installs and manages, in the namespace konveyor-forklift. The migration section is optional: without it, Forklift is installed.

migration:
  enabled: true
  vddk:
    image: "registry.example.com/vddk:8.0.3"
    imagePullSecret: |
      {"auths":{"registry.example.com":{"auth":"<base64 of username:password>"}}}

Not to be confused with kubevirt.migrations, which tunes live migration between nodes.

Turning migration off on a cluster that has Forklift uninstalls the Forklift operator. Its CRDs and the konveyor-forklift namespace stay, with every provider, plan and migration and the ForkliftController. To also stop the components the operator started, delete the ForkliftController first, while the operator is still installed, and then set enabled: false.

VDDK

VMware’s Virtual Disk Development Kit (VDDK) lets Forklift copy disks faster than over the vCenter’s HTTPS datastore path, and it is required for warm migrations. KubeV doesn’t ship it: build an image from the VDDK archive you download from Broadcom, push it to a registry the cluster can reach, and name it in vddk.image.

Forklift then uses it for every vSphere provider that doesn’t name its own image. In KDP mode, providers may not name their own, so vddk.image is the only way to give KDP projects VDDK.

With vddk.image set, Forklift runs the pods of a migration plan as the ServiceAccount kubev-migration, unless the plan names its own. The KubeV controller-manager creates it in each plan’s target namespace. With vddk.imagePullSecret, the credentials are kept as the Secret kubev-vddk-pull in the kubermatic-virtualization namespace and copied into each plan’s target namespace for as long as a plan targets it. Anyone who can read Secrets there can read the copy, so use a token that can only pull.

Platform Feature Gates

featureGates turns on optional KubeV features that are off by default. Unlike kubevirt.developerConfiguration.featureGates, which are KubeVirt’s own, these gates are KubeV’s: an unknown name is refused when the file is loaded.

featureGates:
  CopyOffload: true

KDP Integration

The kdp section runs KubeV as a service of the Kubermatic Developer Platform (KDP): users get virtual machines in their KDP projects, and KDP manages organizations, projects, members and roles. The KDP Integration guide explains how it works and walks through each installation mode, and The kdp section lists every field.

kdp:
  enabled: true
  adminGroup: "kubev-platform-admins"
  domains:
    dashboard: kdp.example.com
    kcp: kcp.example.com

Which other fields the section has decides the mode:

In every mode, a configuration with the dashboard enabled also needs:

• dashboard.auth.oidc. KDP, kcp and the KubeV dashboard share this one OIDC client.
• dashboard.dashboardURL, the absolute http:// or https:// address users open KubeV at.

The nested mode needs dashboard.enabled: false. In the same layer and nested modes, the installer pulls the KDP charts with KUBEV_USERNAME and KUBEV_PASSWORD, even when imagePullSecret is set.

Use when: You run, or want to run, KDP as your internal developer platform, and want teams to get virtual machines from its catalog.

Complete Example

A production-grade configuration with 3-node HA control plane, 2 worker nodes, MetalLB, Longhorn, in-cluster Dex federated to Google and LDAP, the dashboard in OIDC mode, and VM migration from vSphere with a VDDK image:

apiVersion: virtualization.k8c.io/v1alpha1
kind: KubeVCluster

networkConfiguration:
  dnsServerIP: "192.168.1.1"
  networkCIDR: "10.244.0.0/16"
  serviceCIDR: "10.96.0.0/12"
  gatewayIP: "10.244.0.1"
  tunnelInterface: "eth0"

apiEndpoint:
  host: "api.example.com"
  alternativeNames:
    - "192.168.1.100"

controlPlane:
  hosts:
    - address: "192.168.1.10"
      sshUsername: "ubuntu"
      sshPrivateKeyFile: "/home/ops/.ssh/id_rsa"
    - address: "192.168.1.11"
      sshUsername: "ubuntu"
      sshPrivateKeyFile: "/home/ops/.ssh/id_rsa"
    - address: "192.168.1.12"
      sshUsername: "ubuntu"
      sshPrivateKeyFile: "/home/ops/.ssh/id_rsa"

staticWorkers:
  hosts:
    - address: "192.168.1.20"
      sshUsername: "ubuntu"
      sshPrivateKeyFile: "/home/ops/.ssh/id_rsa"
    - address: "192.168.1.21"
      sshUsername: "ubuntu"
      sshPrivateKeyFile: "/home/ops/.ssh/id_rsa"

loadBalancer:
  metallb:
    ipRange: "192.168.1.100-192.168.1.150"

storage:
  longhorn: {}

kubevirt:
  developerConfiguration:
    featureGates:
      - "LiveMigration"
      - "Snapshot"
    useEmulation: false

idp:
  dex:
    issuer: "https://dex.example.com"
    enablePasswordDB: false
    connectors:
      - type: oidc
        id: google
        name: Google
        config:
          issuer: https://accounts.google.com
          clientID: "your-google-client-id"
          clientSecret: "your-google-client-secret"
          redirectURI: "https://dex.example.com/callback"
      - type: ldap
        id: ldap
        name: Corporate Directory
        config:
          host: "ldap.corp.example.com:389"
          startTLS: true
          bindDN: "cn=kubev-svc,ou=serviceaccounts,dc=corp,dc=example,dc=com"
          bindPW: "service-account-password"
          userSearch:
            baseDN: "ou=people,dc=corp,dc=example,dc=com"
            filter: "(objectClass=person)"
            username: uid
            idAttr: uid
            emailAttr: mail
            nameAttr: displayName
    staticClients:
      - id: kubev-dashboard
        name: KubeV Dashboard
        secret: "a-strong-client-secret"
        redirectURIs:
          - "https://dashboard.example.com/auth/callback"

dashboard:
  enabled: true
  dashboardURL: "https://dashboard.example.com"
  auth:
    oidc:
      issuerURL: "https://dex.example.com"
      clientID: "kubev-dashboard"
      clientSecret: "a-strong-client-secret"
      redirectURL: "https://dashboard.example.com/auth/callback"

migration:
  enabled: true
  vddk:
    image: "registry.example.com/vddk:8.0.3"

imagePullSecret: |
  {"auths":{"quay.io":{"auth":"<base64 of username:password>"}}}