Beyond the YAML configuration file, Conformance EE provides additional runtime options.
The conformance-tester CLI is the recommended way to run Conformance EE. After downloading the binary via kubermatic-ee-downloader, launch it:
./conformance-tester
The interactive TUI guides you through all runtime decisions:
| Step | Description |
|---|---|
| Environment | Choose between deploying locally or to an existing cluster |
| Kubeconfig | Select a kubeconfig source (from KUBECONFIG env, default path, or custom file) |
| Provider | Choose the cloud provider to test |
| Kubernetes Versions | Select which Kubernetes versions to include |
| OS Distributions | Pick OS distributions and image sources |
| Datacenters | Choose target datacenters |
| Cluster Settings | Configure CNI, proxy mode, expose strategy, and other cluster modifiers |
| Machine Settings | Configure CPU, memory, disk, and other machine modifiers |
| Review & Deploy | Review the generated test matrix and deploy |
All selections made in the TUI are translated into a configuration file and deployed automatically.
| Variable | Description |
|---|---|
CONFORMANCE_TESTER_CONFIG_FILE | Path to the YAML configuration file (used in automated/in-cluster mode) |
KUBECONFIG | Path to the kubeconfig file (the TUI auto-detects this) |
HIDE_PRESET_CREDENTIALS | Set to true to mask credential values in the TUI |
When running Conformance EE as a Kubernetes Job (see In-Cluster Deployment), additional flags can be passed in the Job spec’s args field to control test execution:
| Flag | Default | Description |
|---|---|---|
--datacenters | "" | Comma-separated list of datacenters to test |
--kube-versions | "" | Comma-separated Kubernetes versions to test |
--skip-cluster-creation | false | Skip cluster creation; use existing clusters |
--skip-cluster-deletion | false | Keep clusters after tests (useful for debugging) |
--update-clusters | false | Upgrade existing clusters before running tests |
--verbose-logs | false | Show additional log output from test code |
Test specific datacenters:
args:
- --datacenters=dc-1,dc-2
Keep clusters alive for debugging:
args:
- --skip-cluster-deletion
Reuse existing clusters from a previous run:
args:
- --skip-cluster-creation
Upgrade existing clusters before testing:
args:
- --update-clusters
- --skip-cluster-creation