Configuring Credentials

KubeOne deploys the provider credentials to the cluster to be used by components such as Kubernetes cloud-controller-manager and Kubermatic machine-controller. Those components communicate with the provider’s API to the create worker nodes, pull the node metadata, provide advanced capabilities such as LoadBalancer Services and more. Besides that, Terraform uses the credentials to provision and manage the infrastructure.

KubeOne can grab credentials from the user’s environment or the user can provide the needed credentials in a dedicated credentials file.

As credentials are deployed to the cluster, it’s recommended to use dedicated, non-administrator credentials whenever it’s possible.

You can skip this document if you’re deploying to bare-metal or provider that’s not natively supported.

The credentials file is KubeOne-specific and it will not work with Terraform. If you are using Terraform, consider the environment variables approach or check the Terraform documentation for other authentication options.

Environment Variables

By default, KubeOne grabs credentials from the user’s environment unless the credentials file is provided. In the following tables, you can find the environment variables used by KubeOne.

You need an IAM account with the appropriate permissions for Terraform to create the infrastructure and for machine-controller to create worker nodes.

Environment VariableDescription
AWS_ACCESS_KEY_IDThe AWS Access Key
AWS_SECRET_ACCESS_KEYThe AWS Secret Access Key
AWS_PROFILEName of the profile defined in the ~/.aws/credentials file. This variable is considered only if AWS_ACCESS_KEY_ID or AWS_SECRET_ACCESS_KEY is unset

The following environment variables are needed by machine-controller for creating worker nodes.

For the Terraform reference please take a look at Azure provider docs.

Environment VariableDescription
ARM_CLIENT_IDAzure ClientID
ARM_CLIENT_SECRETAzure Client secret
ARM_TENANT_IDAzure TenantID
ARM_SUBSCRIPTION_IDAzure SubscriptionID

You need an API Access Token with read and write permission for Terraform to create the infrastructure, machine-controller to create the worker nodes, and for DigitalOcean Cloud Controller Manager.

Environment VariableDescription
DIGITALOCEAN_TOKENThe DigitalOcean API Access Token with read/write permissions

You need an Service Account with the appropriate permissions for Terraform to create the infrastructure and for machine-controller to create worker nodes.

The needed permissions are are:

  • Compute Admin: roles/compute.admin
  • Service Account User: roles/iam.serviceAccountUser
  • Viewer: roles/viewer

If the gcloud CLI is installed, a service account can be created like follow:

# create new service account
gcloud iam service-accounts create k1-cluster-provisioner

# get your service account id
gcloud iam service-accounts list
# get your project id
gcloud projects list

# create policy binding
gcloud projects add-iam-policy-binding YOUR_PROJECT_ID --member 'serviceAccount:YOUR_SERVICE_ACCOUNT_ID' --role='roles/compute.admin'
gcloud projects add-iam-policy-binding YOUR_PROJECT_ID --member 'serviceAccount:YOUR_SERVICE_ACCOUNT_ID' --role='roles/iam.serviceAccountUser'
gcloud projects add-iam-policy-binding YOUR_PROJECT_ID --member 'serviceAccount:YOUR_SERVICE_ACCOUNT_ID' --role='roles/viewer'

A Google Service Account for the platform has to be created, see Creating and managing service accounts.

The result is a JSON file containing the fields:

  • type
  • project_id
  • private_key_id
  • private_key
  • client_email
  • client_id
  • auth_uri
  • token_uri
  • auth_provider_x509_cert_url
  • client_x509_cert_url
# create a new json key for your service account
gcloud iam service-accounts keys create --iam-account YOUR_SERVICE_ACCOUNT k1-cluster-provisioner-sa-key.json

Also, the Compute Engine API has to be enabled for the project in the Google APIs Console.

Once you have the Service Account, you need to set GOOGLE_CREDENTIALS environment variable:

# export JSON file content of created service account json key
export GOOGLE_CREDENTIALS=$(cat ./k1-cluster-provisioner-sa-key.json)
Environment VariableDescription
GOOGLE_CREDENTIALSGCE Service Account

You need a Hetzner API Token for Terraform to create the infrastructure, machine-controller to create worker nodes, and for Hetzner Cloud Controller Manager.

Environment VariableDescription
HCLOUD_TOKENThe Hetzner API Access Token

The following environment variables are needed by Terraform for creating the infrastructure and for machine-controller to create the worker nodes.

Environment VariableDescription
NUTANIX_ENDPOINTThe Nutanix API (Prism Central) endpoint
NUTANIX_PORTThe Nutanix API (Prism Central) port
NUTANIX_USERNAMEThe username of the Nutanix user
NUTANIX_PASSWORDThe password of the Nutanix user
NUTANIX_PE_ENDPOINTThe Nutanix Prism Element endpoint (required by CSI driver)
NUTANIX_PE_USERNAMEThe username of the Prism Element user (might be different than the Prism Central user)
NUTANIX_PE_PASSWORDThe password of the Prism Element user (might be different than the Prism Central user)

Besides that, the following environment variables are available, but optional.

Environment VariableDescription
NUTANIX_INSECUREAllow insecure access to the Nutanix API (default false)
NUTANIX_PROXY_URLThe URL of the proxy to the Nutanix API
NUTANIX_CLUSTER_NAMEThe name of the Nutanix cluster (used by machine-controller if not specified in the MachineDeployment)

The following environment variables are needed by Terraform for creating the infrastructure and for machine-controller to create the worker nodes.

Environment VariableDescription
OS_AUTH_URLThe URL of OpenStack Identity Service
OS_USERNAMEThe username of the OpenStack user
OS_PASSWORDThe password of the OpenStack user
OS_DOMAIN_NAMEThe name of the OpenStack domain
OS_TENANT_IDThe ID of the OpenStack tenant
OS_TENANT_NAMEThe name of the OpenStack tenant

You need an API Access Token for Terraform to create the infrastructure, machine-controller to create worker nodes, and for Packet Cloud Controller Manager.

Environment VariableDescription
PACKET_AUTH_TOKENPacket auth token
PACKET_PROJECT_IDPacket project ID

The following environment variables are needed by machine-controller for creating the worker nodes.

For the Terraform reference, please take a look at vSphere provider docs

Environment VariableDescription
VSPHERE_SERVERThe address of the vSphere instance
VSPHERE_USERThe username of the vSphere user
VSPHERE_PASSWORDThe password of the vSphere user

Credentials File

The credentials file is a key-value YAML file, where the key is the environment variable name from the environment variables section. It has the priority over the environment variables, so you can use it if you want to use different credentials or if you don’t want to export credentials as environment variables.

The credentials file is provided to KubeOne using the --credentials or -c flag, such as:

kubeone apply --manifest kubeone.yaml --credentials credentials.yaml -t tf.json

cloudConfig

Besides credentials, the credentials file can take the cloud-config file, which is provided using the cloudConfig key. This can be useful in cases when the cloud-config contains secrets and you want to keep secrets in a different file.

Example:

VSPHERE_SERVER: "<<VSPHERE_SERVER>>"
VSPHERE_USER: "<<VSPHERE_USER>>"
VSPHERE_PASSWORD: "<<VSPHERE_PASSWORD>>"
cloudConfig: |
    <<VSPHERE_CLOUD_CONFIG>>

csiConfig

Analog to cloudConfig a CSI configuration file can be provided for vSphere clusters.

Example:

VSPHERE_SERVER: "<<VSPHERE_SERVER>>"
VSPHERE_USER: "<<VSPHERE_USER>>"
VSPHERE_PASSWORD: "<<VSPHERE_PASSWORD>>"
csiConfig: |
    <<VSPHERE_CSI_CONFIG>>

registriesAuth

When using containerd as container runtime it’s possible to define configurations per registry. This can be useful if you want to use private registries, local mirrors or your own Docker Hub account to increase pull limits.

See the ContainerRuntimeContainerd reference for configuration options.

Example:

registriesAuth: |
  apiVersion: kubeone.k8c.io/v1beta2
  kind: ContainerRuntimeContainerd
  registries:
    registry-1.docker.io:
      auth:
        username: "<<DOCKERHUB_USERNAME>>"
        password: "<<DOCKERHUB_PASSWORD>>"

For Docker Hub you need to name the registry key registry-1.docker.io. Just docker.io will not work.

Environment Variables in the Configuration Manifest

KubeOne can source values for supported fields in the configuration manifest directly from the environment. The value has to be in the format of env:<<ENVIRONMENT_VARIABLE>>.

In the following table you can find the configuration manifest fields with support for sourcing value using the env: prefix:

VariableTypeDefault ValueDescription
hosts.ssh_agent_socketstring""Socket to be used for SSH