KubeOne comes with example Terraform configs that can be used to create the
infrastructure needed for running a conformant, production-grade Kubernetes
cluster. The example configs are available for all natively supported
providers and can be found on the
GitHub under the examples/terraform
directory. They are also coming along with the binaries when you download a
KubeOne release from GitHub Releases.
The example Terraform configs are supposed to be used as a foundation for building your own configs. The configs are optimized for ease of use and using in E2E tests, and therefore might not be suitable for the production usage out of the box.
Please check the Production Recommendations document for more details about making the example configs suitable for the production usage.
Before getting started, make sure that you have:
If you downloaded KubeOne by using the installation script, downloading the
release, or you’ve checked out the repository, navigate to the
./examples/terraform directory.
If you installed KubeOne using the Arch Linux
package, the example configs are located in the
/usr/share/doc/kubeone/examples/terraform/ directory. You should copy those
configs to some other place, as they might be removed/overwritten when
upgrading the package.
In this directory, you can find directories for each supported provider. Navigate to the appropriate directory depending on the provider you want to use. Each provider’s directory has the following files:
| Variable | Description |
|---|---|
README.md | information about required and available input and output |
main.tf | the main script defining all resources that will be created by Terraform |
output.tf | defines the format of the file generated using the terraform output -json command. The resulting file is used by the KubeOne Terraform Integration to source the information about infrastructure |
variables.tf | available variables that can be used to configure provisioning (e.g. cluster properties, region, instances size, image to be used, and more) |
versions.tf | required Terraform and provider’s plugin versions |
Additionally, if your desired provider doesn’t provide LBaaS (Load Balancer as
a Service), gobetween.sh and etc_gobetween.tpl are used to configure the
GoBetween Load Balancer to be used to access the Kubernetes API.
The first step you’ve to do before using the example configs is to initialize
the Terraform working directory and download the required plugins.
This is done by running the init command:
terraform init
You have to run this command only the first time you’re using the example
configs. If you already initialized the Terraform working directory and want to
upgrade plugins instead, you can run terraform init -upgrade.
It’s recommended that you familiarize yourself with the available variables and
their default values by checking the variables.tf file. Some of the variables
are consistent for all providers, such as the cluster name, the operating
system to be used for the worker nodes (MachineDeployments), and the SSH
information.
You should consider setting at least the following variables:
| Variable | Required | Default Value | Description |
|---|---|---|---|
| cluster_name | yes | cluster name and prefix for cloud resources | |
| ssh_public_key_file | ~/.ssh/id_rsa.pub | path to your SSH public key that’s deployed on instances |
Make sure that you have configured your SSH agent with the appropriate key as described in the Configuring SSH guide. If your setup doesn’t support the SSH agent, you can provide an unencrypted private key instead by following those steps.
The easiest way to set variables is to put them in the terraform.tfvars file
which is parsed by default. The file should be co-located with other Terraform
files and can look like:
cluster_name = "example"
ssh_public_key_file = "~/.ssh/terraform_rsa.pub"
You may want to change other variables as well, like the region. If you decide to change information such as the instance size, the image to be used, or operating system, ensure that you comply with requirements defined in the Requirements and Compatibility documents.
With the variables configured, you’re ready to create the infrastructure by applying the configs.
You can see what changes will be made by running the plan command:
terraform plan
If you agree with the proposed changes, run the apply command to create
the infrastructure. You’ll be asked to type yes to confirm your intention.
terraform apply
Due to how GCP LBs work, initial terraform apply requires variable
control_plane_target_pool_members_count to be set to 1.
terraform apply -var=control_plane_target_pool_members_count=1
Once initial kubeone install or kubeone apply is done, the control_plane_target_pool_members_count should not be
used.
It takes several minutes to provision the infrastructure and for instances to come up.
The last step regarding provisioning infrastructure is to export the Terraform state to be parsed by the KubeOne Terraform Integration for information about instances and worker nodes. The following information is used by KubeOne:
The output.tf file defines the template used to generate the file and the
information to be included in a state file generated by Terraform.
The state file is generated using the output command. KubeOne requires the
file to be in the JSON format.
terraform output -json > tf.json
If you modify variables and/or output.tf file after running
terraform apply, you’re required to run terraform apply and
terraform output commands again after changes. Note that modifying variables
may cause all resources to be recreated causing the data loss!