Architecture

Architecture

With an aim to make machine configurations extendable, OSM introduces the following new resources as Kubernetes Custom Resource Definitions:

OperatingSystemProfile

A resource that contains scripts for bootstrapping and provisioning the worker nodes, along with information about what operating systems and versions are supported for given scripts. Additionally, OSPs support templating so you can include some information from MachineDeployment or the OSM deployment itself.

OSM comes with a set of default OSPs for various supported operating systems and cloud provider combinations. These OSPs are installed by OSM itself. End users can create custom OSPs as well to fit their own use-cases.

OSPs are immutable by design and any modifications to an existing OSP requires a version bump in .spec.version.

OperatingSystemConfig

Immutable resource that contains the actual configurations that are going to be used to bootstrap and provision the worker nodes. It is a subset of OperatingSystemProfile. OperatingSystemProfile is a template while OperatingSystemConfig is an instance rendered with data from OperatingSystemProfile, MachineDeployment, and flags provided at OSM command-line level.

OperatingSystemConfigs have a 1-to-1 relation with the MachineDeployment. A dedicated controller watches the MachineDeployments and generates the OSCs in kube-system(can be overridden).

Kubernetes Secrets

The generated OperatingSystemConfigs are then transformed into configurations accepted by cloud-init or ignition provisioning utilities. They are persisted as kubernetes secrets in cloud-init-settings namespace in the cluster.

The following secrets are generated by OSM:

1. Bootstrap: Configuration used for initially setting up the machine and fetching the provisioning configuration.
2. Provisioning: Configuration with the actual cloud-config that is used to provision the worker machine.

The significance behind using a dedicated namespace i.e. cloud-init-settings is that the secret that holds the provisioning configuration is dynamically fetched during the bootstrap phase. Using a dedicated namespace isolates these secrets and limits the access area.

Single vs management/worker cluster mode

Conventionally OSM operates within a single cluster and expects all of the required resources like machine controller, MachineDeployments etc. to exist within the same cluster.

Along with that, OSM also supports environments where workloads are divided into management and worker clusters. This is useful since it helps with completely abstracting away OSM from the users of worker cluster; OSM will be running in the management cluster.

To use management/worker cluster mode, simply pass on the kubeconfig for management cluster using kubeconfig and worker cluster using the worker-cluster-kubeconfig flags at OSM level. With this topology the OSP and OSC exist within the management cluster while only the bootstrap and provisioning secrets are created in the worker clusters.

Integration with machine-controller

When a MachineDeployment is created, OSM reconciles it and generates the corresponding OSC and secrets based on the selected OSP. The OSP to be used for a MachineDeployment is specified using k8c.io/operating-system-profile annotation on the MachineDeployment. Machine controller waits for the bootstrapping and provisioning secrets to become available. Once they are made available by OSM, machine controller will extract the configurations from those secrets and propagate them as user-data for the machines that are to be provisioned.