Known Issues

This page documents the list of known issues in Kubermatic KubeOne along with possible workarounds and recommendations.

This list applies to KubeOne 1.8 release. For KubeOne 1.7, please consider the v1.7 version of this document. For earlier releases, please consult the appropriate changelog.

AzureDisk and AzureFile CSI drivers are not supported on CentOS 7

StatusMitigation provided
SeverityLow
GitHub issueN/A

Who’s affected by this issue?

This issue affects only Azure clusters that are running CentOS 7. Other CentOS-like and RHEL-like distributions, such as Rocky Linux, are not affected.

Description

Trying to mount a volume created using AzureDisk or AzureFile CSI driver results in an error saying that operation is not supported.

Mitigation

Given that CentOS 7 is reaching end-of-life (EOL) on June 30, 2024, we strongly recommend migrating to another distribution.

If this is not doable for you at the moment, we recommend:

  • Staying at KubeOne 1.7 until migrating to another supported distribution
  • Using KubeOne 1.8 with an older version of AzureDisk and AzureFile CSI drivers. AzureDisk releases up to and including v1.28.5 and v1.29.2 are known to work with CentOS 7.

Cilium CNI is not working on clusters running CentOS 7

StatusKnown Issue
SeverityLow
GitHub issueN/A

Description

Cilium CNI is not supported on CentOS 7 because it’s using too older kernel version which is not supported by Cilium itself. For more details, consider the official Cilium documentation.

Recommendation

Please consider using an operating system with a newer kernel version, such as Ubuntu, Rocky Linux, and Flatcar. See the official Cilium documentation for a list of operating systems and versions supported by Cilium.

Internal Kubernetes endpoints unreachable on vSphere with Cilium/Canal

StatusWorkaround available
SeverityLow
GitHub issuehttps://github.com/cilium/cilium/issues/21801

Description

Symptoms

  • Unable to perform CRUD operations on resources governed by webhooks (e.g. ValidatingWebhookConfiguration, MutatingWebhookConfiguration, etc.). The following error is observed:
Internal error occurred: failed calling webhook "webhook-name": failed to call webhook: Post "https://webhook-service-name.namespace.svc:443/webhook-endpoint": context deadline exceeded
  • Unable to reach internal Kubernetes endpoints from pods/nodes.
  • ICMP is working but TCP/UDP is not.

Cause

On recent enough VMware hardware compatibility version (i.e >=15 or maybe >=14), CNI connectivity breaks because of hardware segmentation offload. cilium-health status has ICMP connectivity working, but not TCP connectivity. cilium-health status may also fail completely.

Recommendation

sudo ethtool -K ens192 tx-udp_tnl-segmentation off
sudo ethtool -K ens192 tx-udp_tnl-csum-segmentation off

These flags are related to the hardware segmentation offload done by the vSphere driver VMXNET3. We have observed this issue for both Cilium and Canal CNI running on Ubuntu 22.04.

We have two options to configure these flags for KubeOne installations:

References