Merge pull request #26786 from neolit123/1.21-cgroup-drivers-new-page
kubeadm: add separate page for configuring / migrating cgroup driver
This commit is contained in:
Kubernetes Prow Robot
GitHub
commit
e7a3482b6a
|
|
@ -48,7 +48,7 @@ Changing the settings such that your container runtime and kubelet use `systemd`
|
|||
stabilized the system. To configure this for Docker, set `native.cgroupdriver=systemd`.
|
||||
|
||||
{{< caution >}}
|
||||
Changing the cgroup driver of a Node that has joined a cluster is strongly *not* recommended.
|
||||
Changing the cgroup driver of a Node that has joined a cluster is a sensitive operation.
|
||||
If the kubelet has created Pods using the semantics of one cgroup driver, changing the container
|
||||
runtime to another cgroup driver can cause errors when trying to re-create the Pod sandbox
|
||||
for such existing Pods. Restarting the kubelet may not solve such errors.
|
||||
|
|
@ -57,6 +57,11 @@ If you have automation that makes it feasible, replace the node with another usi
|
|||
configuration, or reinstall it using automation.
|
||||
{{< /caution >}}
|
||||
|
||||
### Migrating to the `systemd` driver in kubeadm managed clusters
|
||||
|
||||
Follow this [Migration guide](/tasks/administer-cluster/kubeadm/configure-cgroup-driver)
|
||||
if you wish to migrate to the `systemd` cgroup driver in existing kubeadm managed clusters.
|
||||
|
||||
## Container runtimes
|
||||
|
||||
{{% thirdparty-content %}}
|
||||
|
|
|
|||
|
|
@ -244,7 +244,7 @@ sudo mkdir -p /opt/cni/bin
|
|||
curl -L "https://github.com/containernetworking/plugins/releases/download/${CNI_VERSION}/cni-plugins-linux-amd64-${CNI_VERSION}.tgz" | sudo tar -C /opt/cni/bin -xz
|
||||
```
|
||||
|
||||
Define the directory to download command files
|
||||
Define the directory to download command files
|
||||
|
||||
{{< note >}}
|
||||
The `DOWNLOAD_DIR` variable must be set to a writable directory.
|
||||
|
|
@ -295,33 +295,17 @@ See the [Kubeadm Troubleshooting guide](/docs/setup/production-environment/tools
|
|||
The kubelet is now restarting every few seconds, as it waits in a crashloop for
|
||||
kubeadm to tell it what to do.
|
||||
|
||||
## Configure cgroup driver used by kubelet on control-plane node
|
||||
## Configuring a cgroup driver
|
||||
|
||||
When using Docker, kubeadm will automatically detect the cgroup driver for the kubelet
|
||||
and set it in the `/var/lib/kubelet/config.yaml` file during runtime.
|
||||
Both the container runtime and the kubelet have a property called
|
||||
["cgroup driver"](/docs/setup/production-environment/container-runtimes/), which is important
|
||||
for the management of cgroups on Linux machines.
|
||||
|
||||
If you are using a different CRI, you must pass your `cgroupDriver` value to `kubeadm init`, like so:
|
||||
|
||||
```yaml
|
||||
apiVersion: kubelet.config.k8s.io/v1beta1
|
||||
kind: KubeletConfiguration
|
||||
cgroupDriver: <value>
|
||||
```
|
||||
|
||||
For further details, please read [Using kubeadm init with a configuration file](/docs/reference/setup-tools/kubeadm/kubeadm-init/#config-file).
|
||||
|
||||
Please mind, that you **only** have to do that if the cgroup driver of your CRI
|
||||
is not `cgroupfs`, because that is the default value in the kubelet already.
|
||||
|
||||
{{< note >}}
|
||||
Since `--cgroup-driver` flag has been deprecated by the kubelet, if you have that in `/var/lib/kubelet/kubeadm-flags.env`
|
||||
or `/etc/default/kubelet`(`/etc/sysconfig/kubelet` for RPMs), please remove it and use the KubeletConfiguration instead
|
||||
(stored in `/var/lib/kubelet/config.yaml` by default).
|
||||
{{< /note >}}
|
||||
|
||||
The automatic detection of cgroup driver for other container runtimes
|
||||
like CRI-O and containerd is work in progress.
|
||||
{{< warning >}}
|
||||
Matching the container runtime and kubelet cgroup drivers is required or otherwise the kubelet process will fail.
|
||||
{{< warning >}}
|
||||
|
||||
See [Configuring a cgroup driver](/tasks/administer-cluster/kubeadm/configure-cgroup-driver) for more details.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,127 @@
|
|||
---
|
||||
title: Configuring a cgroup driver
|
||||
content_type: task
|
||||
weight: 10
|
||||
---
|
||||
|
||||
<!-- overview -->
|
||||
|
||||
This page explains how to configure the kubelet cgroup driver to match the container
|
||||
runtime cgroup driver for kubeadm clusters.
|
||||
|
||||
## {{% heading "prerequisites" %}}
|
||||
|
||||
You should be familiar with the Kubernetes
|
||||
[container runtime requirements](/docs/setup/production-environment/container-runtimes).
|
||||
|
||||
<!-- steps -->
|
||||
|
||||
## Configuring the container runtime cgroup driver
|
||||
|
||||
The [Container runtimes](/docs/setup/production-environment/container-runtimes) page
|
||||
explains that the `systemd` driver is recommended for kubeadm based setups instead
|
||||
of the `cgroupfs` driver, because kubeadm manages the kubelet as a systemd service.
|
||||
|
||||
The page also provides details on how to setup a number of different container runtimes with the
|
||||
`systemd` driver by default.
|
||||
|
||||
## Configuring the kubelet cgroup driver
|
||||
|
||||
kubeadm allows you to pass a `KubeletConfiguration` structure during `kubeadm init`.
|
||||
This `KubeletConfiguration` can include the `cgroupDriver` field which controls the cgroup
|
||||
driver of the kubelet.
|
||||
|
||||
{{< note >}}
|
||||
|
||||
{{< feature-state for_k8s_version="v1.21" state="stable" >}}
|
||||
|
||||
If the user is not setting the `cgroupDriver` field under `KubeletConfiguration`,
|
||||
`kubeadm init` will default it to `systemd`.
|
||||
{{< /note >}}
|
||||
|
||||
A minimal example of configuring the field explicitly:
|
||||
|
||||
```yaml
|
||||
# kubeadm-config.yaml
|
||||
kind: ClusterConfiguration
|
||||
apiVersion: kubeadm.k8s.io/v1beta2
|
||||
kubernetesVersion: v1.21
|
||||
---
|
||||
kind: KubeletConfiguration
|
||||
apiVersion: kubelet.config.k8s.io/v1beta1
|
||||
cgroupDriver: systemd
|
||||
```
|
||||
|
||||
Such a configuration file can then be passed to the kubeadm command:
|
||||
|
||||
```shell
|
||||
kubeadm init --config kubeadm-config.yaml
|
||||
```
|
||||
|
||||
{{< note >}}
|
||||
Kubeadm uses the same `KubeletConfiguration` for all nodes in the cluster.
|
||||
The `KubeletConfiguration` is stored in a [ConfigMap](docs/concepts/configuration/configmap)
|
||||
object under the `kube-system` namespace.
|
||||
|
||||
Executing the sub commands `init`, `join` and `upgrade` would result in kubeadm
|
||||
writing the `KubeletConfiguration` as a file under `/var/lib/kubelet/config.yaml`
|
||||
and passing it to the local node kubelet.
|
||||
{{< /note >}}
|
||||
|
||||
## Using the `cgroupfs` driver
|
||||
|
||||
As this guide explains using the `cgroupfs` driver with kubeadm is not recommended.
|
||||
|
||||
To continue using `cgroupfs` and to prevent `kubeadm upgrade` from modifying the
|
||||
`KubeletConfiguration` cgroup driver on existing setups, you must be explicit
|
||||
about its value. This applies to a case where you do not wish future versions
|
||||
of kubeadm to apply the `systemd` driver by default.
|
||||
|
||||
See the below section on "Modify the kubelet ConfigMap" for details on
|
||||
how to be explicit about the value.
|
||||
|
||||
If you wish to configure a container runtime to use the `cgroupfs` driver,
|
||||
you must refer to the documentation of the container runtime of your choice.
|
||||
|
||||
## Migrating to the `systemd` driver
|
||||
|
||||
To change the cgroup driver of an existing kubeadm cluster to `systemd` in-place,
|
||||
a similar procedure to a kubelet upgrade is required. This must include both
|
||||
steps outlined below.
|
||||
|
||||
{{< note >}}
|
||||
Alternatively, it is possible to replace the old nodes in the cluster with new ones
|
||||
that use the `systemd` driver. This requires executing only the first step below
|
||||
before joining the new nodes and ensuring the workloads can safely move to the new
|
||||
nodes before deleting the old nodes.
|
||||
{{< /note >}}
|
||||
|
||||
### Modify the kubelet ConfigMap
|
||||
|
||||
- Find the kubelet ConfigMap name using `kubectl get cm -n kube-system | grep kubelet-config`.
|
||||
- Call `kubectl edit cm kubelet-config-x.yy -n kube-system` (replace `x.yy` with
|
||||
the Kubernetes version).
|
||||
- Either modify the existing `cgroupDriver` value or add a new field that looks like this:
|
||||
|
||||
```yaml
|
||||
cgroupDriver: systemd
|
||||
```
|
||||
This field must be present under the `kubelet:` section of the ConfigMap.
|
||||
|
||||
### Update the cgroup driver on all nodes
|
||||
|
||||
For each node in the cluster:
|
||||
|
||||
- [Drain the node](/docs/tasks/administer-cluster/safely-drain-node) using `kubectl drain <node-name> --ignore-daemonsets`
|
||||
- Stop the kubelet using `systemctl stop kubelet`
|
||||
- Stop the container runtime
|
||||
- Modify the container runtime cgroup driver to `systemd`
|
||||
- Set `cgroupDriver: systemd` in `/var/lib/kubelet/config.yaml`
|
||||
- Start the container runtime
|
||||
- Start the kubelet using `systemctl start kubelet`
|
||||
- [Uncordon the node](/docs/tasks/administer-cluster/safely-drain-node) using `kubectl uncordon <node-name>`
|
||||
|
||||
Execute these steps on nodes one at a time to ensure workloads
|
||||
have sufficient time to schedule on different nodes.
|
||||
|
||||
Once the process is complete ensure that all nodes and workloads are healthy.
|
||||
Loading…
Reference in New Issue