127 lines
5.3 KiB
Markdown
127 lines
5.3 KiB
Markdown
---
|
|
title: About cgroup v2
|
|
content_type: concept
|
|
weight: 50
|
|
---
|
|
|
|
<!-- overview -->
|
|
|
|
On Linux, {{< glossary_tooltip text="control groups" term_id="cgroup" >}}
|
|
constrain resources that are allocated to processes.
|
|
|
|
The {{< glossary_tooltip text="kubelet" term_id="kubelet" >}} and the
|
|
underlying container runtime need to interface with cgroups to enforce
|
|
[resource management for pods and containers](/docs/concepts/configuration/manage-resources-containers/) which
|
|
includes cpu/memory requests and limits for containerized workloads.
|
|
|
|
There are two versions of cgroups in Linux: cgroup v1 and cgroup v2. cgroup v2 is
|
|
the new generation of the `cgroup` API.
|
|
|
|
<!-- body -->
|
|
|
|
|
|
## What is cgroup v2? {#cgroup-v2}
|
|
{{< feature-state for_k8s_version="v1.25" state="stable" >}}
|
|
|
|
cgroup v2 is the next version of the Linux `cgroup` API. cgroup v2 provides a
|
|
unified control system with enhanced resource management
|
|
capabilities.
|
|
|
|
cgroup v2 offers several improvements over cgroup v1, such as the following:
|
|
|
|
- Single unified hierarchy design in API
|
|
- Safer sub-tree delegation to containers
|
|
- Newer features like [Pressure Stall Information](https://www.kernel.org/doc/html/latest/accounting/psi.html)
|
|
- Enhanced resource allocation management and isolation across multiple resources
|
|
- Unified accounting for different types of memory allocations (network memory, kernel memory, etc)
|
|
- Accounting for non-immediate resource changes such as page cache write backs
|
|
|
|
Some Kubernetes features exclusively use cgroup v2 for enhanced resource
|
|
management and isolation. For example, the
|
|
[MemoryQoS](/blog/2021/11/26/qos-memory-resources/) feature improves memory QoS
|
|
and relies on cgroup v2 primitives.
|
|
|
|
|
|
## Using cgroup v2 {#using-cgroupv2}
|
|
|
|
The recommended way to use cgroup v2 is to use a Linux distribution that
|
|
enables and uses cgroup v2 by default.
|
|
|
|
To check if your distribution uses cgroup v2, refer to [Identify cgroup version on Linux nodes](#check-cgroup-version).
|
|
|
|
### Requirements
|
|
|
|
cgroup v2 has the following requirements:
|
|
|
|
* OS distribution enables cgroup v2
|
|
* Linux Kernel version is 5.8 or later
|
|
* Container runtime supports cgroup v2. For example:
|
|
* [containerd](https://containerd.io/) v1.4 and later
|
|
* [cri-o](https://cri-o.io/) v1.20 and later
|
|
* The kubelet and the container runtime are configured to use the [systemd cgroup driver](/docs/setup/production-environment/container-runtimes#systemd-cgroup-driver)
|
|
|
|
### Linux Distribution cgroup v2 support
|
|
|
|
For a list of Linux distributions that use cgroup v2, refer to the [cgroup v2 documentation](https://github.com/opencontainers/runc/blob/main/docs/cgroup-v2.md)
|
|
|
|
<!-- the list should be kept in sync with https://github.com/opencontainers/runc/blob/main/docs/cgroup-v2.md -->
|
|
* Container Optimized OS (since M97)
|
|
* Ubuntu (since 21.10, 22.04+ recommended)
|
|
* Debian GNU/Linux (since Debian 11 bullseye)
|
|
* Fedora (since 31)
|
|
* Arch Linux (since April 2021)
|
|
* RHEL and RHEL-like distributions (since 9)
|
|
|
|
To check if your distribution is using cgroup v2, refer to your distribution's
|
|
documentation or follow the instructions in [Identify the cgroup version on Linux nodes](#check-cgroup-version).
|
|
|
|
You can also enable cgroup v2 manually on your Linux distribution by modifying
|
|
the kernel cmdline boot arguments. If your distribution uses GRUB,
|
|
`systemd.unified_cgroup_hierarchy=1` should be added in `GRUB_CMDLINE_LINUX`
|
|
under `/etc/default/grub`, followed by `sudo update-grub`. However, the
|
|
recommended approach is to use a distribution that already enables cgroup v2 by
|
|
default.
|
|
|
|
### Migrating to cgroup v2 {#migrating-cgroupv2}
|
|
|
|
To migrate to cgroup v2, ensure that you meet the [requirements](#requirements), then upgrade
|
|
to a kernel version that enables cgroup v2 by default.
|
|
|
|
The kubelet automatically detects that the OS is running on cgroup v2 and
|
|
performs accordingly with no additional configuration required.
|
|
|
|
There should not be any noticeable difference in the user experience when
|
|
switching to cgroup v2, unless users are accessing the cgroup file system
|
|
directly, either on the node or from within the containers.
|
|
|
|
cgroup v2 uses a different API than cgroup v1, so if there are any
|
|
applications that directly access the cgroup file system, they need to be
|
|
updated to newer versions that support cgroup v2. For example:
|
|
|
|
* Some third-party monitoring and security agents may depend on the cgroup filesystem.
|
|
Update these agents to versions that support cgroup v2.
|
|
* If you run [cAdvisor](https://github.com/google/cadvisor) as a stand-alone
|
|
DaemonSet for monitoring pods and containers, update it to v0.43.0 or later.
|
|
* If you use JDK, prefer to use JDK 11.0.16 and later or JDK 15 and later, which [fully support cgroup v2](https://bugs.openjdk.org/browse/JDK-8230305).
|
|
|
|
## Identify the cgroup version on Linux Nodes {#check-cgroup-version}
|
|
|
|
The cgroup version depends on the Linux distribution being used and the
|
|
default cgroup version configured on the OS. To check which cgroup version your
|
|
distribution uses, run the `stat -fc %T /sys/fs/cgroup/` command on
|
|
the node:
|
|
|
|
```shell
|
|
stat -fc %T /sys/fs/cgroup/
|
|
```
|
|
|
|
For cgroup v2, the output is `cgroup2fs`.
|
|
|
|
For cgroup v1, the output is `tmpfs.`
|
|
|
|
## {{% heading "whatsnext" %}}
|
|
|
|
- Learn more about [cgroups](https://man7.org/linux/man-pages/man7/cgroups.7.html)
|
|
- Learn more about [container runtime](/docs/concepts/architecture/cri)
|
|
- Learn more about [cgroup drivers](/docs/setup/production-environment/container-runtimes#cgroup-drivers)
|