Control Groups (cgroup v2) Explained: Limiting and Accounting for Resources

If namespaces answer “what can a process see,” control groups (cgroups) answer “how much can a process use.” Every CPU share, memory limit, and I/O bandwidth cap applied to a container ultimately resolves to cgroup settings, exposed as an ordinary — if unusual — filesystem hierarchy.

v1’s problem: independent hierarchies per controller

cgroup v1 let each controller (CPU, memory, blkio, and so on) be mounted as its own, entirely independent hierarchy, meaning a process could belong to different, unrelated group structures for CPU limits versus memory limits. This flexibility turned out to be more trouble than it was worth in practice — tooling had to reconcile multiple parallel hierarchies, and delegating a subtree of control to a container runtime meant delegating potentially several unrelated trees at once.

v2’s fix: one unified hierarchy

cgroup v2 (stable since Linux 4.5, and the default on all current major distributions) uses a single hierarchy that all controllers attach to simultaneously. A process belongs to exactly one cgroup at any point in the tree, and that cgroup can have any combination of controllers enabled on it.

mount -t cgroup2 none /sys/fs/cgroup   # typically already mounted by the init system
cat /sys/fs/cgroup/cgroup.controllers  # controllers available at this level

Creating a cgroup and enabling controllers

A cgroup is just a directory. Creating one is mkdir; enabling a controller for its children happens by writing to the parent’s cgroup.subtree_control:

mkdir /sys/fs/cgroup/myapp
echo "+cpu +memory +io" > /sys/fs/cgroup/myapp/cgroup.subtree_control

Controllers must be explicitly enabled top-down — a child cgroup can only use a controller its parent has enabled for it, which is what keeps the whole tree’s resource accounting consistent and prevents a subtree from escaping limits its parent doesn’t know about.

Assigning processes

Moving a process into a cgroup means writing its PID to that cgroup’s cgroup.procs:

echo $$ > /sys/fs/cgroup/myapp/cgroup.procs

Every process in /sys/fs/cgroup/myapp/cgroup.procs is now accounted and limited according to whatever this cgroup (and its ancestors) specify.

Limiting CPU

The cpu controller exposes cpu.max, expressed as a quota and a period (in microseconds) — “this many microseconds of CPU time per period”:

# Allow 50% of one CPU core: 50000us out of every 100000us period
echo "50000 100000" > /sys/fs/cgroup/myapp/cpu.max

cpu.weight (default 100) controls relative priority when CPUs are contended rather than an absolute cap — a cgroup with cpu.weight=200 gets roughly twice the CPU time of a sibling at the default weight, but only when there’s actual contention forcing a choice.

Limiting memory

The memory controller’s memory.max is a hard ceiling — a cgroup that exceeds it faces the kernel’s OOM killer scoped to that cgroup, rather than a system-wide OOM event:

echo "512M" > /sys/fs/cgroup/myapp/memory.max
cat /sys/fs/cgroup/myapp/memory.current

memory.high sets a softer throttling point below the hard memory.max — processes are throttled and pushed to reclaim memory rather than killed outright, giving well-behaved applications a chance to back off before the hard limit forces a kill.

Limiting I/O

io.max caps read/write bandwidth or IOPS per block device, identified by major:minor device number:

echo "8:0 rbps=10485760 wbps=10485760" > /sys/fs/cgroup/myapp/io.max

Watching resource usage live

Each controller exposes its own .stat and .current files for monitoring without any extra tooling:

cat /sys/fs/cgroup/myapp/memory.stat
cat /sys/fs/cgroup/myapp/cpu.stat

systemd-cgtop, on systems running systemd, gives a live top-like view across the whole cgroup tree rather than reading individual files by hand:

systemd-cgtop

How this connects to systemd and containers

On a modern Linux distribution, systemd itself manages the top levels of the cgroup v2 tree — every service unit gets its own cgroup automatically, and Service directives like MemoryMax= or CPUQuota= in a unit file are really just a declarative front-end that writes to these same underlying cgroup files:

[Service]
MemoryMax=512M
CPUQuota=50%

Container runtimes work the same way one level deeper — docker run --memory=512m or a Kubernetes pod’s resource limits ultimately create a cgroup for the container and write the corresponding memory.max/cpu.max values, which is why cat /sys/fs/cgroup/.../memory.max from inside a running container is often the fastest way to confirm what limit is actually in effect, independent of whatever the orchestration layer claims was configured.