Control Groups (cgroup v2) Explained: Limiting and Accounting for Resources

Control groups organize processes into a hierarchy for resource distribution, accounting, protection, and limits. Namespaces can change which resources a process sees; cgroups govern how tasks compete for resources. Neither mechanism alone is a complete container boundary.

cgroup v2’s single hierarchy makes composition and delegation more coherent than v1, but its filesystem is a kernel API with strict topology rules—not an ordinary directory tree to edit casually.

Identify the active hierarchy

Inspect mounts and the current process:

findmnt -t cgroup,cgroup2
cat /proc/self/cgroup
stat -fc %T /sys/fs/cgroup

cgroup2fs at the expected mount indicates v2. Hybrid systems can expose both versions, and containers may see a namespaced or read-only view. Do not mount another hierarchy over /sys/fs/cgroup on a system managed by systemd or a container runtime.

One hierarchy changes membership semantics

In v2, a process belongs to one cgroup in the unified hierarchy; all enabled v2 controllers use that placement. Resource decisions compose through ancestors: a child’s maximum cannot escape a parent’s limit, and siblings compete according to their configured weights and protections.

The root’s cgroup.controllers lists controllers available from the kernel at that point. A non-root subtree may expose fewer because its parent did not delegate them.

cat /sys/fs/cgroup/cgroup.controllers
cat /sys/fs/cgroup/cgroup.subtree_control

Availability does not mean enabled for children.

Controller enablement is top-down

To make a controller available to child cgroups, the parent writes +controller to its own cgroup.subtree_control. A child can only enable what its parent delegated. Writing +memory inside a leaf does not apply a memory limit to that leaf; it enables memory control for the leaf’s children.

Direct examples that create directories at the root often conflict with the service manager and require powerful privileges. On a systemd host, use units/scopes or an explicitly delegated subtree instead.

The no-internal-process rule matters

For domain controllers, a non-root cgroup generally cannot both contain processes and distribute domain resources to children. This “no internal process constraint” keeps resource ownership unambiguous. Move tasks into leaves before enabling controllers below an internal node.

Threaded cgroups are a specialized exception with their own valid topology and controller restrictions. Do not set cgroup.type without reading the kernel’s threaded-mode rules; an invalid transition can leave a management tree unusable.

systemd owns the host tree on many systems

systemd maps slices, scopes, and services onto cgroups. A transient scope is a safer experiment than raw root-tree writes:

systemd-run --user --scope \
  -p MemoryHigh=512M -p MemoryMax=768M \
  -p CPUQuota=50% your-command

User managers can only apply controllers delegated to them, and property support depends on systemd/kernel configuration. Use a disposable workload, not a production shell. Inspect the generated unit with systemctl --user status and systemctl --user show.

CPU bandwidth and weight solve different problems

cpu.max contains quota and period. max 100000 means no quota; 50000 100000 allows up to half of one CPU’s time per period for that cgroup, subject to ancestors. It is a bandwidth ceiling, not CPU pinning.

cpu.weight is proportional distribution among active siblings when contended. A group with twice a sibling’s weight is intended to receive a larger share, but topology, runnable tasks, CPU affinity, and ancestor weights affect observations. Idle capacity is normally usable; weights are not reservations.

Read cpu.stat for usage and throttling evidence before concluding a workload is CPU-starved.

CPU sets require valid masks

The cpuset controller constrains CPU and NUMA-node placement. Child masks must fit ancestors and need valid memory nodes. Effective files reveal what the kernel actually grants:

cat cgroup.path/cpuset.cpus.effective
cat cgroup.path/cpuset.mems.effective

Do not isolate host CPUs by editing these files ad hoc. Coordinate with systemd CPU affinity, scheduler isolation parameters, IRQ placement, and container orchestration.

Memory has protections, throttle, and hard limits

Key v2 files express different policies:

memory.current reports current accounted usage.
memory.low is best-effort protection; memory.min is stronger protection.
memory.high applies reclaim pressure/throttling and may be exceeded; crossing it does not invoke the OOM killer by itself.
memory.max is a hard limit. If usage cannot be reduced, cgroup OOM handling can occur.
memory.swap.max separately constrains swap where supported.

Do not set memory.max equal to a measured steady-state number. Allow for startup, caches, allocator behavior, kernel-accounted memory, and workload spikes; introduce memory.high first and monitor memory.events.

OOM behavior needs explicit evidence

At a hard-limit failure, victim selection and grouping depend on the constrained cgroup, ancestors, memory.oom.group, task eligibility, and kernel behavior. “The cgroup OOM killer kills exactly one offender and never affects the host” is too strong.

Monitor memory.events, memory.events.local, unit result, journal, and application logs. A host can still experience overall memory pressure when aggregate limits/protections are poorly designed.

I/O limits are per device

io.max keys limits by block-device major:minor and supports bandwidth/IOPS fields such as rbps, wbps, riops, and wiops. Determine the actual backing device:

findmnt -no SOURCE,TARGET /path/to/data
lsblk -o NAME,MAJ:MIN,TYPE,MOUNTPOINTS

Device mapper, LVM, encryption, network filesystems, buffered writeback, and layered storage can make the visible device different from where cost occurs. Validate with io.stat; do not paste 8:0 from an example.

PID limits stop fork explosions

pids.max limits the number of tasks in a cgroup subtree. pids.current, pids.peak, and pids.events show usage and limit hits where supported. Because threads count as tasks, an application with large thread pools can hit the limit without creating many Unix processes.

Leave enough headroom for supervision, graceful shutdown, diagnostics, and the workload’s legitimate peak. A limit so tight that a service cannot fork its recovery helper makes incidents harder to resolve.

Pressure metrics show contention, not only consumption

Pressure Stall Information appears through cpu.pressure, memory.pressure, and io.pressure. “some” and “full” totals describe time tasks were stalled, complementing usage counters. A service can remain under its byte/CPU quota while experiencing severe ancestor or device contention.

Compare leaf and ancestor metrics over the same interval. Resetting or sampling counters incorrectly can manufacture a misleading before/after story.

Moving processes has race and permission constraints

Writing a PID to cgroup.procs migrates a process, subject to permissions and topology. Forks can race with manual migration, and threads may require thread-aware handling. Service managers can move the process again.

Use an orchestrator/service-manager API for managed workloads. For delegation, grant the smallest subtree and documented controller files, not ownership of the host cgroup root. The kernel delegation model requires safe containment of migrations and controller changes.

Containers translate higher-level limits

OCI runtimes and Kubernetes ultimately program cgroups, but the exact path can be namespaced and the configured request may not equal an enforced limit. Inspect the host unit/runtime metadata plus relevant cgroup files. Inside-container observations can be useful but are not authoritative when the view is hidden or virtualized.

CPU requests/weights, CPU limits/quotas, memory requests/protections, and memory limits/maxima are different concepts. Verify the runtime/version mapping rather than assuming one universal conversion.

Build a safe acceptance test

Create a disposable workload through the manager, capture its cgroup path, and test below/above each boundary. Record cpu.stat, memory.events, io.stat, pids.events, pressure files, unit exit status, and ancestor configuration. Remove the transient unit and prove the tree disappears.

The policy is correct when the workload stays healthy below expected peaks, the intended controller responds under controlled excess, siblings/host remain stable, monitoring detects the event, and rollback removes every limit without orphaning tasks.

Related:

Sources: