Skip to content
LinuxDeep Dive Published Updated 3 min readViews unavailable

Linux Pressure Stall Information: Measuring CPU, Memory, and I/O Contention Directly

A practical reading of Linux PSI totals, some/full averages, cgroup scope, polling triggers, workload correlation, and capacity decisions without guesswork.

Traditional utilization tells how busy a resource was; Linux Pressure Stall Information (PSI) tells how much runnable work lost time because CPU, memory, or I/O was unavailable. That makes pressure closer to user-visible contention, but the numbers need careful interpretation. PSI does not name the guilty process or prescribe a capacity threshold by itself.

Read some, full, averages, and totals separately

System-wide files live under /proc/pressure/:

cat /proc/pressure/cpu
cat /proc/pressure/memory
cat /proc/pressure/io

A typical line has avg10, avg60, avg300, and total. The averages express the recent percentage of wall-clock time represented by pressure over those windows. total is cumulative stall time in microseconds since boot for that pressure state, useful for calculating exact deltas over a monitoring interval.

some means at least one task was stalled on the resource while other tasks may still have progressed. full means all non-idle tasks in the measured group were stalled simultaneously. Full pressure is a strong sign of lost throughput, but its availability and meaning differ by resource; CPU exposes some contention rather than a useful full line in the same way as memory and I/O.

Do not add the moving averages or compare total directly between hosts with different uptimes. Collect monotonic total deltas and label reboots. An avg10 spike can disappear from avg300, while a growing total proves real accumulated delay even when a scrape misses the peak.

Scope pressure to the workload

With cgroup v2, resource-specific *.pressure files let operators measure contention inside a service or container boundary. System pressure can be low while one constrained cgroup experiences severe CPU throttling or memory reclaim. Conversely, a cgroup can look healthy while host-level I/O contention delays everyone.

Collect PSI alongside cgroup CPU, memory, I/O, and event counters; scheduler latency; reclaim and OOM events; device latency; and application request metrics. PSI answers “was useful work stalled?” Those other signals help answer why.

For example, memory some pressure plus rising major faults may point to working-set churn. Memory full pressure and cgroup memory.events increments can indicate a hard limit or reclaim collapse. I/O pressure with low device latency may instead be caused by throttling or a stacked device. CPU pressure with idle cores can reveal affinity, quota, or a single-thread bottleneck rather than host saturation.

Use event triggers for responsive control

The PSI interface supports registering a threshold and window on an open pressure file, then waiting with poll(). A trigger conceptually asks to be notified when at least a chosen amount of stall time occurs within a window. This avoids high-frequency scraping for latency-sensitive reactions.

A controller must debounce notifications and cap its response. Starting more replicas during shared I/O pressure can make the pressure worse; killing caches during memory reclaim can create additional reads. Record the trigger, action, and recovery, and retain an ordinary time series for post-incident analysis.

Establish thresholds from service objectives

Run representative load, correlate pressure windows with latency and throughput, and establish separate warning and critical conditions per workload. An interactive service may care about short CPU pressure; a batch pipeline may tolerate it while reacting strongly to prolonged full I/O stalls. Avoid copying one fleet’s percentage into another.

When an alert fires, compare system and cgroup PSI, identify the interval’s total delta, inspect limits and throttles, and correlate with application saturation. PSI is most useful as a common language connecting kernel waiting time to service degradation—not as a single score of machine health.

Related:

Sources:

Comments