FQ-CoDel on Linux: Flow Queueing, Delay Control, and Safe Validation

FQ-CoDel is a transmit queueing discipline that combines flow queueing with Controlled Delay active queue management. It addresses two related problems: one large flow monopolizing a FIFO queue, and a persistently deep queue adding latency while a link is busy. The fq_codel qdisc does not create bandwidth, guarantee a rate, or eliminate buffers in the modem, switch, NIC, or remote path. It manages the queue it controls.

That boundary is essential in production. A host may show a short software queue while packets accumulate in hardware offload rings or in a downstream access device. Likewise, a test that is not saturating the actual bottleneck cannot demonstrate bufferbloat control. Measure latency under controlled load and inspect the location where packets wait before changing qdisc parameters.

The algorithm has two cooperating parts

FQ-CoDel hashes packets into a configurable set of internal queues, generally using transport and network flow fields. Hashing is stochastic: distinct flows can collide into one bucket, so the algorithm does not promise a mathematically isolated queue for every connection. Each queue has its own CoDel state and FIFO order. The scheduler services queues with a deficit-round-robin style policy, using byte credits so large and small packets do not receive fairness merely by packet count.

CoDel observes packet sojourn time, the time a packet spends waiting in the queue. It uses a target and interval control loop to react to persistent delay rather than dropping immediately for every short burst. Depending on packet ECN capability and configuration, congestion signaling can use ECN marking or packet drops. The goal is not zero queueing delay; short transient queues can absorb bursts, while sustained delay should create feedback to senders.

The combination is different from a simple per-flow rate limiter. A flow that is alone may use available link capacity. Under contention, queued flows are scheduled separately, subject to hashing and qdisc policy. A transport flow hidden inside an opaque encrypted tunnel may be invisible to the classifier and share one outer flow queue with other traffic in that tunnel. Treat fairness as a useful scheduling property, not a per-user billing guarantee.

What the qdisc controls

Linux traffic control attaches qdiscs to network devices. An egress root qdisc determines how packets handed to that device are queued for transmission. On ingress, the receive path has different hooks and limitations; do not assume that adding an egress qdisc shapes incoming traffic directly. In practice, upload traffic is easiest to control at the local interface. Download shaping often requires an upstream bottleneck or an IFB-based redirection so packets are queued before the access link has already delivered them.

FQ-CoDel’s limit bounds queued packets, while memory_limit caps the total bytes queued when supported by the installed iproute2/kernel combination. flows changes the number of hash buckets. target and interval tune the CoDel control loop; quantum controls the byte credit used in flow scheduling; ecn and noecn select congestion marking behavior. Read the installed tc-fq_codel(8) manual and tc qdisc help fq_codel for the local implementation’s parameters and defaults. A manual page or default can change independently of a distribution’s network manager configuration.

Do not set target to the unloaded ping time or raise flows because a graph looks uneven. Choose parameter changes from measured bottleneck RTT, packet sizes, workload mix, and qdisc counters. More buckets use memory and do not remove the possibility of collisions. Lower packet limits can make burst absorption worse; higher limits can permit more waiting and memory use.

Inspect before you replace anything

Start with the device’s qdisc hierarchy and counters. These commands are observational:

ip -details -statistics link show dev "$IFACE"
tc -s -d qdisc show dev "$IFACE"
tc -s -d class show dev "$IFACE"

The interface may have a classful qdisc with FQ-CoDel below it rather than fq_codel directly at the root. Counters can reveal backlog, drops, overlimits, ECN marks, and packet/byte totals depending on the kernel and iproute2 version. They do not all have identical semantics across qdisc types. Compare counter deltas over a known interval instead of interpreting a lifetime total as an instantaneous rate.

Check the actual device and path. VLANs, bridges, tunnels, containers, and virtual NICs create multiple possible egress points. A qdisc on a guest’s virtual NIC may not control the host’s physical bottleneck. Hardware TX queueing, segmentation offload, and downstream equipment can move buffering beyond the software qdisc. Inspect NIC statistics with ethtool -S where available and compare them with tc output.

A controlled test without touching production networking

Do not replace a production root qdisc as a diagnostic shortcut. Save the current tc -s -d qdisc output, understand the manager that owns the interface, and test in a disposable network namespace or VM first. A namespace test needs a veth pair, an address plan, and cleanup; it is safest to use a script with a trap that deletes only resources it created. Avoid running an example interface name copied from an article against a live host.

A meaningful test offers a repeatable unloaded baseline, then saturates the suspected bottleneck while measuring both throughput and round-trip latency to a target beyond that bottleneck. Use multiple concurrent flows and at least one interactive or latency-sensitive flow. Record packet loss and retransmissions. Run enough repetitions to avoid mistaking Wi-Fi contention, CPU frequency changes, route changes, or a remote server limit for qdisc behavior.

Observe the qdisc backlog and drop/mark counters during the loaded interval. If ping latency rises but the local qdisc has no backlog, the queue may be elsewhere or the probes may traverse a different path. If one flow starves another, verify that both are classified into different buckets and are traversing the qdisc under test. A hashed bucket collision or a tunnel can invalidate a simplistic “one TCP connection equals one queue” assumption.

Shaping and hierarchy are separate decisions

FQ-CoDel is commonly combined with a shaping qdisc when the administrator needs to create a bottleneck slightly below a real access rate so that the controllable software queue becomes the bottleneck. That is a hierarchy design choice, not a property of FQ-CoDel by itself. A shaper imposes a rate; FQ-CoDel manages delay and flow scheduling among packets at its attachment point. Misconfigured parent/child handles or filters can mean the packets never reach the intended child.

When a link has asymmetric rates, shape each direction where its queue can be controlled. For downstream, a router at the edge may shape egress toward the LAN; a client cannot retroactively dequeue packets already queued by an ISP device. Use network-manager configuration hooks or a systemd unit only after the namespace/VM experiment is stable. Record the previous hierarchy and a tested rollback procedure before deployment.

FQ-CoDel is also not application prioritization. If an organization needs explicit service classes, access control, or guaranteed rates, use a designed class hierarchy and filters, then select the per-class qdisc intentionally. Avoid using packet classification to promise fairness at a granularity the visible headers cannot support.

Acceptance criteria

Record the kernel release, iproute2 version, device topology, qdisc tree, link rate, offload state, and network manager. Under a reproducible load, compare unloaded and loaded median and tail latency, goodput, loss, retransmits, and qdisc backlog. The interactive flow should remain within a declared latency objective while bulk flows use available capacity. Confirm that the result persists after a network restart only if persistence is part of the deployment.

Test bursty short flows, long bulk transfers, many concurrent connections, and an encrypted tunnel if those occur in production. Verify behavior under ECN-capable and non-ECN traffic when relevant. Ensure that qdisc replacement does not leave a partial hierarchy or remove existing policy. Keep a rollback command that restores the captured original configuration, and run it in the same test environment before enabling automated rollout.

The success condition is measured queue control at a known bottleneck, not merely seeing qdisc fq_codel in output. If latency remains high, map the packet path and locate the next queue. Fixing the wrong queue can produce a clean configuration with no effect on users.

Related:

Sources: