Skip to content
Shell & TerminalHow-To Published Updated 4 min readViews unavailable

How to Use Named Pipes and Process Substitution Safely

FIFOs connect unrelated processes through a filesystem name; process substitution makes similar streams temporary, and both need careful failure handling.

An anonymous pipeline connects processes launched together. A named pipe, or FIFO, gives the kernel pipe a filesystem name so separately started processes can open the same stream.

Create a private FIFO

workdir=$(mktemp -d) || exit 1
fifo=$workdir/events
cleanup() {
    status=$?
    trap - EXIT
    rm -rf -- "$workdir" || printf '%s\n' 'warning: could not remove FIFO work directory' >&2
    exit "$status"
}
trap cleanup EXIT
trap 'exit 129' HUP
trap 'exit 130' INT
trap 'exit 143' TERM
mkfifo "$fifo" || exit 1

Using a private directory prevents another user from substituting a different object at a predictable path. The FIFO contains no stored file payload; writers and readers exchange bytes through the kernel while both ends are open.

Start both sides deliberately

Opening only the read end normally blocks until a writer opens the FIFO, and vice versa. Arrange startup so neither side waits forever:

consume_events <"$fifo" &
consumer_pid=$!

produce_events >"$fifo"
producer_status=$?
wait "$consumer_pid"
consumer_status=$?

Check both statuses. If a consumer exits early, the producer may receive SIGPIPE. If a producer never closes the FIFO, a consumer waiting for EOF will never finish. Multiple writers can interleave output beyond the system’s atomic pipe-write limit, so a FIFO is not automatically a record-safe message queue.

Use process substitution for one command graph

In Bash, Zsh, or Ksh, a command that insists on a filename can consume a stream with <(...):

diff <(sort left.txt) <(sort right.txt)

The shell supplies a /dev/fd path or manages a temporary FIFO. This is concise for processes launched within one shell command, but it is not POSIX sh syntax and its child statuses may require explicit handling.

Know when to choose another mechanism

FIFOs provide no durable queue, authentication, replay, schema, or backpressure policy beyond blocking writes. Use Unix-domain sockets for bidirectional local protocols, ordinary files for durable handoff, and a real message broker when delivery guarantees matter. Named pipes are excellent for bounded streaming-not a substitute for every IPC system.

Set permissions and reason about atomic writes

The private directory in the example is mode-restricted, so users who cannot traverse it cannot open the FIFO by pathname. If a FIFO must live in a shared directory, set a restrictive umask before creating it, or use a platform-specific mode option only after checking that implementation’s manual. Do not assume mkfifo -m is in the portable POSIX interface.

For multiple writers, a single write() no larger than the platform’s PIPE_BUF is atomic with respect to other writers under the documented pipe/FIFO contract; larger writes can interleave. Atomicity does not make a multi-write record transactional, and a reader still needs framing if it must distinguish records in a byte stream. Query the target’s limit and keep each encoded record within it, or serialize writers with a separate protocol.

Confirming a stuck FIFO isn’t the actual bug before debugging further

If a script or command using a FIFO appears to simply hang with no output at all, confirm first whether it’s genuinely stuck waiting for the other end to open the pipe (the expected, documented blocking behavior covered above) rather than a separate bug entirely - lsof or fuser against the FIFO’s path shows which processes currently have it open, immediately confirming whether a writer or reader is actually missing versus both ends being present but one side stalled for an unrelated reason.

Fanning one stream out to several independent readers

A single FIFO connects exactly one writer’s bytes to however many readers happen to have it open concurrently, but the kernel doesn’t duplicate the data per reader - bytes are consumed once, by whichever reader happens to read them, splitting the stream unpredictably across multiple simultaneous readers rather than giving each one a full independent copy. To genuinely fan one source out to multiple independent consumers, pair tee with several separate FIFOs instead of trying to share one:

mkfifo "$workdir/a" "$workdir/b"
producer | tee "$workdir/a" >"$workdir/b" &
consumer_a <"$workdir/a" &
consumer_b <"$workdir/b" &

tee writes its standard input to every named destination in full, plus its own standard output, which is what actually gives each downstream reader a complete, independent copy of the stream rather than a fragment of it.

mkfifo versus the older, lower-level mknod

mkfifo is the portable, POSIX-specified way to create a named pipe; the older mknod utility can also create one (mknod path p) but is a considerably lower-level tool historically used for device nodes as well, with a less consistent interface across systems and often requiring elevated privileges for its other, non-FIFO uses. Unless a script specifically needs mknod’s broader device-node capabilities, mkfifo is the clearer, more portable, and more narrowly-scoped choice for the FIFO-creation task this guide covers - it does exactly one thing, and its argument syntax doesn’t vary by what kind of node you’re asking for.

Related:

Sources:

Comments