Bash coproc: Bidirectional Pipes Without Building a Named-Pipe Protocol

Command substitution captures one command’s output after it finishes. A pipeline moves data in one direction while all stages run. Sometimes a shell script needs an ongoing helper that accepts requests and emits responses: a stateful parser, calculator, encoder, or protocol adapter. Creating two named pipes can work, but introduces filesystem cleanup, open-order deadlocks, and a private naming scheme.

Bash’s coproc compound command starts an asynchronous command with a pipe to its standard input and another pipe from its standard output. Bash publishes the descriptors in an array and the child PID in a variable. That is enough to build a small bidirectional worker, provided the script treats pipe framing, buffering, descriptor ownership, and termination as a real protocol.

A named coprocess publishes three handles

The named form is coproc NAME { command-list; }. Bash sets NAME[0] to the parent’s read descriptor for the coprocess output, NAME[1] to the parent’s write descriptor for its input, and NAME_PID to the asynchronous process ID.

coproc NORMALIZER {
    while IFS= read -r line; do
        printf '%s\n' "${line^^}"
    done
}

normalizer_pid=$NORMALIZER_PID

Without a name, Bash uses COPROC and COPROC_PID. A script with more than one helper should always name them. The array variables are shell state and can be unset when the coprocess terminates, so capture the PID and duplicate descriptors needed for later cleanup immediately.

Duplicate descriptors into owned variables

The descriptors Bash publishes have lifecycle rules tied to the coprocess. Duplicating them with Bash’s dynamic descriptor syntax gives the surrounding function clear ownership:

exec {to_normalizer}>&"${NORMALIZER[1]}"
exec {from_normalizer}<&"${NORMALIZER[0]}"

printf '%s\n' 'first request' >&"$to_normalizer"
IFS= read -r -u "$from_normalizer" reply
printf 'reply=%s\n' "$reply"

Check every redirection. If duplication fails, terminate and wait for the child rather than continuing with an empty descriptor variable. Do not expose these numbers through environment variables; file descriptor numbers make sense only inside the process that owns them.

Close duplicates exactly once. A descriptor leak into another long-lived child can keep a pipe open and prevent the coprocess from ever observing end-of-file.

The application needs a framing protocol

A pipe is a byte stream. It does not preserve one printf as one message. Line-delimited requests are convenient only when data cannot contain a newline or when escaping is defined. Binary data needs a length prefix or another unambiguous framing scheme.

For a line protocol, use IFS= read -r so backslashes and leading whitespace are not transformed. Define how an empty line differs from end-of-stream, how errors are represented, and whether one request always produces exactly one response.

Never generate shell code as the protocol and feed it to eval. Pass data, not syntax. A stateful helper can be written in awk, Python, Perl, or a compiled program while Bash handles only process lifecycle.

Pipes impose backpressure

The kernel pipe has finite capacity. If the parent writes requests without reading responses, the child can block on its output while the parent blocks on its next input write. Both processes remain alive and wait forever.

Use a strict request-response sequence, or create separate reader and writer logic with bounded queues. If responses can be large, drain output concurrently. A shell script is rarely the best place for a high-volume multiplexed protocol; switch to a language with explicit nonblocking I/O when complexity grows.

The child may also buffer its own standard output because it is connected to a pipe rather than a terminal. Configure line buffering in the child where supported or flush after each response. Bash cannot force an arbitrary program’s language runtime to flush.

EOF is a shutdown signal

To tell a reader loop there will be no more requests, close every parent copy of the write end:

exec {to_normalizer}>&-

while IFS= read -r -u "$from_normalizer" reply; do
    printf '%s\n' "$reply"
done
exec {from_normalizer}<&-

if wait "$normalizer_pid"; then
    status=0
else
    status=$?
fi

If the original array descriptor or another duplicate remains open, the child will not receive EOF. Inventory descriptors instead of adding a timeout that hides the leak.

Waiting is still required. End-of-file on output means no more bytes, not necessarily a successful child exit. Capture the wait status in a conditional so set -e does not terminate before cleanup and reporting.

stderr is a separate design decision

By default, the coprocess’s standard error follows the shell’s standard error. That is useful for diagnostics but can interleave with other jobs. Redirect it to a dedicated log inside the coproc command if messages need attribution.

Do not merge stderr into stdout with 2>&1 when stdout carries a machine protocol. One warning would become an invalid response. If errors must travel through the protocol, encode a response type and keep operational diagnostics separate.

Limit log size and avoid writing request contents that may contain secrets. The helper is long-lived, so a noisy warning loop can fill a disk while the parent continues to work.

Signals require parent-owned cleanup

The coprocess runs asynchronously, usually as a subshell environment around the command. Variable assignments it makes do not update the parent. The parent must decide how interruption propagates.

Install traps before starting the worker. On INT or TERM, stop sending, close the input pipe, signal the captured PID if it does not exit, wait for it, close output, and preserve the original signal outcome. A helper that spawns descendants may require its own process-group or supervisor design.

Do not use an unscoped kill 0 from an interactive shell. It targets the process group and can signal unrelated work. Record exactly what this script started.

Failure can arrive on any operation

A write can fail with a broken pipe because the helper exited. A read can see EOF before the expected response. wait can report a nonzero command status. Treat these as related evidence but preserve which boundary failed first.

Use timeouts only with a defined recovery action. Bash’s read -t can bound a response wait, but after a timeout the protocol may be out of sync. Usually the safe response is to terminate the helper, discard that session, and start a new one rather than send another request into an unknown state.

Test zero requests, one request, payload at the pipe-size boundary, embedded delimiters, child exit before reading, child exit after partial output, parent interruption, slow reader, slow writer, and a leaked descriptor in a spawned process.

Use coproc for one understandable relationship

A coprocess is ideal when one shell orchestration script needs one stateful streaming helper and both can agree on a simple bounded protocol. It avoids filesystem FIFO races and keeps the descriptors private to the process tree.

It is not an RPC framework. Once a script needs multiple concurrent clients, request IDs, retries after partial writes, authentication, or persistent recovery, a Unix socket and a purpose-built service will be easier to test.

Bash supplies the two pipes and asynchronous PID. Reliability comes from everything around them: owned descriptor duplicates, explicit framing, balanced reads and writes, EOF-driven shutdown, separate diagnostics, and an exit status that the parent actually waits to collect.

Related:

Sources: