Bash mapfile and readarray: Loading Lines Without a Fragile Read Loop

Bash’s mapfile builtin reads records from standard input into an indexed array. readarray is an exact synonym. For line-oriented input, it replaces a manual loop with one operation that preserves spaces, wildcard characters, and backslashes without asking the script author to reproduce every read rule.

It is a Bash feature, not POSIX shell syntax. A script intended for sh, Dash, or Bash 3.2 must use another design or declare a newer Bash requirement explicitly.

Read lines and remove only the delimiter

The common form is:

mapfile -t lines < input.txt
printf 'count=%d\n' "${#lines[@]}"
printf '<%s>\n' "${lines[@]}"

-t removes the trailing delimiter, which defaults to newline. It does not trim leading spaces, trailing spaces before the newline, glob characters, or backslashes. Always expand the array as "${lines[@]}" when each line must remain one argument.

Without -t, each complete input line retains its newline. That can be useful when reproducing a file byte-for-byte, but it surprises comparisons and command arguments.

An empty line becomes an empty array element. If the file ends without a final newline, the final partial line is still a record. Test both cases when a downstream format distinguishes an empty record from no record.

Avoid the pipeline subshell trap

This looks natural but often leaves the parent array unchanged:

producer | mapfile -t lines

Bash normally runs pipeline components in subshell environments. mapfile fills the array in that child, then the child exits. lastpipe can change behavior for the final component in a noninteractive shell with job control disabled, but relying on that option makes a script’s state depend on invocation settings.

Use process substitution so mapfile runs in the current shell:

mapfile -t lines < <(producer)

Process substitution is also Bash-specific. If portability is required, write the producer output to a temporary file safely, then redirect that file into a portable while IFS= read -r loop whose state is consumed within the same redirection scope.

Process substitution has an error-reporting caveat: mapfile can succeed even if producer later fails. When producer status is security- or correctness-critical, run it separately into a protected temporary file or design an explicit status channel rather than assuming $? belongs to both operations.

Select a descriptor, count, skip, or append

The complete builtin supports several controls:

mapfile -t -n 100 -s 10 records < input.txt

-n count copies at most that many records; zero means no limit. -s count discards records before copying. These options do not create random access into a huge file: Bash still reads and discards the skipped records.

-u fd reads from an already open file descriptor, which is useful when standard input belongs to the caller:

exec 3< input.txt
mapfile -t -u 3 first_batch
exec 3<&-

Without -O, mapfile clears the target array before assignment. -O origin begins assigning at an explicit index and preserves other elements.

mapfile -t batch_a < first.txt
mapfile -t -O "${#batch_a[@]}" batch_a < second.txt

Indexed arrays can have gaps, so ${#array[@]} is not always one greater than the highest index. The append expression is safe only when earlier operations kept the array dense. For sparse arrays, determine the intended origin explicitly.

Use a different delimiter without parsing text twice

-d delim uses the first character of delim instead of newline. An empty delimiter means a NUL byte. That is the safe partner for tools that emit NUL-separated pathnames:

mapfile -d '' files < <(find . -type f -print0)
printf '%q\n' "${files[@]}"

This preserves filenames containing spaces, tabs, quotes, glob characters, and newlines. Do not convert the stream to newline-separated text in the middle, or the safety property is lost.

The delimiter is a byte-oriented interface in the shell’s input stream. If records are structured CSV, JSON, or another grammar with quoting and escapes, use a parser for that format. mapfile separates records; it does not understand fields.

Callbacks are for progress, not a hidden parser

-C callback evaluates a callback after each quantum of records, controlled by -c. Bash appends the next array index and current record as arguments. If -C is supplied without -c, the default quantum is 5000.

The callback runs after reading the record but before assigning it. Quoting and evaluation rules make inline callback strings easy to get wrong. Prefer the name of a small function, keep it free of untrusted evaluation, and use it for progress reporting or bounded bookkeeping rather than transforming arbitrary content.

For very large input, remember that mapfile stores every selected record in memory. A streaming loop is the correct design when each line can be processed and discarded. mapfile is best when later logic genuinely needs random access, a count, sorting, or several passes over the complete record set.

Preserve status separately from records

mapfile reports whether its own read and assignment succeeded. It does not automatically carry the exit status of a producer hidden behind process substitution. If downstream work must run only on complete producer output, stage the records and status explicitly:

tmp=$(mktemp) || exit 1
trap 'rm -f -- "$tmp"' EXIT

if producer >"$tmp"; then
  mapfile -t lines <"$tmp" || exit 1
else
  status=$?
  printf 'producer failed: %d\n' "$status" >&2
  exit "$status"
fi

This also prevents consumers from acting on a valid prefix when the producer fails halfway through. Use a private temporary directory or an already open descriptor when the records are sensitive, and ensure cleanup signals cannot replace the intended status.

When inspecting the result, never flatten it with ${lines[*]} and then try to reconstruct boundaries. Iterate with quoted array expansion or print shell-escaped elements with printf '%q\n' "${lines[@]}". Compare the record count and, when relevant, a checksum of the original stream before transforming it.

Finally, declare the interpreter requirement at the top of the script. #!/usr/bin/env bash selects Bash through PATH; a fixed path selects one installed binary. Neither guarantees a recent enough version. Check the minimum Bash version when deployment includes older macOS or appliances, because a script that parses in modern Bash can fail before any fallback code runs.

Related:

Sources: