sed In-Place Editing: Portability, Backups, and Safe File Replacement

sed is a stream editor: it reads text records, applies a script, and writes the resulting stream. The portable interface is naturally “input in, transformed output out.” In-place options are implementation extensions with materially different command-line syntax, and even when they work, they do not make a multi-file edit transactional. Treating sed -i as a universal safe-edit primitive can break a macOS developer script, overwrite an original after a partial transformation, or replace a symlink unexpectedly.

For scripts that must run across Unix families, prefer writing transformed bytes to a temporary file in the destination directory, checking the transformation status, preserving required metadata, and then renaming the completed file. If using an implementation’s in-place extension, pin the implementation, spell its backup behavior explicitly, and test it on the exact operating systems in scope.

Separate the editing program from the mutation policy

For inspection or a generated artifact, let sed write standard output and redirect it deliberately:

sed -E 's/[[:space:]]+$//' < "$input" > "$output"

The command does not change input; the caller chooses where the result goes. Keep the sed program in a quoted argument or reviewed -f script file. Avoid interpolating untrusted values into sed source. If a value must be inserted into a regular expression or replacement, define how delimiters, backslashes, ampersands, and newlines are escaped; shell quoting alone does not escape sed’s language.

POSIX sed supports -e and -f, and the -E option for extended regular expressions in the current Issue 8 specification. Historical systems may differ, so a script with an older portability target should test that option and use the required expression dialect. GNU-specific commands such as -z or address extensions should be labeled as such. Portability means specifying a minimum utility contract, not merely writing syntax that happens to run on one laptop.

Why -i is not one portable interface

GNU sed accepts -i with an optional backup suffix. BSD-derived sed traditionally requires an argument after -i, with an empty string meaning no backup on several variants. Thus sed -i ‘s/a/b/’ file may work on GNU sed but be parsed as a missing suffix or consume the script as the suffix on BSD/macOS. Conversely, sed -i ‘’ … has a different interpretation on GNU sed. A shell script should not guess from the operating-system name if the actual binary may be GNU sed installed through a package manager.

The option can also change file identity. GNU sed documents an implementation that creates temporary output and renames it over the original; other implementations may use another strategy. Replacing a path can break hard-link relationships, change inode identity, affect ACLs or extended attributes, and alter symlink behavior. A process holding the old inode open may continue seeing old content while later opens see the replacement. These are observable semantics, not cosmetic details.

GNU sed documents a hazardous interaction: -n disables automatic printing, while -i replaces the file. If no explicit p command emits content, the result can be an empty file. Test the complete script and option combination against a copy, and use a backup suffix when the chosen implementation supports it and a backup is part of the recovery plan.

Use a checked same-directory replacement when the file matters

For a text configuration where content is the primary artifact, a temporary file in the same directory lets a successful transformation be renamed into place without first truncating the original:

file=$1
dir=$(dirname "$file") || exit 1
tmp=$(mktemp "$dir/.normalize.XXXXXX") || exit 1
cleanup() {
    status=$?
    trap - EXIT
    rm -f -- "$tmp" || printf '%s\n' 'warning: could not remove temporary file' >&2
    exit "$status"
}
trap cleanup EXIT
trap 'exit 129' HUP
trap 'exit 130' INT
trap 'exit 143' TERM
if sed -E 's/[[:space:]]+$//' < "$file" > "$tmp"; then
    chmod 0640 "$tmp" || exit 1
    mv -f "$tmp" "$file" || exit 1
else
    printf '%s\n' 'transformation failed; original not replaced' >&2
    exit 1
fi

This sample assumes a mktemp template form and a mode suitable for the file. It deliberately does not pretend that 0640 is right for every destination. Production code should validate arguments, preserve or set the intended owner and mode, handle ACLs and extended attributes, and fail closed if required metadata cannot be set. If the destination is a secret-bearing file, a permissive fallback is a security regression.

The temporary file should be on the same filesystem as the destination for a rename to be atomic with respect to pathname observers. A cross-filesystem move is normally copy-and-delete, so readers may observe partial content. Same-filesystem rename does not guarantee crash durability; a power loss can still lose data unless the file and directory are synced under the platform’s contract. Shell utilities do not automatically provide a complete transaction.

Preserve metadata deliberately

Replacing a file can change owner, group, mode, timestamps, ACLs, security labels, capabilities, extended attributes, hard links, and symlink behavior. Decide which properties are part of the deployment contract. A configuration renderer might intentionally install a new root-owned mode-0640 file rather than preserve arbitrary existing metadata. A source formatter may need to preserve executable bits. A policy may require rejecting symlinks and validating that the destination directory cannot be modified by an attacker.

Do not use a check-then-write sequence as a race-free symlink defense: another process can swap the path between the check and use. Shell-only sequences cannot fully prevent hostile concurrent mutation. For a directory writable by untrusted users, use a trusted private directory, a safe file API with no-follow semantics, or a privileged helper designed to avoid time-of-check/time-of-use races.

Metadata reads can race too. A script that stats the old file, transforms it, then applies previously observed mode bits could overwrite a concurrent administrator’s change. If concurrency matters, use a lock and revalidate identity or version immediately before replacement. Atomic rename prevents readers from seeing a half-written new file; it does not prevent lost updates.

Validate output before publishing it

Transformation success is not semantic validity. sed can exit zero after a pattern matched zero lines. If exactly one replacement is required, count or validate the expected change in a separate, unambiguous step, or use a tool that reports replacement counts. Keep the old file until validation succeeds, and make rollback behavior explicit. Do not make an empty or syntactically invalid configuration live because the command returned zero.

For multi-file changes, one rename per file is not an all-or-nothing transaction. If readers must never observe a mixed version, build a complete versioned directory tree and switch a single pointer or directory name under a platform-specific deployment protocol. Coordinate readers, backups, locking, and cleanup. A loop of ten in-place edits can stop after the fourth file and leave a partial migration.

Test implementation and failure modes

Run the exact command with the sed found in PATH, and record sed –version only where that option exists; BSD sed may not implement it. Use fixtures for a normal match, no match, metacharacters, backslashes in replacement text, empty files, final lines without newlines, a read-only directory, symlinks, hard links, and filenames with spaces. Verify content and metadata after the operation. Simulate a failure before rename and confirm the original remains intact and temporary files are cleaned up.

When the script targets macOS and Linux, test on both native sed implementations rather than relying on a developer’s GNU package. If GNU sed on macOS is an explicit dependency, invoke the intended binary and declare it in setup checks. For automated deployments, a formatter or configuration generator may be better than a one-line sed command because it can parse the target syntax and validate the resulting document.

Use sed for line-oriented transformations whose grammar is genuinely regular and whose replacement policy is understood. Keep portability, identity, metadata, validation, concurrency, and rollback in the surrounding workflow. That is what makes a transformation safe to ship, rather than the presence of an -i flag.

Related:

Sources: