Skip to content
Shell & TerminalDeep Dive Published Updated 8 min readViews unavailable

Bash DEBUG Traps: Trace Commands Without Changing Their Meaning

Use Bash's DEBUG trap as a bounded diagnostic hook, understand function inheritance and extdebug behavior, and avoid altering the command being traced.

Bash’s DEBUG trap is a pseudo-signal that runs immediately before selected shell commands. It can expose the command text, call stack, and source location while diagnosing a difficult script. It is not a normal operating-system signal, and it is not a free-form replacement for set -x. A trap that writes to standard output, changes shell variables, runs commands that fail, or inherits into every function can alter the program it is meant to observe.

Use DEBUG only as a temporary or carefully bounded diagnostic facility. Keep the handler short, send output to a dedicated descriptor, avoid invoking application logic from it, and restore the previous trap and option state after the diagnostic scope. For routine command tracing, Bash’s xtrace option with a controlled BASH_XTRACEFD is usually simpler. The DEBUG trap is most useful when a trace needs source-level context or selective filtering that ordinary xtrace does not provide.

Understand when the handler runs

GNU Bash documents DEBUG as running before simple commands and several compound constructs, including for, case, select, arithmetic commands, and conditionals. It also runs before the first command in a shell function. This means a single logical line can result in multiple handler invocations, and a handler sees Bash syntax rather than a normalized application-level event. Do not assume that one trap call corresponds to one external process.

trace_debug() {
    local status=$?
    local source_file=${BASH_SOURCE[1]}
    local source_line=${BASH_LINENO[0]}
    local function_name=${FUNCNAME[1]}
    printf 'status-before=%s source=%s line=%s function=%s command=%s\n' \
        "$status" "$source_file" "$source_line" "$function_name" "$BASH_COMMAND" >&9 || :
    return 0
}

exec 9>>"$TRACE_LOG"
trap trace_debug DEBUG
run_workload
trap - DEBUG
exec 9>&-

This is a diagnostic sketch, not a turnkey audit logger. It treats trace writes as best-effort and deliberately returns success even if the log write fails; with extdebug enabled, a nonzero DEBUG-trap status can skip the next command. If trace failure must stop a workload, enforce that policy outside the observer hook rather than letting a logging write accidentally control execution. The sketch assumes the top-level script owns the DEBUG trap and descriptor state; trap - DEBUG clears the handler rather than restoring a prior one. A sourced library or instrumentation added to a caller’s shell must preserve and restore any existing trap and relevant option state. The arrays describe Bash’s current execution context, and indexing into the call-stack arrays must be tested in the exact function and source context being diagnosed. Validate that TRACE_LOG is set to a protected path before opening it, and decide what should happen if opening the log fails. The separate descriptor keeps trace output away from ordinary command output, but the log still needs appropriate permissions, rotation, and secret-handling policy.

BASH_COMMAND identifies the shell command associated with the trap; it is not a reliable record of fully expanded argument values. It may still contain sensitive literal text, variable references, or command substitutions from the source command. Xtrace is different: it prints expanded command words and can expose values supplied through variables. Treat both channels as potentially sensitive, and avoid logging either wholesale in production or in CI logs visible to broader audiences. A safer diagnostic may log a source file, line, and a small allowlisted command category while omitting argument data.

Function inheritance is opt-in

The DEBUG trap is not inherited by shell functions by default. Bash lets a function inherit it when the function has the trace attribute or the functrace shell option is enabled. set -T is the short form of set -o functrace; it also affects RETURN-trap inheritance. This setting can dramatically expand trace volume and handler execution scope, especially in scripts that call many library functions.

trace_debug() {
    printf 'command=%s\n' "$BASH_COMMAND" >&9
}
exec 9>/dev/null
set -o functrace
trap trace_debug DEBUG
outer() {
    inner
}
inner() {
    printf '%s\n' 'inside'
}
outer
set +o functrace
trap - DEBUG

The example toggles inheritance around a small operation, but that state change is shell-wide unless carefully scoped. In Bash, functions can use supported local-option mechanisms, but changing how a DEBUG trap propagates should be tested rather than assumed to behave like a local variable. If only one function needs tracing, use an intentional function-level tracing attribute or place the diagnostic at that boundary rather than enabling inheritance across the whole script.

Subshells and sourced files add more context. A sourced script executes in the current shell, so trap and option changes can affect its caller. A subshell has a distinct execution environment whose trap behavior should be verified for the Bash version in use. Avoid installing a global trap in a reusable library unless that library owns the shell lifecycle and documents the side effect. A helper intended to be sourced should save and restore caller state or expose an explicit opt-in tracing function.

Keep the handler from changing the observed status

Trap code runs in the shell that is executing the script, so its commands can affect shell state and observable status. Capture $? immediately if the previous status matters; any command in the trap can replace it. Use local variables in the handler to avoid clobbering application variables, but remember Bash dynamic scope means a local name can shadow a same-named variable in an active caller. Pick distinctive internal names and do not assign to BASH_COMMAND or stack variables.

Avoid running commands that may invoke the DEBUG trap recursively. Bash suppresses some recursive cases while a trap is executing, but relying on that behavior can make handlers difficult to reason about. Keep the body to a small set of builtins, or temporarily disable the trap if the operation demands external commands, then restore it in a cleanup-safe way. Do not call date, a logger, or a command formatter for every shell word in a hot loop without measuring the cost; the tracing machinery itself can dominate runtime.

With extdebug, DEBUG traps gain additional control-flow effects. Bash documents that a nonzero trap status can cause the following command to be skipped and that trap behavior is altered in other debugger-oriented ways. Therefore, a handler written only for logging must return success reliably when extdebug might be enabled. Do not combine a tracing hook with accidental false, arithmetic status, or a conditional whose final status can be nonzero. Inspect shopt -p extdebug in the diagnostic session and keep its setting explicit.

The trap can affect errexit as well. set -e is context-sensitive, and adding diagnostic commands may shift which command status becomes the last status in an expression. A trace handler should not be used to enforce security policy or to decide whether a command is safe. If it needs to block an operation, that is a separate authorization check in the command path, not an incidental DEBUG hook.

Prefer xtrace unless context requires more

set -x traces commands after expansion in a form designed for debugging; PS4 can include useful line or function context, and BASH_XTRACEFD can direct xtrace to a dedicated file descriptor. That is usually easier than a custom DEBUG hook because Bash manages the trace event format. It still can reveal secrets, and xtrace can expose expanded arguments, so enable it only in controlled runs with a protected output destination.

Use DEBUG when the investigation needs to filter particular command classes, observe a location before execution, or correlate the shell call stack with a failure. Do not maintain two independent full traces by turning on xtrace and DEBUG at the same time unless you need the duplicate data. The combined output can be expensive and can make ordering hard to interpret when subprocesses also write to logs.

For an intermittent failure, reproduce it in a disposable environment with the smallest affected function. Record the Bash version, shell options, whether the script was sourced or executed, and whether it ran in a subshell. Add a trace identifier to the output if parallel processes can share the log. Stop tracing when the issue is understood, remove temporary hooks from tracked scripts, and preserve only durable observability that does not leak commands or change execution semantics.

Validate trace behavior rather than just syntax

Run a tiny fixture that includes a simple command, a function call, a pipeline, a conditional, a sourced file, and a subshell. Check which events appear with the default settings, then enable functrace in a controlled block and compare. Test with extdebug off and on if the script can inherit user shell options. Confirm a failing traced command returns the same status with the handler installed as it did without the handler.

Also test a command argument containing a secret-shaped value and confirm the trace policy redacts or excludes it. Verify that handler output cannot contaminate machine-readable stdout and that descriptor failures do not break the application path unexpectedly. When the handler appends to a file, test permissions, disk-full behavior, and log rotation. A diagnostic facility is trustworthy only when it has its own failure policy and does not silently change the observed program.

The essential rule is separation of observation from control. A DEBUG trap can make shell execution visible, but its inheritance, status, and side effects are part of the program’s runtime. Keep it narrow, preserve caller state, protect its output, and remove it after use. For ordinary persistent tracing, prefer explicit application logging or Bash xtrace with a reviewed format.

Related:

Sources:

Comments