Linux Uprobes: ELF File Offsets, Probe Identity, and Bounded User-Space Tracing

An uprobe instruments an instruction in a user-space executable or shared object. It can reveal whether a particular application function runs without requiring that application to emit a log message. Linux’s tracefs interface identifies the probe by an object path and a file offset, not by the randomized virtual address printed in a debugger. Confusing those coordinate systems is a common reason for silent captures, rejected definitions, or probes at the wrong instruction.

This workflow uses tracefs uprobe events rather than a BPF program or a sampling profiler. It is intended for a controlled Linux laboratory workload with a known binary. Kernel configuration, architecture, compiler optimization, permissions, and local tracing policy determine availability. The examples are definitions to adapt to a verified object; the hexadecimal values below are illustrative and are not offsets that can be pasted into an arbitrary executable.

Choose a question and identify the exact object

Start with a question such as whether a particular parser function is entered during one test request. Avoid beginning with every function in a large library. Record the process identity, executable, architecture, build identity, and mapped object before creating an event:

readlink /proc/1234/exe
cat /proc/1234/maps
readelf -hW /absolute/path/to/lab-app
readelf -nW /absolute/path/to/lab-app
readelf -sW /absolute/path/to/lab-app

Replace PID 1234 with the controlled workload’s PID. readelf exposes ELF headers, notes, and symbol tables; a build ID, when present, helps associate a capture with the actual executable build. An executable replaced during an upgrade may leave an old object mapped in a running process. A pathname that now points to the new file is not sufficient evidence that it represents the old process’s instructions.

Stripped binaries may omit useful symbols, and inlined functions may have no independent callable instruction sequence. Optimized code can also create multiple implementations or transform a source-level function. Do not infer a stable probe location from the source name alone. For a lab, preserve a non-inlined function and matching debug artifacts, then inspect the resulting machine code.

Translate the symbol value into a file offset

For an ELF symbol whose value is a virtual address within an executable PT_LOAD segment, calculate its file offset using that segment’s program-header fields:

file_offset = symbol_value - segment_p_vaddr + segment_p_offset

Select the segment that actually contains the instruction bytes. Check the symbol lies in the file-backed range rather than memory that has no corresponding bytes in the file. Inspect program headers and disassembly together:

readelf -lW /absolute/path/to/lab-app
objdump -d /absolute/path/to/lab-app

For an illustrative symbol value of 0x1230 in a segment with p_vaddr=0x1000 and p_offset=0x1000, the file offset is 0x1230. If those segment fields differ, subtracting a fixed guessed base address gives the wrong answer. Position-independent executables and shared libraries are a reason to understand the ELF layout, not a reason to assume every symbol value already equals a file offset.

The running process’s mapping can corroborate which object and segment it uses, but ASLR’s runtime load address is not the value expected after the colon in a tracefs definition. Store the calculation and selected segment with the capture notes. Recalculate after a rebuild; even an apparently unrelated source change can move instructions.

Register only an owned probe

Confirm tracefs is mounted and the kernel supports uprobe events. The documented kernel configuration option is CONFIG_UPROBE_EVENTS; distribution policy may additionally restrict access to tracing controls. Inspect existing state before writing:

findmnt -t tracefs
sudo cat /sys/kernel/tracing/uprobe_events
sudo cat /sys/kernel/tracing/tracing_on

Coordinate ownership with other tracing users. Probe registration is shared state. In particular, writing an empty value to the uprobe event list can remove definitions owned by other operators. Use a distinct event group and append a single definition in a dedicated lab tracing session:

printf '%s\n' 'p:article_lab/parser_entry /absolute/path/to/lab-app:0x1230' |
    sudo tee -a /sys/kernel/tracing/uprobe_events >/dev/null

The path and offset must first pass the object checks above. This example does not capture arguments. Registers, calling conventions, compiler transformations, pointer validity, and data sensitivity all matter when adding fetch arguments. A copied %di example is not a portable argument decoder across Linux architectures.

The kernel exposes the event’s schema in its format file. Read that before writing a parser or filter:

sudo cat /sys/kernel/tracing/events/article_lab/parser_entry/format

An event registered successfully starts disabled. Registration, enablement, tracing state, and workload activity are separate conditions, so diagnose them separately rather than repeatedly changing the offset.

Filter the workload and bound the capture

Trace-event filters can use fields from the event format, including common task fields. For a single-threaded lab process, a filter on its thread ID can limit recorded events:

printf '%s\n' 'common_pid == 1234' |
    sudo tee /sys/kernel/tracing/events/article_lab/parser_entry/filter >/dev/null
printf '%s\n' 1 |
    sudo tee /sys/kernel/tracing/events/article_lab/parser_entry/enable >/dev/null

For a multithreaded process, common_pid identifies the task or thread that generated the event, so filtering only the process leader can exclude worker-thread hits. Build the intended filter from the actual thread inventory or use a tracing tool with suitable process-scoping semantics. A filter limits recorded evidence; do not assume it removes every instrumentation cost.

On an exclusively owned lab tracing session, enable tracing for one bounded workload interval, disable the event afterward, and export the resulting trace. Save the previous tracing state before changing it. Do not clear a shared trace buffer to obtain a cleaner screenshot. If the root tracing session is already in use, coordinate a supported isolated instance or use another tracing tool rather than taking ownership implicitly.

Read uprobe_profile as an additional hit-count diagnostic. Its counters do not by themselves prove that all corresponding records fit in the trace buffer or passed your filter. An empty exported trace can mean no execution, a wrong mapped object, a disabled event, a wrong thread filter, or overwritten records. Check those explanations before concluding the application never calls the function.

Distinguish entry evidence from latency evidence

An entry event establishes that the probed instruction was reached. A return probe can observe returns, but pairing entry and exit is a separate analysis problem. Recursion, concurrent threads, nonlocal control flow, process termination, and lost records can invalidate simplistic matching. An elapsed interval also includes scheduling effects; it is not automatically exclusive CPU time.

If the operational question is CPU consumption, use a suitable sampling profile alongside the function-entry evidence. If it is request latency, correlate the capture with a controlled request identifier and application timing. Avoid inferring a fleet-wide performance result from one instrumented test process. Compare baseline and traced workload behavior because instrumentation can perturb a hot path.

Remove the owned event and preserve reproducibility

Disable the event, remove its filter if it was changed, and delete the specific definition once no consumer uses it:

printf '%s\n' 0 |
    sudo tee /sys/kernel/tracing/events/article_lab/parser_entry/enable >/dev/null
printf '%s\n' '-:article_lab/parser_entry' |
    sudo tee -a /sys/kernel/tracing/uprobe_events >/dev/null

Restore any other session controls you changed and verify the owned definition is absent. Preserve the kernel release, binary build identity, offset calculation, filter, capture interval, lost-record evidence, and workload input with the trace. Those details make the result reproducible and prevent a future investigation from treating an obsolete hexadecimal address as a universal function location.

Related:

Sources: