Skip to content
LinuxDeep Dive Published Updated 6 min readViews unavailable

BPF CO-RE and BTF: Relocating Kernel Type Access at Load Time

Understand how BPF CO-RE uses BTF and relocation records to adapt compiled programs to target kernel layouts without shipping matching headers.

BPF Compile Once - Run Everywhere (CO-RE) addresses a narrow but painful compatibility problem: kernel structure layouts and fields can change between builds, while a BPF program may need to read those structures. CO-RE does not make every program portable or remove kernel-version constraints. It gives a capable loader enough type information to adjust supported accesses for the running kernel.

The object contains local BTF type descriptions in its ELF .BTF section and CO-RE relocation records in .BTF.ext. At load time, libbpf compares the program’s recorded access against target BTF, commonly exposed as /sys/kernel/btf/vmlinux, then patches eligible instruction operands. This is different from compiling a fresh object against each machine’s kernel headers.

The relocation contract

A CO-RE access records what the program meant to access, such as a named structure member, rather than treating one build’s byte offset as permanently correct. Relocation kinds can describe field offsets, sizes, existence, and related type properties. The kernel’s BTF is the target-side evidence; the compiled object’s BTF is the local-side description.

#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_core_read.h>

SEC("tracepoint/syscalls/sys_enter_openat")
int observe_openat(void *ctx)
{
    struct task_struct *task = bpf_get_current_task_btf();
    int pid = BPF_CORE_READ(task, pid);

    /* Emit pid through a bounded ring/perf event in a real program. */
    return pid < 0;
}

char LICENSE[] SEC("license") = "GPL";

This fragment shows the access pattern and required CO-RE headers, not a complete production observer: the context is intentionally unused, and the return value only keeps the read visible for illustration. A real tracepoint program must use the exact context definition for that tracepoint, report events through a bounded map, and handle helper and verifier constraints for its program type. Generate vmlinux.h from BTF on the build host, compile with debug/BTF information, and ship the object together with the libbpf-based loader. The build-host header supplies local type descriptions; it is not the target kernel’s runtime layout.

What the relocation can and cannot change

Field-based relocations can adjust a member’s byte offset or size, report whether a member exists, and describe properties such as signedness or bitfield placement. Type-based relocations can answer whether a type exists or provide a compatible target type’s size or ID. Enum relocations can test whether an enumerator exists and resolve its target value. Libbpf applies a supported relocation to the BPF instruction stream before verification. The relocation is based on BTF type/member identity and access path, not on guessing from the Linux version string.

This lets one object tolerate compatible layout changes, such as a structure member moving to another offset. It does not prove that the member still means the same thing, that a pointer is safe to dereference, that the field’s lifecycle is unchanged, or that the kernel’s behavior has identical semantics. A field can remain present while its interpretation or synchronization contract changes. Review the target kernel’s API and tracepoint documentation as well as the BTF match.

Optional fields need explicit feature-aware code. Libbpf provides CO-RE existence helpers that create an existence relocation; use the result to select a supported path and test the resulting object against both kernels where the field is present and absent. Do not replace a missing field with an invented offset or silently read unrelated bytes. If a field is required for correctness, fail load or disable that feature rather than pretending the observation is complete.

CO-RE is also distinct from helper, kfunc, map, program-type, and attach-point compatibility. A target can have matching task_struct BTF and still reject the program because a helper or kfunc is unavailable, the chosen tracepoint is missing, required privileges are denied, or the verifier cannot prove the access safe. Preserve verifier logs and treat a relocation failure as a compatibility result to diagnose, not as a reason to skip verification.

Build and inspect a relocatable object

The kernel must expose usable BTF for the target layout. A common target is /sys/kernel/btf/vmlinux; availability depends on kernel configuration and distribution packaging. On systems without it, the loader may have a separate target-BTF artifact, but a local build header by itself is not target evidence. Check the running system before blaming the source:

test -r /sys/kernel/btf/vmlinux
bpftool btf show
bpftool btf dump file /sys/kernel/btf/vmlinux format c | head -40

The dump is for inspection, not a substitute for type matching in libbpf. In a controlled build directory, generate the local C declarations from a known BTF source and compile with a matching architecture and toolchain:

bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h
clang -target bpf -O2 -g -c prog.bpf.c -o prog.bpf.o
llvm-objdump -h prog.bpf.o

This example assumes bpftool, Clang/LLVM with the BPF backend, libbpf headers, and a supported target architecture are installed. -g is used so the compiler emits BTF metadata and CO-RE records. Set the architecture-specific libbpf/compiler defines when the program or headers require them; do not copy an x86-specific define into an arm64 build. Confirm that .BTF and .BTF.ext exist in the object, then inspect the object BTF and relocation log using the toolchain versions used in deployment. Do not hand-edit generated offsets to silence a load failure.

For a release, record the source revision, compiler/LLVM and libbpf versions, build headers/BTF artifact, target architectures, kernel support floor, and representative verifier logs. Test the actual loader and attach path in a disposable host or isolated test VM before rollout. A successful compile only proves that the object was produced; it does not prove target BTF match, load permission, verifier acceptance, successful attachment, or correct event interpretation.

Verify the target and the complete load path

Inspect target BTF and the object before debugging a relocation failure:

test -r /sys/kernel/btf/vmlinux && echo "kernel BTF is present"
bpftool btf dump file /sys/kernel/btf/vmlinux format c | head

Then check the libbpf and kernel versions, object architecture, capabilities or BPF token policy, program type, attach mechanism, and verifier log. A missing target type may be a genuine compatibility boundary rather than a reason to disable CO-RE checks. Feature probing and graceful fallback are safer than assuming that every kernel exposes the same fields.

When a load fails, classify the stage instead of labeling every failure “CO-RE”: object parsing, target-BTF discovery, CO-RE relocation, BPF verifier, link creation, and attach policy are distinct stages. Preserve the full libbpf diagnostics and verifier output; check that the object architecture matches the host, the target BTF came from the kernel being tested, and the loader is not pointed at a stale BTF file. If the object loads but emits nothing, verify attachment, tracepoint availability, map capacity, event-reader health, and filtering logic. If it emits plausible but wrong values, validate the field’s meaning and the sample’s process/context semantics against the running kernel rather than assuming relocation success proves semantic compatibility.

CO-RE reduces the need to build against target headers, but does not remove the need for a compatibility matrix. Test against the oldest and newest supported kernels, include kernels with and without optional configuration and optional types, and keep the BPF object, loader, and verifier diagnostics together in the release evidence. For each supported target, acceptance should include: required BTF is available; all intended relocations resolve; the verifier accepts the program; the attach mechanism succeeds; known test events produce expected output; and the program can be detached cleanly. If a target lacks BTF or a required field/helper, make the feature unavailable explicitly and expose that state to operators.

Related:

Sources:

Comments