Skip to content
FreeBSDDeep Dive Published Updated 7 min readViews unavailable

FreeBSD inotify: Native Watch Queues, Event Lifetimes, and Linux Compatibility Boundaries

Use FreeBSD 15's native inotify API for event queues, watch lifetimes, overflow recovery, capability-safe paths, and Linux compatibility boundaries.

FreeBSD 15 adds a native libc interface for applications built around Linux inotify. That is a useful source-compatibility layer, not a promise that every Linux inotify behavior or Linux binary ABI is identical. The inotify(2) manual is explicit about differences in hard-link and unlinked-file notifications, while FreeBSD 15 release notes separately describe the native interface and Linuxulator support. Porting and operations should keep those paths distinct.

The core model is a queue descriptor with watches registered against it. Read events from that descriptor, treat each record as a length-delimited structure, and regard overflow as loss of history rather than a recoverable single-file notification. That model is more important than memorizing a list of event constants: it determines how a watcher handles rename pairs, removed watches, process handoff, and resynchronization.

Start with the descriptor and watch scope

inotify_init1() creates a descriptor. IN_NONBLOCK makes an empty read return EWOULDBLOCK rather than waiting, and IN_CLOEXEC closes the descriptor at execve(). A watch is added with a pathname and event mask; FreeBSD also provides inotify_add_watch_at() to resolve a relative path against a directory descriptor:

int queue = inotify_init1(IN_NONBLOCK | IN_CLOEXEC);
int watch = inotify_add_watch_at(queue, rootfd, "incoming",
    IN_CREATE | IN_MOVED_FROM | IN_MOVED_TO | IN_DELETE);

This is a fragment, not a complete program: check both return values, retain the queue and watch identities, and define how the application will drain and close the queue. A watch descriptor is an integer unique within its inotify descriptor, not a durable pathname or global object identity. If the same object is watched twice on the same descriptor, the default operation replaces its mask; IN_MASK_ADD combines masks, while IN_MASK_CREATE requests an error rather than changing an existing watch.

Watching a directory reports events for the directory and entries in that directory. It is not a recursive subscription to every descendant. For a tree, install watches on the relevant directories and update that set when subdirectories are created, moved, or removed. Treat this as a cache of filesystem state: a concurrent rename or removal can occur while the application is adding or deleting watches, so a later scan may be needed to reconcile the tree.

The manual contrasts inotify with EVFILT_VNODE on kevent(2): a directory can be monitored without opening each object inside it. That is a useful difference when deciding which API suits a program, but not a claim that inotify replaces kqueue for process, socket, signal, timer, or other event sources. Keep the platform abstraction explicit if a program needs both filesystem events and broader event-loop integration.

Parse records as a queue protocol

A successful read() can return several event records. Each begins with struct inotify_event; the len field gives the size of the following name area, including NUL bytes and any padding. A name is absent for events about the watched object itself. Do not advance through a read buffer with strlen(): use the structure length, check that the header and the entire record fit in the bytes returned, and accept that padding may contain more than one NUL byte.

Rename operations can produce IN_MOVED_FROM and IN_MOVED_TO records. Their cookie values associate the two sides of a rename when both are observed, but a watcher must not assume that every cookie pair is present in the same read or that a process which begins watching mid-operation will see both records. Track unmatched cookies only for a bounded period and reconcile against the directory when evidence is incomplete.

Watch removal is also represented in the stream. IN_IGNORED means the watch has been removed, for example, after explicit removal, one-shot completion, or deletion of the watched object. Once it is generated, the descriptor no longer produces events for that watch; do not issue a second removal merely because the ignored record arrived. Keep watch state keyed by descriptor and clear it as these records are consumed.

The most important loss signal is IN_Q_OVERFLOW. FreeBSD documents that events beyond the per-descriptor queue limit are discarded and an overflow record appears. The missing sequence cannot be reconstructed from the queue. Mark affected state uncertain, rescan the watched directory or tree, rebuild the watch set as needed, and resume incremental processing only after the new baseline is established. A program that continues applying later events to a stale cache can silently miss a create, delete, or rename.

Start capacity review with read-only inspection of the documented tunables:

freebsd-version -ku
sysctl vfs.inotify.max_events vfs.inotify.max_user_instances vfs.inotify.max_user_watches

max_events bounds queued records per descriptor; the other settings bound descriptors and watches per user. A failure to create a descriptor, a failed watch, and an overflow are separate conditions. Preserve the exact errno and the queue state instead of treating every failure as “raise the inotify limit.” In particular, ENOSPC indicates the per-user watch limit, while EMFILE and ENFILE concern descriptor limits documented by the manual.

Treat Linux compatibility as a tested contract

FreeBSD’s manual documents two behavioral differences that can invalidate a Linux watcher assumption. First, accessing a file through any of its hard links can generate a FreeBSD event even when the accessed link is in an unwatched directory; Linux reports only accesses through the watched-directory link. If a watched directory itself contains multiple hard links to the same file, FreeBSD can report an event for each link there.

Second, after a file is unlinked, FreeBSD stops generating events for that file even if it remains open and is still accessed. The manual says this behaves as though IN_EXCL_UNLINK were always set, and that flag currently has no effect. Linux by default continues reporting events for the unlinked object. A portable application should not use post-unlink notifications as a durable indication of writes or lifetime.

These details are reasons to probe the target release, not to infer incompatibility from the operating-system name. Make a small filesystem fixture that watches one directory, performs create/write/rename/unlink operations, and records masks, cookies, names, and watch-removal events. Add hard-link cases if the application depends on inode identity or multiple names. Run equivalent tests on the Linux versions the program supports, then keep the observed contract with the portability layer.

FreeBSD 15.0’s release notes say the native implementation provides an API-compatible libc interface, while its Linuxulator section separately records Linux inotify system-call support. A FreeBSD-native program should use the native interface; a Linux binary under Linuxulator exercises the Linux compatibility path. Do not use a successful native test as evidence that an unchanged Linux binary, syscall numbers, or kernel ABI has been validated.

Use directory descriptors carefully in capability mode

In capability mode, the pathname-based inotify_add_watch() is not the right boundary: the manual documents ECAPMODE. inotify_add_watch_at() can instead resolve a relative name through a pre-opened directory descriptor. The documented errors include ECAPMODE when AT_FDCWD is used in capability mode and ENOTCAPABLE when a .. component would escape the supplied directory hierarchy.

This makes the API fit a service that already has a constrained directory capability, but it does not make an arbitrary path trustworthy. Open the intended root before entering capability mode, use a narrow relative name, reject path traversal in application policy, and test the exact FreeBSD release’s resolution behavior. Keep a separate test for a permitted child and a rejected escape; a watch installed successfully in an unrestricted process says nothing about the sandboxed case.

Acceptance checks for a watcher

Test a disposable tree and record the release, event mask, queue flags, errno values, and number of records returned per read. Cover initial directory inventory, child creation, same-filesystem rename, watch deletion, queue overflow recovery, descriptor close-on-exec, and process inheritance if the queue crosses a fork. In capability mode, repeat with a pre-opened directory descriptor and a rejected path escape. Never force a queue overflow on a production host just to confirm recovery logic.

A reliable watcher has an explicit “state uncertain” path, can rebuild from a directory scan, and does not confuse FreeBSD’s native API with Linuxulator behavior. Keep implementation claims tied to the release-specific manual: this interface first appeared in FreeBSD 15.0, and a different target release may have different documentation or support.

Related:

Sources:

Comments