FreeBSD nullfs Overlays: Mount Shared Trees Without Copying Data

nullfs creates another pathname for an existing file-system subtree. It is a stackable filesystem layer, not a copy, Linux implementation bind mount, or snapshot. Reads and writes through the second path reach the underlying objects, subject to mount options and permissions. This can expose selected content to a jail, present a stable directory layout, or create a read-only view. It also means that a mistaken write through an alias can modify the original data.

The operational model is “one source, multiple names.” A nullfs mount does not duplicate blocks, provide independent capacity, or isolate content from changes through the source path. It has its own mount entry and can have a different device number reported by stat(2). Treat the alias as a namespace view and keep source, mount point, access mode, and teardown order explicit.

Verify the layer and paths

Record the release, mount table, and canonical paths before changing anything:

freebsd-version -kru
realpath /srv/shared/app
realpath /srv/views/app
mount -p

The source and mount point must exist and be the same kind of object: directory over directory or file over file. mount_nullfs(8) supports both, but mounting a directory over a file or the reverse is unsupported. Avoid relying on a path that resolves through an unexpected symlink; canonicalize and document the actual source and target.

Use a dedicated empty mount-point directory for a directory tree:

install -d -m 0755 /srv/views/app
mount_nullfs /srv/shared/app /srv/views/app
mount -p | grep '/srv/views/app'

The FreeBSD manual example is mount_nullfs /usr/ports /home/devel/ports. After mounting, inspect the view and compare it with the source using a few known files. Files that were already in the target directory are covered while the mount is active and become visible again after unmounting.

Read-only views are not immutable sources

To prevent writes through one alias, use a read-only mount option when supported by the installed utility:

mount_nullfs -o ro /srv/shared/app /srv/views/app
mount -p | grep '/srv/views/app'
touch /srv/views/app/should-not-exist

The touch is an intentional negative test and should fail; remove any test file only if a writable test view was deliberately used. A read-only nullfs mount constrains writes through that mounted path. It does not make the source filesystem read-only: a process using /srv/shared/app may still modify the same underlying files. If the goal is a consistent snapshot or immutable release, combine the view with an appropriate snapshot or deployment process rather than treating ro as copy-on-write.

Permissions and ownership remain properties of the underlying objects. Changing an owner through a writable nullfs view changes the source object as well. On ZFS, a dataset property or snapshot has separate semantics; mounting an alias does not create a new dataset or change quota accounting. Verify ownership and access from both paths as the actual service user.

Persist mounts with explicit dependencies

An /etc/fstab entry can describe a nullfs layer:

/srv/shared/app  /srv/views/app  nullfs  ro  0  0

Use the exact source, target, and options chosen for the service. fstab(5) records mount configuration, but the source path must exist when mounting is attempted. A parent filesystem or dataset must be available first. If the source is itself a mount, document and test order; otherwise an alias can be mounted over an empty directory before the real source arrives.

Test a configuration change without rebooting during a maintenance window. First unmount any existing layer, verify the source, and mount the intended entry explicitly. mount -a behavior includes exclusions and ordering rules; consult the installed mount(8) page rather than assuming every type is handled identically at boot. For a jail, coordinate nullfs mounts with its start and stop process so the service cannot start before paths are ready.

Keep source and target on separate lines in the runbook, with whether the mount is rw or ro, who owns the underlying files, and the command that verifies the live mount. Do not hide an essential source mount behind an undocumented alias. After reboot, operators need to distinguish “the path exists” from “the path is mounted from the correct source.”

Understand layering and nested mounts

Nullfs is implemented as a stackable vnode layer. mount_nullfs(8) explains that it duplicates a subtree into another location in the global namespace and permits filesystems to be mounted on the virtual copy without affecting the original mount layout. This can be useful, but nested mounts increase the number of layers that must be inspected and unmounted in order.

If /srv/views/app/data is a separate filesystem mounted below an alias, do not assume a command run at /srv/views/app describes every descendant. Compare the mount table and inspect the specific paths:

mount -p
df -h /srv/shared/app /srv/views/app /srv/views/app/data
stat -f '%m %d %i' /srv/shared/app /srv/views/app

Device numbers and mount points can differ across aliases. Capacity output must be interpreted with the underlying filesystem in mind; nullfs is not a separate allocation pool. Avoid scripts that deduplicate filesystems solely by device number when they depend on path-level mount policy.

mount_nullfs(8) documents metadata cache options, including nocache and cache. Do not change them to “fix slow nullfs” without workload-specific measurements. Disabling metadata caching can increase lock contention for some access patterns; forcing it may not be correct for every lower filesystem. Start with default behavior, measure path lookups and application latency, then compare controlled tests on the actual filesystem type and release.

Diagnose wrong content and stale aliases

When a service sees missing or outdated files, inspect the exact path from that service’s mount namespace or jail, then compare the host mount table. A common failure is that the target directory exists but the expected mount did not happen. Another is that the source was replaced or remounted after the alias was configured. A pathname being present proves neither the source nor the mount options.

Use mount -p for mount records and df/stat for the path under investigation. Compare a file’s checksum or metadata through source and alias when read-only verification is appropriate. For a service in a jail, inspect the jail’s configured mount rules and verify the path from inside it; host-side presence does not prove that the jail received the intended view.

If unmount says the layer is busy, find open files, working directories, or child mounts that refer to the target. fstat -f /srv/views/app can help identify processes holding objects from that mounted filesystem. Also inspect processes whose current directory is within the mount and check for nested filesystems. Do not use forced or deferred unmount as the first fix; that can hide a reference without resolving the service lifecycle problem.

Before changing a live mount, check whether applications have open files through the alias. A mount transition can leave existing descriptors referring to old vnode references while new path lookups see a different layer. Stop or quiesce the consumer according to its documentation, perform the mount change, validate representative paths, and restart only when consistency requirements are met.

Do not treat nullfs as a security boundary

Nullfs changes path visibility; it does not copy, encrypt, or independently protect the source tree. A read-write alias can alter original files. A read-only alias is a guard on one route, but does not constrain other routes to the same source. Jail isolation depends on the jail configuration and kernel boundaries, not on nullfs alone. Use the jail’s documented mount and privilege controls, expose only needed paths, and test from the jail process context.

Do not mount sensitive host paths into a less-trusted environment just because a nullfs option is read-only. Names, metadata, and readable contents may still disclose information. Ensure mount options, source permissions, jail configuration, and application identity align with the intended exposure.

Acceptance criteria

A nullfs deployment is healthy when source and target types match, the live mount table records the expected source and options, representative data matches through both paths, write behavior matches policy, and boot or jail startup ordering is tested. A teardown is complete when dependent processes and child mounts are stopped, umount succeeds without force, and the underlying target contents are understood.

The advantage of nullfs is path reuse without copying data. The operational cost is that every alias shares underlying state and storage. Record that relationship, verify it after boot and deployment, and never infer isolation or independent capacity from a second pathname.

Related:

Sources: