FreeBSD Jails: Lightweight OS-Level Virtualization Done Right

FreeBSD introduced jails in version 4.0, back in 2000 — long before “container” was a word anyone used for this. A jail partitions the operating system’s global environment into isolated instances, each with its own view of the filesystem, process table, network stack, and set of privileges, all sharing a single running kernel. If that sounds like what Docker does on Linux, the resemblance is not a coincidence: the modern container ecosystem re-derived, piece by piece, ideas jails already had.

The core idea: partitioning, not emulation

A jail is not a virtual machine. There’s no hypervisor, no virtualized hardware, and no second kernel booting underneath it. Instead, the FreeBSD kernel itself enforces boundaries around a process tree: processes inside a jail can only see and signal other processes in the same jail, can only bind to IP addresses assigned to that jail, and are confined to a specific directory as their filesystem root.

# The classic, low-level way to start a jail
jail -c path=/jails/webserver \
     host.hostname=webserver.local \
     ip4.addr=192.168.1.50 \
     command=/bin/sh /etc/rc

Everything a jail can or can’t do is a kernel-enforced restriction, not a userland convention — even root inside a jail is a jailed root, stripped of the privileges that would let it escape (mounting filesystems, loading kernel modules, reconfiguring network interfaces outside the jail’s own address).

Building a jail’s filesystem root

Because a jail needs its own root filesystem, the first step is populating one — typically by extracting a base system distribution:

mkdir -p /jails/webserver
tar -xf base.txz -C /jails/webserver

# Give the jail its own resolver config
cp /etc/resolv.conf /jails/webserver/etc/resolv.conf

In practice, almost nobody drives jails with the raw jail(8) command directly. /etc/jail.conf centralizes definitions declaratively, and service jail start webserver handles the lifecycle:

# /etc/jail.conf
webserver {
    path = "/jails/webserver";
    host.hostname = "webserver.local";
    ip4.addr = "192.168.1.50";
    exec.start = "/bin/sh /etc/rc";
    exec.stop = "/bin/sh /etc/rc.shutdown";
    mount.devfs;
}

Thin jails and ZFS: don’t duplicate the base system

Extracting a full base system per jail wastes disk space fast once you have a dozen jails. The common pattern on ZFS is a thin jail: a read-only “template” dataset holding the base system, cloned per jail via zfs clone, so every jail’s root is a copy-on-write delta against the same shared base.

zfs snapshot zroot/jails/base@release
zfs clone zroot/jails/base@release zroot/jails/webserver

Updating the base system means updating the template and letting each jail’s clone inherit the change, rather than patching every jail’s copy of /usr and /lib individually — a pattern tools like ezjail and iocage automate.

VNET jails: a real network stack per jail

By default, jails share the host’s single network stack and are restricted to specific IP addresses on existing interfaces. VNET jails go further, giving each jail its own virtualized network stack — its own routing table, its own firewall rules, its own interfaces — connected to the host via an epair(4) virtual Ethernet pair.

vnet_jail {
    vnet;
    vnet.interface = "epair0b";
    exec.start = "/bin/sh /etc/rc";
}

This is the difference between “a jail that’s given an IP on the host’s interface” and “a jail that could plausibly run its own firewall and routing daemon as if it had a dedicated NIC” — closer in spirit to a Linux network namespace than to a chroot.

Resource limits: rctl

Isolation without resource accounting just means one jail can starve the others. FreeBSD’s rctl (resource limits) subsystem lets you cap CPU, memory, and process counts per jail:

rctl -a jail:webserver:memoryuse:deny=1G
rctl -a jail:webserver:pcpu:deny=50

rctl requires a kernel built (or configured) with options RACCT and options RCTL — on GENERIC kernels this is typically already enabled, but it’s worth confirming with sysctl kern.racct.enable before relying on it in production.

Where jails still have an edge

Because jails share one kernel with zero virtualization overhead, their performance overhead is close to zero — no VM exit traps, no virtualized device drivers. For homogeneous FreeBSD workloads (running many isolated instances of FreeBSD-native services on one host), that makes jails hard to beat on density and simplicity. The trade-off mirrors any shared-kernel design: a jail can only run FreeBSD, and a kernel vulnerability is a shared attack surface across every jail on the box — which is exactly why jails are complemented, not replaced, by bhyve when true cross-OS isolation is the requirement.