Fixing FreeBSD Jail Networking When VNET Jails Can't Reach the Network

A VNET jail has its own interfaces, addresses, routing table, sockets, and firewall context. It reaches the network only through plumbing the host explicitly creates: commonly an epair(4) connected to a bridge, or a routed/NAT design with forwarding. A jail that starts successfully proves only that the jail lifecycle completed; it does not prove that either end of the virtual link, a default route, DNS, or firewall policy is correct.

Confirm the jail model and define the failed hop

Ask jls(8) for the jail’s actual parameters rather than inferring them from a manager’s template:

jls -j web vnet host.hostname path
jexec web ifconfig -a
jexec web netstat -rn

Replace web with the name or JID. A VNET jail should see its own loopback and assigned interface. A host-networking jail instead uses addresses constrained through ip4.addr or ip6.addr in the host stack; epair and in-jail route troubleshooting does not apply to it.

Define connectivity as sequential tests:

jexec web ping -c 1 192.0.2.1
jexec web route -n get default
jexec web ping -c 1 1.1.1.1
jexec web drill example.com

Use the real local gateway and an approved external test address. Failure to reach the gateway implicates local interface, Layer 2, or subnet policy. Reaching the gateway but not the Internet points to routing, forwarding, NAT, or upstream filtering. Reaching an IP but not a name is DNS, not VNET transport.

ping can mislead because raw sockets require jail permission and ICMP may be filtered. If it reports “Operation not permitted,” check allow.raw_sockets and use TCP tests such as nc or fetch before concluding the network is down. Grant raw-socket access only when needed.

Verify both epair ends and their ownership

An epair acts like two connected Ethernet interfaces. One end remains on the host; vnet.interface moves the other into the jail’s VNET. Names are not guaranteed to be epair0a and epair0b forever, especially when lifecycle scripts create multiple pairs or rename them.

ifconfig -a
jexec web ifconfig -a
ifconfig bridge0

Confirm the host end exists and is UP, and the jail end exists inside the jail with the intended address and mask. If both ends remain on the host, the vnet.interface move failed or ran in the wrong order. If the host end disappears after restart, inspect exec.prestart and exec.poststop: a stale interface from a failed start can make the next ifconfig ... create fail, while an overbroad cleanup can destroy an interface another jail uses.

The FreeBSD Handbook’s VNET example creates the epair before jail startup, adds its a end to a bridge, passes the b end with vnet.interface, configures the address and route inside, and removes the host end after shutdown. Lifecycle commands should be idempotent enough to recover from a partial start and should log failures to the jail console log.

Determine whether the design is bridged or routed

Do not add the physical NIC to a bridge by reflex. In a bridged design, the jail appears on the same Layer-2 segment and IP subnet as the host’s external network. The bridge normally includes the physical interface and each jail’s host-side epair:

ifconfig bridge0
ifconfig bridge0 addm epair154a up

Verify the physical interface is already a member before adding anything. The jail’s IP must be unique on that LAN, its prefix must match, and the upstream switch must allow additional source MAC addresses on the port. Wi-Fi client interfaces and provider networks often cannot transparently bridge arbitrary MAC addresses, making a routed design more appropriate.

In a routed design, the epair uses a distinct subnet. The host routes between that subnet and the outside, with gateway_enable="YES" for IPv4 or the corresponding IPv6 forwarding setting. Private addresses typically need NAT at the external boundary. The physical interface does not have to join the jail bridge. A default route inside the jail points to the host-side address, not directly to an unreachable external router.

Inspect the host’s routing and forwarding state:

netstat -rn
sysctl net.inet.ip.forwarding
route -n get 192.0.2.10

Substitute the jail address. Also ensure the upstream network has a return route to the jail subnet when NAT is not used. Outbound packets can leave successfully and still fail because replies follow a different router.

Trace packets across each boundary

Run simultaneous captures on the jail end, host epair, bridge or routed interface, and external interface as needed:

jexec web tcpdump -ni epair154b
tcpdump -ni epair154a
tcpdump -ni em0 host 192.0.2.10

The interface names and filter are examples. If a request appears inside the jail but not on the host epair, investigate interface state or VNET assignment. If it reaches the bridge but not the physical NIC, inspect bridge membership and filtering. If it leaves the host and no reply returns, inspect upstream route, NAT, switch policy, and duplicate addresses. If the reply reaches the host but not the jail, inspect stateful firewall rules and the return path.

Check ARP or NDP at the failing hop. Repeated unresolved neighbor requests indicate Layer-2 reachability, VLAN, mask, or bridge problems; they are not fixed by changing DNS.

Audit PF and DNS without disabling protection

PF can filter traffic on physical, epair, and bridge-related paths depending on rules and pfil configuration. Inspect the loaded rules and counters, not only the source file:

pfctl -sr -v
pfctl -sn
pfctl -si
pfctl -ss

Use labels and pflog to see the exact blocking rule. Temporarily disabling the entire firewall on a remote host is risky and removes useful evidence. Add a narrow logged diagnostic rule for the jail address and expected interface, then remove it after the test. In routed private-address designs, confirm the NAT rule matches the actual egress interface and source subnet.

If IP connectivity works, inspect /etc/resolv.conf inside the jail. A loopback resolver address such as 127.0.0.1 refers to the jail’s loopback, not a resolver running only on the host. Configure a reachable resolver or intentionally run one inside the jail. Confirm UDP and TCP port 53 where required.

Make the repair survive the jail lifecycle

After identifying the failed hop, put interface creation, membership, address, route, and cleanup in one authoritative configuration—jail.conf, rc settings, or the jail manager—not a mixture of manual commands. Restart the jail and verify that no stale epairs accumulate, bridge membership returns, routes are correct, and PF state is recreated.

Test host-to-jail, jail-to-gateway, jail-to-external-IP, DNS, and a real application socket. Then restart networking or reboot the host in a controlled window. A VNET repair is complete only when the lifecycle reconstructs the same working topology without shell history.

Related:

Sources: