FreeBSD Networking Internals: Interfaces, Routing, and netstat

FreeBSD exposes networking as several distinct layers: drivers create interfaces, ifconfig(8) controls link and address state, the routing subsystem selects next hops, sockets bind protocol endpoints, and BPF consumers such as tcpdump(1) observe packets. Diagnosis is fastest when those layers are inspected separately instead of treating “the network” as one service to restart.

The stack descends from BSD networking work, but current behavior is defined by the installed FreeBSD release and its manual pages. Commands in an old BSD article may still look familiar while interface capabilities, routing syntax, and offload behavior have changed.

Interface naming: driver plus unit number

FreeBSD normally names an interface after its driver and discovery unit: em0, igb0, re0, and vtnet0 are representative examples. The name is evidence about the attached driver, not a guarantee about the marketing model of the NIC. Interfaces can also be renamed, and cloned interfaces such as bridge0, lagg0, vlan100, epair0a, and tap0 do not correspond one-to-one with physical PCI devices.

ifconfig -a
ifconfig -m em0
pciconf -lv
dmesg | grep -i -E 'ethernet|network|em0'

ifconfig -m lists supported media for a specific interface. pciconf -lv correlates PCI functions with drivers, while dmesg shows probe and attach messages. Do not infer a missing driver solely from an absent em0; another driver may own the hardware, firmware may have disabled it, or the interface may have been renamed.

For each interface, distinguish UP from RUNNING and status: active. UP is administrative intent; carrier and driver state determine whether traffic can actually flow. Check MTU, media, MAC address, IPv4/IPv6 addresses, prefix lengths, and error counters before changing configuration.

Configuring interfaces

Interactive changes made with ifconfig are immediate and normally disappear at reboot. Persistent host configuration belongs in /etc/rc.conf, preferably changed with sysrc so quoting and replacement are controlled.

# Interactive, non-persistent
ifconfig em0 inet 192.168.1.10/24
ifconfig em0 up

sysrc ifconfig_em0="inet 192.168.1.10/24"
sysrc defaultrouter="192.168.1.1"

Addresses are examples from private space. Confirm that the address is unused, the prefix is correct, and the gateway is reachable on-link. Applying an address or restarting netif over the only SSH path can terminate the administrative session; schedule console access and a rollback command.

For DHCP, the rc system invokes dhclient when the interface configuration requests it:

sysrc ifconfig_em0=DHCP
service netif restart em0

The targeted restart still disrupts that interface. SYNCDHCP is available when boot must wait for a lease; ordinary DHCP may continue startup asynchronously according to rc behavior. Inspect /var/db/dhclient.leases.*, /var/run/resolvconf, and logs when an address exists but DNS configuration does not.

IPv6 has separate rc variables and routing behavior. ifconfig_em0_ipv6="inet6 accept_rtadv" is appropriate only on a host meant to accept router advertisements. A router should not copy a host recipe without considering forwarding and RA policy.

Bridges, VLANs, and lagg

Cloned interfaces compose Layer 2 functions, but composition requires a topology. A bridge learns MAC addresses and forwards frames between member interfaces. The Handbook says that when the bridge host needs an IP, the address should be placed on the bridge rather than its members.

ifconfig bridge0 create
ifconfig bridge0 addm em0 addm tap0 up
ifconfig em0 up
ifconfig tap0 up

ifconfig vlan100 create
ifconfig vlan100 vlan 100 vlandev em0 up

ifconfig em0 up
ifconfig em1 up
ifconfig lagg0 create
ifconfig lagg0 laggproto lacp laggport em0 laggport em1 inet 192.0.2.10/24 up

For LACP, the switch ports must be configured in the matching aggregation and links must have compatible speed and duplex. One flow is normally pinned by the hash to one member; two 1-Gbit links do not make one TCP flow run at 2 Gbit/s. ifconfig lagg0 shows whether members are collecting and distributing.

VLAN parent MTU and hardware offloads can affect tagged traffic. Verify ifconfig -m, interface capabilities, switch VLAN membership, and packet size. A VLAN that passes pings can still fail larger traffic when an encapsulation path has an MTU mismatch.

Bridges can create loops. Enable and understand STP where redundant Layer 2 paths exist. Packet filters can see bridged packets at multiple hooks; the Handbook recommends choosing member-interface versus bridge filtering deliberately when direction matters.

Persistent equivalents use cloned_interfaces, ifconfig_bridge0, vlans_<parent>, create_args_<vlan>, and ifconfig_lagg0 variables documented by the Handbook and rc.conf(5). Generate them from the tested topology instead of translating commands mechanically.

The routing table

The routing table maps destination prefixes to gateways and interfaces. Inspect it numerically to avoid turning a route problem into a DNS lookup delay:

netstat -rn
route -n get 203.0.113.20

Destination        Gateway            Flags     Netif
default             192.168.1.1        UGS       em0
192.168.1.0/24      link#1              U         em0

Adding a static route for a specific destination network follows the same shape as the default route entry:

route add -net 10.0.0.0/24 192.168.1.254

# /etc/rc.conf — a persistent static route
static_routes="internal"
route_internal="-net 10.0.0.0/24 192.168.1.254"

Before adding a route, route -n get <destination> shows the kernel’s selected route, gateway, interface, and source context. Confirm the proposed gateway is directly reachable through an existing connected route. A static route whose gateway itself needs the static route is recursive misconfiguration.

IPv4 and IPv6 use separate default routes. A working IPv4 default does not diagnose IPv6, and a partially working IPv6 path can cause applications to appear slow before fallback. Inspect both address families explicitly with netstat -rn -f inet and netstat -rn -f inet6.

FreeBSD supports multiple forwarding information bases. The number of FIBs is a boot-time design choice configured through the documented net.fibs tunable. setfib(1) runs a process using a selected FIB, and jail parameters can select a FIB for jailed processes. FIBs separate route selection; they do not create interfaces, duplicate firewall policy automatically, or replace VNET.

setfib 1 netstat -rn
setfib 1 route -n get 203.0.113.20
setfib 1 ping -c 3 203.0.113.20

Validate every service from its actual FIB. Testing only from the host’s default FIB can prove the wrong route table.

netstat: more than just connections

netstat reports routes, protocol statistics, interfaces, and mbuf use. sockstat is usually clearer when the question is which process owns a listening or connected socket:

netstat -an              # all sockets, numeric addresses
sockstat -46 -l           # IPv4/IPv6 listeners with process ownership
netstat -s -p tcp        # per-protocol statistics
netstat -I em0 -w 1      # live interface throughput
netstat -m                # mbuf (kernel network buffer) usage

Treat counters as deltas. One historical TCP retransmission is not a current incident; a rate increasing during the failed request is evidence. Before tuning mbuf-related values, capture netstat -m, workload, kernel version, interface queues, and the relevant manual-page guidance. Arbitrarily raising a loader tunable can merely move memory pressure elsewhere.

tcpdump for packet-level inspection

tcpdump uses BPF to show packets at a chosen observation point:

tcpdump -i em0 -n port 443
tcpdump -i em0 -n -s 0 -w capture.pcap

Use a narrow capture filter, protect capture files because they may contain credentials or personal data, and record whether checksum offload makes outbound checksums appear invalid before hardware completion. On bridges, VLANs, jails, and NAT systems, capture on more than one relevant interface to locate the transition where a packet changes or disappears.

A layer-by-layer incident workflow

Start with link evidence: ifconfig flags, media, carrier, counters, driver messages, cabling, and switch port. Next verify the expected address and prefix and use arp -an or ndp -an for neighbor state. Then inspect route -n get for the exact destination and source context.

Test the next hop before a remote service. If IP connectivity works, separate DNS with drill or host from the application connection. Confirm the listener with sockstat, firewall state with the configured firewall’s tools, and finally packet flow with tcpdump.

For intermittent faults, collect synchronized timestamps, interface counters, protocol counters, routes, and captures from both ends. Do not restart networking until the transient evidence is captured. A restart can clear the state that identified a duplex problem, expiring neighbor entry, DHCP renewal, or flapping route.

Putting it together: diagnosing a connectivity problem

A methodical diagnosis follows the packet’s path and asks one bounded question at each layer. Interface state does not prove routing; a route does not prove neighbor resolution; a SYN leaving does not prove it arrived; and an accepted TCP connection does not prove the application protocol is healthy. FreeBSD’s tools expose each boundary directly, provided they are used before state-changing “fixes.”

Related:

Sources: