FreeBSD ARP Cache Operations: Trace IPv4 Neighbor Resolution on Ethernet

IPv4 packets on an Ethernet network need a link-layer destination address before the interface can transmit them. Address Resolution Protocol (ARP) maps an IPv4 address to an Ethernet address on the directly connected link, and FreeBSD keeps that mapping in a per-interface link-level table. When a remote application stops responding, an ARP table entry is one useful clue, not a complete reachability verdict: it says what mapping the local host currently has, not whether the router, firewall, remote service, or return path works.

The first operational question is which neighbor the host must resolve. If the destination is on the local subnet, the host resolves the destination’s address. If the destination is routed, it generally resolves the next-hop router instead. Looking for the final remote server in the local ARP cache can therefore send an investigation in the wrong direction.

Establish the route and interface first

Capture the destination, the route decision, and interface configuration before editing any cache entry:

freebsd-version -kru
route -n get 192.0.2.80
ifconfig -a

Replace the documentation-only address with the affected target. route -n get identifies the selected route and interface; inspect the gateway field to determine whether the host is on-link or reached through a router. Check the interface’s IPv4 address, prefix, link state, and VLAN or bridge membership. A correct ARP entry on the wrong interface does not fix an incorrect route.

ARP applies to the interface’s Ethernet-like link and its IPv4 neighbors. IPv6 uses Neighbor Discovery rather than ARP; use the documented ndp(8) tools for IPv6. A point-to-point route does not require ordinary Ethernet ARP resolution. Do not diagnose IPv6 loss by clearing an IPv4 ARP entry.

Read the cache with interface context

Display entries numerically to avoid hostname lookup affecting the view:

arp -an
arp -an -i em0

Replace em0 with the interface returned by the route lookup. The ARP utility accepts -i interface and -a; -n prevents name resolution. The cache is per-interface, so always keep the interface in the incident notes. A command run without -i can show entries from multiple links and make a duplicate private address look like one global entry.

Interpret flags and age according to the installed arp(8) manual. A dynamic entry may be absent before the first packet is sent, may expire after inactivity, or may be replaced after a legitimate network change. A permanent entry is a local configuration decision and can become stale when hardware or network topology changes. Neither a fresh-looking entry nor an arp -a row proves the current peer is healthy.

Generate one bounded request and immediately compare the table:

ping -n -c 3 192.0.2.80
arp -an -i em0

If route -n get shows a gateway, inspect the gateway’s address in the ARP table rather than expecting 192.0.2.80 to appear. A successful ping can still fail because ICMP is filtered while the application works; an unsuccessful ping can reflect remote policy instead of neighbor resolution. Keep the protocol’s role narrow.

Capture the exchange when the cache is ambiguous

When authorized on the affected LAN, capture a small number of ARP frames on the selected interface:

tcpdump -ni em0 -c 20 'arp'

This is a read-only observation, but captures may disclose network addresses. Protect the output and stop after the evidence is sufficient. A request with no reply suggests a link, VLAN, peer, address, or filtering problem; a reply from an unexpected MAC may point to an address conflict, proxy ARP, failover transition, or deliberate virtual MAC. A packet capture at one host cannot establish what a switch or remote NIC received.

Capture on the VLAN child or bridge interface that actually owns the Layer-3 address. A parent physical interface carrying tagged frames may show packets differently than an em0.20 VLAN interface. Check ifconfig and netstat -rn first, then select the interface at the correct layer. For a bridge, determine which member port and broadcast domain are in scope without assuming that the host-side bridge view includes all switch traffic.

If repeated traffic is needed, constrain the duration or packet count and write to a protected incident directory:

install -d -m 0700 /var/tmp/arp-case
tcpdump -ni em0 -c 100 -w /var/tmp/arp-case/arp.pcap 'arp'

Do not leave a broad capture running indefinitely. The capture is evidence for a specific window and interface, not a permanent monitoring system.

Distinguish stale state from an address conflict

If the neighbor recently changed hardware or failed over, a stale dynamic entry may be involved. Before deleting it, record the current mapping and confirm the expected mapping with the network owner or the device’s trusted inventory. Then remove only the specific entry and observe the next resolution:

arp -an -i em0 | grep '192.0.2.80'
arp -d 192.0.2.80
ping -n -c 3 192.0.2.80
arp -an -i em0

Substitute the actual next-hop address if the route is indirect. Deleting an entry interrupts packets until a new mapping is learned. In FreeBSD 15.1, arp(8) documents -i for displaying one or all entries and deleting all entries on an interface, but not for scoping a single-entry delete. If the same address could exist on multiple interfaces, do not assume arp -d -i em0 address is interface-scoped; confirm the installed tool’s behavior and use a planned, narrow action. Do not clear the entire table on a production router or server as a generic remedy: that can create a burst of requests and disrupt multiple active peers.

An address conflict often has a different pattern from simple expiration. Look for the same local IPv4 address being associated with changing hardware addresses, unsolicited announcements, duplicate-address warnings, or inconsistent responses from different switch ports. FreeBSD’s arp(4) documents net.link.ether.inet.log_arp_movements, which logs address changes and is enabled by default; inspect its live value and the installed manual before changing logging. A local capture may reveal replies but cannot identify the physical owner by itself. Confirm the switch MAC table, DHCP lease history, hypervisor virtual NICs, CARP configuration, or network access-control system as appropriate.

Do not install a permanent static entry just to suppress a conflict symptom. A static mapping can pin traffic to a wrong device and hide failover behavior. arp -s supports manually added entries, including temporary and published forms, but published/proxy ARP makes the host answer on behalf of another address. Use those forms only when the network design explicitly requires them and document persistence, interface scope, and rollback. For ordinary dynamic hosts, fix the duplicate address or upstream topology rather than turning a transient symptom into a hidden local exception.

Correlate the host with routed and virtual networks

When the route points to a gateway, the local ARP exchange proves only that the host can resolve the gateway on its attached link. It does not prove that the gateway has a route, that a firewall permits the flow, or that the destination is reachable. Continue with the next layer: inspect the gateway’s route and neighbor state through its own approved tooling, test the application from a separate client, and compare the reverse path.

In a jail or virtual machine, keep the network namespace explicit. A VNET jail has its own interface and routing context, so perform route and arp checks inside the jail as well as on the host when needed. A non-VNET jail may share host networking. CARP and bridge configurations can intentionally present a shared or virtual link-layer address, so changes in a MAC mapping are not automatically hostile or erroneous; compare against the failover design.

Use interface counters and link diagnostics to distinguish failed neighbor resolution from physical errors. ifconfig reports link state and driver statistics; netstat and device-specific tools can show drops or errors. A link can be up while a VLAN configuration is wrong, and an ARP reply can be visible while higher-layer packets are filtered. Make the packet path explicit in your notes rather than labeling every timeout “ARP.”

Static entries and careful cleanup

If a documented static mapping is required, use the arp(8) syntax appropriate to the site and understand its lifetime. The utility accepts arp -s hostname ether_addr [temp] [blackhole | reject] [pub]; entries are permanent unless temp is specified, while pub configures a proxy response. A published entry is not a harmless static pin. A blackhole or reject entry discards traffic and is not equivalent to a firewall rule with logging or policy semantics. The FreeBSD 15.1 arp(8) synopsis has no -i selector for adding a single entry, so verify the route/interface context that will use the mapping rather than assuming the display selector also applies to writes.

To restore normal dynamic resolution after an approved temporary entry, delete only the specific hostname/address and verify the route and new response. If the same address can exist in multiple interface scopes, account for the single-entry delete limitation described above before proceeding. If entries are loaded from a file with arp -f, review every line and keep the file under configuration management. Avoid storing stale MAC addresses in an rc script with no owner or expiry plan.

Acceptance criteria

An ARP investigation is complete when the route’s selected interface and next hop are known, the cache is read in that interface’s context, and a bounded packet observation or controlled refresh explains whether resolution succeeds. A fix is accepted when the expected IPv4-to-Ethernet mapping appears on the correct link, the route and higher-layer test succeed, and any static configuration has an owner and rollback.

Do not use cache manipulation to compensate for a wrong subnet, VLAN, route, duplicate address, or failed peer. ARP state is a symptom-bearing cache near the first hop. Start there, correlate it with the real topology, and make the smallest change that the evidence supports.

Related:

Sources: