FreeBSD bhyve PCI Passthrough: Host Reservation, IOMMU, and Rollback
Plan bhyve PCI passthrough by verifying VT-d eligibility, reserving the correct function for ppt, wiring guest memory, and preserving host recovery.
PCI passthrough gives a bhyve guest direct access to a host PCI device instead of an emulated device model. The guest can use the hardware’s native driver, but the host no longer owns the assigned function while it is reserved for the guest. That is a deliberate device-ownership change, not just another virtual-machine option. A mistaken bus/slot/function can detach the host’s only network adapter, storage controller, USB controller, or console path.
FreeBSD’s vmm(4) manual states that PCI passthrough requires VT-d support and is available only on amd64. The hypervisor supports other CPU architectures for virtual machines in current releases, but that does not make this passthrough path available on those architectures. Verify the exact release’s vmm(4), bhyve(8), and bhyve_config(5) before planning a deployment.
Inventory the device before changing ownership
Start with read-only inventory. Record the exact PCI address, vendor and device identifiers, driver currently attached, all functions on the same card, and which host workloads depend on them:
pciconf -lv
devinfo -rvDo not select a function by marketing name alone. A multi-function adapter can expose separate interfaces for storage, networking, audio, or management. Passing one function can affect a sibling function or shared device behavior, and firmware or the device’s reset capabilities can limit whether it is safe to assign. Confirm the bus/slot/function tuple from the current host, not an old output pasted into a template.
The ppt driver reserves a host device for passthrough. The documented loader variable uses bus/slot/function tuples, for example pptdevs="10/0/0"; that number is illustrative only. Add a reviewed tuple to /boot/loader.conf only after confirming the exact device and a recovery path. The host’s ordinary driver will not manage a function that ppt has claimed. A reboot is required for the boot-time reservation path, so this is a maintenance action with an expected host service interruption.
The vmm(4) manual also says the vmm kernel module normally needs to load through loader.conf(5), rather than only through kld_list in rc.conf(5), so it gets first refusal on devices it may claim. If a deployment has many functions, the manual documents pptdevs2 and pptdevs3 when the first loader variable would exceed its length limit. Do not copy those variables into a production configuration without checking the exact release and complete device inventory.
# Illustrative only. Replace with a reviewed PCI B/S/F tuple.
# Do not paste this line onto a production host unchanged.
pptdevs="10/0/0"After reboot, verify the binding with pciconf -lv and confirm the host still has the network, storage, and console access required to manage the VM. If the device did not bind to ppt, stop and inspect the boot logs and release-specific driver guidance. Do not repeatedly force driver changes on the only management or root-storage device.
Make the guest configuration match the reserved function
bhyve(8) documents pass-through backends by pptN name or by a host PCI selector. A device attached to ppt can be selected by that name. The guest memory must be wired with bhyve’s -S option when a passthrough device is configured. This is a correctness and resource-planning requirement: account for the host memory commitment and do not assume the guest’s normal paging behavior applies to wired memory.
A bhyve_config(5) PCI node names the guest-side PCI address and separates the device model from the host-side ppt name. In this illustrative example, pci.0.2.0 is the guest PCI bus/slot/function and ppt0 must be the already-reserved host function:
# bhyve_config(5) fragment; choose a guest address that is not already in use.
memory.wired="true"
pci.0.2.0.device="passthru"
pci.0.2.0.pptdev="ppt0"For a direct command-line configuration, the equivalent backend belongs in the guest’s -s device list and the command must include -S. Do not copy a full guest command from another system: PCI slot numbers inside the guest, firmware, ROM requirements, guest operating system, and device reset behavior all matter. Some guest firmware may ignore optional boot-order metadata or require a compatible ROM image.
Do not confuse PCI passthrough with a virtual network bridge, a virtio device, or USB redirection. Those expose different devices and trust boundaries. Before enabling a guest, decide whether the guest is trusted to issue arbitrary device requests and whether the data reachable through that hardware is appropriate for it. The cited FreeBSD 15.1 manual specifically requires VT-d for this path; that requirement does not make every card, firmware, or sibling topology safe. Validate the platform firmware and device isolation according to the hardware vendor and FreeBSD release.
Keep rollback independent of the guest
The recovery plan must work without the guest starting. Preserve a local console or out-of-band management path, a backup copy of the previous loader configuration, the PCI inventory, and exact commands needed to return the device to its host driver. Do not rely on a NIC being passed to the guest if that NIC is also the host’s only route to repair the configuration.
FreeBSD’s vmm(4) manual documents devctl operations for detaching ppt and attaching a host driver, and for assigning it back to ppt. Such a live reassignment is not proof that the guest can safely release or reacquire a device during active I/O. Use it only when the specific device, host driver, guest lifecycle, and release documentation support the planned transition. Stop the guest cleanly before changing ownership, and verify that no filesystem or network workload still depends on the device.
Validate on a disposable host or maintenance system
Start with a non-critical device and a disposable guest. Confirm that the host binding is expected, bhyve starts with wired memory, the guest detects the device, the guest driver initializes, and the workload can perform bounded I/O. Then stop the guest and verify that the host can reclaim the device if that is part of the operational plan. Test a host reboot and a guest shutdown separately. Preserve kernel messages, pciconf output before and after, guest logs, exact FreeBSD and guest versions, and the result of each recovery step.
Do not use the only copy of important data or the production host’s boot device for initial tests. A device visible inside the guest is not by itself proof of correct isolation or recoverable ownership. The production acceptance gate is a known device mapping, tested guest use, a host restoration path, and a tested reboot sequence that does not depend on the assigned hardware for management.
Related:
- How to Set Up a bhyve Virtual Machine Step by Step
- Fixing bhyve VMs That Refuse to Boot With UEFI Firmware
Sources: