bhyve: FreeBSD's Native Type-2 Hypervisor

Where jails share a single kernel across isolated userlands, bhyve (BSD hypervisor) runs genuinely separate kernels — including non-FreeBSD guests like Linux or Windows — by relying on hardware virtualization extensions (Intel VT-x/EPT or AMD-V/RVI). It shipped as experimental in FreeBSD 10.0 and has been production-ready for years, powering much of the FreeBSD-based virtualization ecosystem, including large parts of illumos-derived and orchestration platforms built on top of it.

What bhyve actually is

bhyve is deliberately minimal compared to hypervisors like VMware ESXi or even QEMU/KVM: it’s a userland process (bhyve(8)) paired with a kernel module (vmm.ko) that exposes the CPU’s virtualization extensions. Each running VM is just a regular FreeBSD process from the host’s point of view, which means standard tools — ps, top, kill — work on it directly.

kldload vmm
ifconfig tap0 create

Building a minimal VM by hand

The classic way to understand bhyve is to build a VM’s command line manually before reaching for a wrapper script. bhyveload (for FreeBSD/BSD guests) or a UEFI firmware image (for anything else) bootstraps the guest, then bhyve itself takes over:

bhyveload -m 1024 -d /dev/zvol/tank/vms/guest1 guest1

bhyve -c 2 -m 1024 -H -A \
  -s 0,hostbridge \
  -s 2,virtio-net,tap0 \
  -s 3,virtio-blk,/dev/zvol/tank/vms/guest1 \
  -s 31,lpc \
  -l com1,stdio \
  guest1

Each -s slot,device flag attaches a virtual PCI device at a specific slot — virtio-net for networking, virtio-blk for disk, lpc for legacy platform devices like the serial console. This explicit, slot-by-slot device model is very different from QEMU’s more automatic defaults, and it’s deliberate: bhyve exposes exactly the virtual hardware you ask for and nothing more.

Networking: tap and bridge

A guest’s virtio-net device needs a host-side tap(4) interface to actually reach the network, typically bridged with a physical interface so the VM appears on the same network segment as the host:

ifconfig tap0 create
ifconfig bridge0 create
ifconfig bridge0 addm em0 addm tap0
ifconfig bridge0 up

UEFI guests and vm-bhyve

Booting anything other than FreeBSD/illumos-family guests generally means using the bundled UEFI firmware (BHYVE_UEFI.fd) instead of bhyveload, since bhyveload only understands BSD-style bootloaders:

bhyve -c 2 -m 2048 -H -A \
  -s 0,hostbridge \
  -s 2,virtio-net,tap0 \
  -s 3,virtio-blk,/vms/linux-guest.img \
  -s 29,fbuf,tcp=0.0.0.0:5900 \
  -s 30,xhci,tablet \
  -s 31,lpc \
  -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
  linux-guest

In practice, almost nobody assembles these command lines by hand for daily use — vm-bhyve (a ports-collection management layer) and sysutils/vm-bhyve handle VM lifecycle, templates, and networking configuration declaratively:

vm create -t debian -s 20G linux-guest
vm install linux-guest debian-installer.iso
vm start linux-guest
vm console linux-guest

Device passthrough

Because bhyve is built directly on the CPU’s virtualization extensions, it also supports PCI passthrough — handing a physical PCI device (a GPU, a NIC) directly to a guest via VT-d/IOMMU, bypassing host driver involvement entirely:

# /boot/loader.conf: detach the device from host drivers before boot
pptdevs="2/0/0"

bhyve ... -s 6,passthru,2/0/0 ...

This is what makes bhyve viable for workloads needing near-native hardware access from inside a guest — a use case a shared-kernel design like jails simply can’t offer, since jails don’t have separate kernels to hand hardware to in the first place.

Choosing between bhyve and jails

The practical dividing line is straightforward: if every workload is FreeBSD (or another BSD) and you want maximum density with minimal overhead, jails win. The moment you need a different guest kernel, stronger isolation guarantees against a kernel-level exploit, or GPU/device passthrough to a specific guest, bhyve is the right tool — and unlike some virtualization stacks, it’s already in the base system, no third-party hypervisor installation required.