FreeBSD GEOM Multipath Operations: Map Shared-Storage Paths Safely

GEOM multipath presents multiple paths to the same storage device as one logical provider. It is useful when a host has more than one controller, fabric route, or target port to a shared LUN. The central requirement is that the paths really identify the same backing device. Two independent disks with similar sizes are not multipath paths; combining them can corrupt data.

FreeBSD’s gmultipath(8) offers manual and metadata-backed configuration. It provides path selection and failover operations, but it is not a storage-array design guide and does not guarantee that every HBA, target, transport, or active/active combination behaves identically. Validate the supported topology with the storage vendor and the relevant FreeBSD driver documentation before production use.

Prove LUN identity before creating a device

Inventory every path with the storage administrator and with host tools. Capture target identifiers, LUN number, serial or WWID, capacity, logical and physical sector sizes, controller port, fabric, and path state. FreeBSD device names such as da0 and da1 are assigned during discovery; they are not a reliable proof that two nodes refer to the same LUN.

camcontrol devlist -v
camcontrol inquiry da0 -S
camcontrol inquiry da1 -S
geom disk list
gmultipath status

Use the actual daN names reported by the host. Compare stable identifiers from the array and device inquiry output. If identifiers disagree or are unavailable, stop and resolve identity before attaching paths. The manual creation method checks media and sector sizes, but matching geometry is not sufficient evidence of same-LUN identity.

Confirm that neither path is independently mounted, partitioned for a different purpose, configured as swap, or already owned by ZFS, a mirror, or another GEOM class. For shared storage, coordinate with the array administrator before any operation that writes metadata. Maintain an out-of-band console path during testing so a path or controller change cannot strand remote access.

Understand manual and automatic configuration

With manual gmultipath create, no metadata is written to the underlying devices. The administrator must configure the path group again when it is needed, and additional paths are not discovered automatically. This mode is useful for controlled experiments or environments where external orchestration deliberately owns the topology. The startup procedure must recreate the same mapping every time.

The label method writes identifying metadata in the last sector of the first provider named, including a device name and UUID. The manual says the remaining providers are retasted and only paths with matching metadata are added. This is designed to avoid accidentally grouping unrelated providers, but it is still a metadata write. Never label a path until the intended LUN and its paths are confirmed, any existing data has been backed up, and the storage owner has approved the change.

A generic form of the manual command is:

gmultipath create lun01 /dev/da0 /dev/da1
gmultipath status
gmultipath list

The group name is an administrator-chosen label; the provider normally appears under /dev/multipath. Confirm the actual output before creating a partition or filesystem. Do not run newfs on an existing production LUN. In a shared SAN deployment the LUN may already contain a partition table and filesystem, and formatting it would destroy data.

Manual create and metadata-backed label are alternative initializers. Choose one method for a given group; do not run both in sequence against the same paths. For a metadata-backed group, the syntax is:

gmultipath label lun01 /dev/da0 /dev/da1
gmultipath status

This operation writes metadata. It is not an interchangeable synonym for create and should not be used on a LUN whose contents must remain byte-for-byte unchanged without a validated plan. The metadata occupies the last sector according to the manual; confirm that writing there is compatible with the array, partitioning scheme, and any other multipath software before proceeding.

Map the logical device into higher layers

Once the group is visible, use only the multipath provider for consumers that are intended to access that LUN. Inspect GEOM topology and filesystem state before adding mounts:

geom disk list
geom part list
gmultipath status
mount
swapinfo

If the LUN already has GPT partitions, inspect the provider and paths carefully, then confirm the expected partition node appears through the multipath device. Stable labels, UUIDs, or filesystem identifiers can be preferable to transient unit names in fstab, but use a naming method supported by the layer in this particular stack.

Do not mount the individual daN path while the same LUN is also active through /dev/multipath. Accessing one shared filesystem through multiple paths as separate devices can bypass serialization and create data corruption. The multipath abstraction exists to ensure consumers use one logical device while the kernel manages multiple physical paths beneath it.

If the LUN is newly provisioned and intentionally blank, creating a partition table or filesystem is a separate destructive change. Verify the exact multipath provider and obtain approval before commands such as gpart create, gpart add, or newfs. Capture the LUN identifiers and the command output in the change record.

Observe and operate path state

The manual exposes operations including add, remove, fail, restore, rotate, prefer, getactive, configure, and status. Their exact semantics and flags are version-specific; consult gmultipath(8) for the installed FreeBSD release before using them. Start with read-only status and inventory commands. A planned failover test should be coordinated with the storage team and performed while the application has a verified recovery path.

Check which path is active and whether all expected members are present:

gmultipath status
gmultipath list
gmultipath getactive lun01
dmesg | tail -100

A path remaining visible is not enough to prove application I/O can continue over it. Collect storage-array port state, HBA logs, CAM messages, application latency, and errors before and after the test. Exercise one path at a time under an approved window, observe active path selection, and verify recovery when the path is restored. Do not pull cables or disable a fabric port as a casual diagnostic.

The -A option enables Active/Active mode, while -R enables Active/Read mode; without them the documented default is Active/Passive. Do not enable a mode based on intuition. Active/Active requires the array and transports to support concurrent path use correctly; Active/Read behavior depends on the target’s path model and failover semantics. Verify with the array vendor whether ALUA, asymmetric access states, or vendor-specific coordination is involved. The existence of a command-line flag alone does not establish correct interoperability.

If a path fails, preserve the evidence and inspect the device, controller, link, and array state. The fail operation changes logical path handling but cannot repair a cable, HBA, target, or LUN. Restore a path only after the underlying fault is corrected and the array has returned it to the expected state. Avoid repeatedly rotating or preferring paths while the topology is unstable, because that can complicate incident analysis.

Persistence and boot ordering

Automatic metadata discovery and manually recreated groups have different boot requirements. The gmultipath manual documents geom_multipath_load=“YES” in loader.conf when the class is a loadable module. Confirm the current module name and that storage discovery occurs early enough for the root filesystem or application mount that depends on it. Do not assume a userland service can assemble a device before the root filesystem has mounted.

For non-root data LUNs, test a complete reboot under controlled conditions. Verify all paths, the group, partitions, filesystem, mount, and application startup order. A group appearing after a manual command is not proof of unattended boot recovery. Use rc dependencies and fstab configuration only after the provider’s stable name and creation behavior have been verified on the actual host.

For shared or clustered storage, coordinate fencing and ownership. gmultipath aggregates paths from one host; it does not prevent another host from writing the same filesystem concurrently. Cluster-aware filesystems, reservations, fencing, and application-level ownership are separate concerns. Never interpret path redundancy as host-level or data-level high availability.

Change and recovery checklist

Before a change, record the LUN ID, every host path, device firmware and driver versions, current consumers, array state, and a backup or snapshot appropriate to the data. Establish how to revert without destroying metadata or disconnecting the active application. Confirm console access and assign a rollback owner.

Afterwards, verify that the group contains only paths for the intended LUN, no duplicate consumer is using a raw path, the expected partition and filesystem are visible, and application transactions succeed. Record the active path, the outcome of each planned failover, any path-recovery delay, and storage-side telemetry. A clean gmultipath status is necessary but not sufficient for accepting an end-to-end storage service.

Do not destroy a group or clear metadata simply to make stale output disappear. First unmount and stop all consumers, confirm no open references, preserve status, and consult the manual for the exact destroy or clear semantics. If a LUN is being retired, use the storage change process to remove host mappings and array presentations in the correct order. Multipath operations can affect access to all data on that LUN.

Related:

Sources: