FreeBSD GEOM Gate: Exporting Block Devices Without Pretending They Are Files

Copying files over a network and exporting a block device solve different problems. A filesystem protocol understands names, permissions, directories, and concurrent file access. A block protocol presents numbered sectors and leaves every filesystem invariant to the client that mounts them. FreeBSD’s GEOM Gate framework provides the second abstraction: it creates a GEOM provider whose I/O requests are serviced outside the local disk driver.

The provider can be backed by a remote ggated server through ggatec, or by a local userland daemon through ggatel. Once created, tools see a device under /dev/ggate* and can put a filesystem or another GEOM class on top. That composability is useful, but it also makes a network interruption look like a disk problem deep inside a storage stack.

GEOM preserves block semantics

GEOM connects storage consumers and providers in a directed graph. Partitioning, encryption, mirroring, filesystems, and physical disks attach through the same request model. Gate inserts a provider whose reads, writes, and flush-related operations are handed to a cooperating process.

This is not a network filesystem. Two hosts must not mount the same ordinary UFS or non-cluster-aware filesystem read-write merely because both can reach an exported device. Their buffer caches and allocation decisions will diverge and corrupt the shared metadata. Single-writer ownership or a filesystem designed for concurrent block access is mandatory.

The abstraction also does not make a regular file identical to a physical disk. Sparse-file allocation, host filesystem caching, discard behavior, stable writes, and error reporting can differ. Document what the backing object actually guarantees.

ggatec and ggated form the network path

ggated accepts requests for resources declared in its configuration. A client uses ggatec to create a local Gate device that forwards I/O to that server. Access can be read-only or read-write according to server policy and client request.

A minimal operational sequence resembles:

# Server: authorize only the intended client and backing resource in ggated.conf
service ggated start

# Client: create a read-only provider first for inspection
ggatec create -o ro storage.example.net /dev/ada1p2
ggatec list

# When finished, unmount every consumer before destroying the provider
ggatec destroy -u 0

Exact unit numbers and options should come from the command output, not from an assumed /dev/ggate0. Automation must capture the created unit, verify the requested access mode, and unwind partially completed steps after a failure.

The native transport is not encrypted

The ggated manual states that the connection between ggatec and ggated is not encrypted. An authorization rule is not confidentiality or integrity protection against an attacker on the path. Do not expose the daemon directly to an untrusted network.

Place it on a dedicated, filtered storage network or inside an authenticated encrypted tunnel such as IPsec or WireGuard. Restrict the listening address and firewall source, and expose only the necessary backing resource. A tunnel must preserve ordering and reliable delivery, but its existence does not repair unsafe multi-writer use.

Treat a compromised client as having the block-level access granted to it. Filesystem permissions inside the exported volume do not protect sectors from a client that can issue arbitrary writes. Use read-only exports for recovery and inspection whenever possible.

ggatel lets userland implement the backend

ggatel creates and controls a Gate provider served by a local process. This is useful when the backing store needs custom transformation, a test harness, or integration that does not belong in a kernel storage driver. The userland service receives requests and completes them through the Gate control interface.

That service now sits on the durability path. It must validate offsets and lengths, handle short I/O, preserve required ordering, report failures rather than fabricate success, and remain responsive under cancellation or shutdown. Returning success before data reaches the promised durability level can make a filesystem believe its journal is safe when it is not.

A prototype can intentionally inject latency and errors to test upper layers. A production backend needs explicit limits on outstanding requests, memory, retry duration, and shutdown time. Backpressure is safer than accepting an unlimited queue that disappears with the daemon.

Failure is observed as storage failure

If the remote server, tunnel, network, or userland worker stalls, consumers above GEOM may block waiting for disk I/O. The filesystem cannot distinguish a slow backing disk from a dead transport until the Gate layer returns an error or times out. Application latency can therefore spread far beyond the process that created the provider.

Choose timeout and retry behavior according to the upper layer’s recovery model. Blindly replaying writes after an ambiguous disconnect risks applying an operation twice or out of order. Conversely, failing immediately may force a filesystem offline even though a brief link interruption would have recovered.

Monitor both ends: provider state and queue depth on the client, daemon health and backing-device errors on the server, plus network loss and latency between them. Alerts should identify which layer first stopped making progress.

Stacking order changes the trust boundary

GEOM makes it possible to place encryption, partitioning, or redundancy above or below Gate. The order changes what crosses the network and where keys or failure domains live. Encrypting on the client before a remote Gate backend prevents the server from seeing plaintext sectors, while server-side encryption may protect media at rest but exposes plaintext to the server and transport endpoint.

Do not improvise a stack during recovery. Record the exact graph, sector size, metadata locations, key requirements, and creation order. geom inspection commands and ggatec list should be captured in routine diagnostics so operators know which provider feeds which consumer.

Snapshots and backups also belong below or above a clearly defined consistency boundary. Copying a live backing file while a client writes through Gate is no more consistent than copying a physical disk during writes unless the stack coordinates a snapshot or quiesce.

Recovery begins by stopping new writes

After an unexpected disconnect, do not create a second read-write client simply to see whether the data is present. Fence the previous writer, restore the secured transport, and decide whether the original provider can safely resume. If continuity is uncertain, expose the backing resource read-only and inspect it from one controlled host.

Test recovery before trusting the design: terminate ggated with writes in flight, break the tunnel, fill the backing filesystem, return short reads from a userland backend, and reboot each endpoint separately. Verify that errors propagate, mounts do not silently continue on stale assumptions, and the documented reattachment procedure works.

GEOM Gate is valuable because it exports blocks honestly. It does not pretend to coordinate files or hide the transport. Used with single-writer discipline, encrypted isolation, bounded failure handling, and a recorded GEOM stack, it can be a precise building block instead of a remote corruption mechanism.

Related:

Sources: