FreeBSD ZFS TRIM Operations: Reclaim Free Space Without Surprises

ZFS TRIM tells a lower storage layer which ranges in a pool are no longer allocated. That information can help SSDs, thin-provisioned LUNs, and file-backed virtual devices reclaim capacity or manage their allocation more efficiently. It is a storage optimization signal, not a filesystem check, a pool repair, a secure erase procedure, or a substitute for redundancy and backup.

FreeBSD exposes two different controls: the pool property autotrim for periodic processing of newly freed ranges, and the zpool trim command for an explicit on-demand pass. Choose between them based on the actual device path, workload, and maintenance policy. Do not assume that every controller, USB bridge, hypervisor, or virtual disk passes discard requests through correctly.

Start with the device stack, not the property

Record the pool topology and the path used by each leaf vdev before changing trim behavior:

zpool status -v tank
zpool list -v tank
zpool get autotrim tank
camcontrol devlist
geom disk list

The last two commands provide useful device inventory on CAM-backed systems, but they do not prove that a particular transport supports discard end to end. Trace the complete path: ZFS vdev, GEOM provider, controller or HBA, transport, virtual-storage layer if present, and physical or remote device. Consult the controller, hypervisor, and storage-array documentation where a layer is outside FreeBSD.

The ZFS autotrim documentation describes support for block-device vdevs that accept discard and file vdevs whose underlying filesystem supports hole punching. A disk appearing as an SSD is not enough evidence. USB-SATA bridges, hardware RAID controllers, SAN LUNs, virtual disks, and cloud volumes can filter, translate, emulate, or reject discard. If a device does not support the operation, an enabled property cannot create that capability.

Save the baseline status, device identifiers, pool health, and performance measurements. Device node numbers can change across boots, so correlate them with serial numbers, stable labels, enclosure slots, or the storage platform’s LUN identifiers. If a pool is degraded, undergoing replacement, or experiencing unexplained latency, first resolve that operational state. Adding a full-pool trim during an incident changes the I/O workload and complicates diagnosis.

Understand autotrim before enabling it

Autotrim defaults to off. When enabled, ZFS periodically processes space that has recently been freed and is no longer allocated. It intentionally delays small ranges so they can be aggregated into larger operations. Therefore, freeing a dataset or deleting a large file does not imply that the corresponding blocks will be discarded immediately.

Read the current property and pool health, then enable it only for the named pool after deciding that the device stack benefits from it:

zpool get autotrim tank
zpool set autotrim=on tank
zpool get autotrim tank

Property changes are pool-specific. Do not use a wildcard or copy a fleet-wide command before checking pool names, device support, and maintenance policy. For rollback, setting autotrim=off stops automatic processing of newly freed ranges; it does not undo previously issued discard requests.

Automatic trim can impose significant device work, depending on the device’s implementation. The FreeBSD manual explicitly warns that this can stress some lower-end devices and notes that periodic manual trim may be preferable for those devices. Measure latency and throughput under the workload that matters instead of treating autotrim as free background work. Keep one change at a time so that an observed performance shift can be attributed.

Confirm that the pool actually has releasable space

Deleting a file is not sufficient evidence that its blocks became free in the pool. A snapshot or clone can continue to reference old blocks after the live filesystem no longer exposes the file. ZFS trim communicates about ranges that the pool has freed; it cannot discard blocks that remain allocated to a dataset, snapshot, clone, or other live object.

Inspect pool capacity and retained snapshots before interpreting a small reclaim result:

zpool list tank
zfs list -t snapshot -r tank
zfs list -o name,used,refer -r tank

Review snapshot retention with the data owner before destroying anything. Snapshot usage and dataset referenced space have different meanings, and shared blocks complicate attribution. A snapshot that appears inexpensive today can preserve a large amount of historical data as the live dataset changes. Do not remove snapshots simply to make a trim operation appear more productive.

Thin provisioning adds another accounting layer. A pool can report reusable space while the storage array continues to allocate extents until it receives and processes discard, and an array can report reclaimable capacity differently from FreeBSD’s logical pool free space. Capture both the host and storage-side baselines, then compare after the operation completes and the array’s own reclamation interval has elapsed.

Use on-demand trim as a controlled maintenance action

An explicit trim can be started regardless of the current autotrim property. It requests trim for free space in the pool and can therefore be a large operation. Schedule it when the array, virtual-storage provider, and application owners can tolerate the work. Use the official zpool-trim(8) syntax for the installed FreeBSD release; flags and implementation details should always be checked locally.

For a pool named tank, choose one of these alternatives:

zpool trim tank

The first command starts the operation and returns without waiting. If the maintenance job must remain attached until completion, use the wait form instead:

zpool trim -w tank

The -w form waits for devices to finish their trim work. Do not run both examples back-to-back or issue another trim while an identical job is already running just to create a second observer. If the operation must be paced, use the documented -r option instead; it accepts a rate in bytes per second and is applied per leaf vdev:

zpool trim -r 67108864 tank

That example requests an upper rate of 64 MiB/s per leaf. It is a control input, not a promise of that throughput or a guarantee of a particular latency. Use a conservative initial value, observe the array and application, and adjust only after measuring. If a pool contains many leaf devices, a per-vdev cap can still create substantial aggregate I/O.

The manual documents -s to suspend trimming and -c to cancel it. Suspension is resumable by running the trim command again for the relevant pool or target device; cancellation is a deliberate stop, not a pause. For either action, FreeBSD documents that if even one named target is invalid or is not currently being trimmed, the command fails without suspending or cancelling any target. Avoid broad multi-device commands unless the target set and current trim state have been verified.

Distinguish discard from secure deletion

Ordinary TRIM informs the device that blocks are no longer required. It does not guarantee that old bytes are unrecoverable, and a device can handle the notification internally. Secure trim, requested with -d where supported, is a separate device capability. The FreeBSD manual says secure trim requires device support and is not supported by all SSDs.

Do not use zpool trim -d as a compliance erasure workflow without verifying the exact drive model, firmware, controller path, and applicable sanitization policy. The command’s request passing through a controller does not independently prove that every physical flash cell or remapped sector was sanitized. For data disposition, use an approved sanitization procedure and retain its evidence. Likewise, never substitute the low-level trim(8) command for ZFS pool-aware trimming on active pool members; raw block operations can destroy data or pool metadata.

Monitor safely and define completion

Capture the command start time, pool identity, property state, selected rate, and any device-specific warning. Observe application latency and device errors throughout the maintenance interval. On FreeBSD 15.1, zpool status -t tank displays per-vdev TRIM status; confirm the installed release’s manual before parsing output in automation.

For a script, make failure visible and preserve the exit status. A wait command may run for a long time, so run it from a supervised maintenance job with an external timeout and logging rather than an unattended terminal. If the command is interrupted, inspect the pool and per-device trim state before deciding whether to resume, suspend, or cancel. Do not infer success because the shell returned to a prompt after a disconnect.

Acceptance should answer the original goal. For thin-provisioned storage, confirm reclamation at the storage platform with its own authoritative telemetry; a FreeBSD command returning success only proves the host-side operation completed as reported. For SSDs, compare sustained behavior over a suitable observation window, not a single benchmark. For a pool where trim provides no measurable benefit, disabling autotrim may be the sensible outcome.

Record before-and-after values, exact pool and leaf identifiers, the software release, storage firmware or service, timing, error messages, and the decision to retain or revert the setting. Do not clear device or ZFS error counters before preserving evidence. Keep scrubs, resilvers, backups, and trim runs conceptually separate: a successful trim does not verify data checksums, repair a damaged block, restore redundancy, or validate a backup.

Related:

Sources: