FreeBSD zpool history: Reconstruct ZFS Administrative Changes

zpool history displays the command history recorded by ZFS for one or more pools. It is a valuable first stop after an unexpected pool change: an operator may have added a vdev, changed a property, exported or imported a pool, or started a scrub. The history can provide context that is absent from an application log, but it is not a complete shell audit, tamper-proof change record, or guarantee that every administrative action was captured.

Use it as one evidence source among several. Pool health and current topology come from zpool status, properties from zpool get, system activity from logs, and business intent from the change system. A history entry can show that a command was recorded; it does not prove the resulting operation completed successfully or that the pool is currently healthy.

Capture a baseline before interpreting events

Start with pool identity and current health:

zpool list -o name,size,alloc,free,health
zpool status -v
zpool history tank

Verify that tank is the intended pool, especially on systems with imported pools or similarly named test environments. Preserve the host, FreeBSD release, pool GUID if available, local time zone, command output, and collection time. A pool can be renamed or imported on a different host, so a human-friendly pool name alone is not sufficient to correlate evidence over its lifecycle.

The standard history view displays recorded command history for the selected pool. Compare its timestamps and operations to messages, dmesg, change-management records, and service logs. A missing entry can mean the action was not recorded in this pool history, not that the action never happened.

For a broader view, -i includes internally logged ZFS events in addition to user-initiated entries. -l requests long-format records, which include the user name, host name, and zone for entries where that information is available:

zpool history -i tank | less
zpool history -l tank | less

Use -i and -l deliberately. Long output may disclose usernames, hostnames, and administrative context. Internal events can increase volume and are not a replacement for zpool events or kernel logs when diagnosing device faults. Keep an original copy in a protected incident directory if the timeline may be needed later.

Reconstruct the timeline, not just the command list

Read entries in order and identify state transitions. A recorded zpool add, zpool attach, zpool replace, zpool set, zpool scrub, or import operation should be correlated with the current topology and status. Ask what operation the command requested, what event followed, and whether a later action reversed or superseded it.

For example, if an unexpected device appears in a mirror, examine pool history for relevant attach/replace commands, then inspect the current zpool status -v tree and device identifiers. Do not infer that the most recent command is the current truth; a replacement may have completed, failed, been reverted, or been superseded by a later action.

When history includes only a command request but not a clear completion event, use current pool state and logs to determine the outcome. A scrub command in history does not mean the scrub finished. Verify progress and results with zpool status; use the scrub/resilver operational procedure for repair planning. Similarly, a zpool set entry does not prove the property remains at that value, since a later change may have occurred.

Treat timestamps as chronology hints. Confirm clock synchronization, time zone, system reboots, and host transfers before combining records from multiple machines. A pool may move between systems and the history includes host context only in the relevant long format; that context does not provide a cryptographically trusted identity.

Scope and limitations of pool history

Pool history records ZFS administrative operations, not every command typed into a shell. It will not record unrelated cp, rm, application SQL, package manager, or service commands merely because they affected files stored on the pool. It also is not a filesystem access log and should not be presented as proof of which user read or modified each file.

History is associated with the pool and is only available while the pool metadata can be read. A failed pool, missing devices, import issue, corruption, or destroyed pool can limit access. Do not rely on the history as the only copy of change evidence. Exporting or moving a pool can move its stored history context, but incident procedures should preserve external copies before destructive recovery work.

The history output is not a complete configuration snapshot. For effective state, collect zpool status, relevant properties, dataset properties, mount state, and system configuration. Record the exact command options and software versions because output details may differ across releases.

Use history during change review

After a planned change, query history and compare it with the approved plan:

zpool history -l tank | tail -80
zpool status -v tank
zpool get all tank

Review whether the expected commands appear, whether there are unexpected operations, and whether the final status matches the acceptance criteria. Redact secrets from any command line before broader distribution; ZFS history may record command arguments relevant to administration. Do not assume credentials are safe merely because an operation used an interactive prompt.

For recurring operations, monitor history only as a secondary signal. A script that scans for command strings can misclassify different syntax or miss operations. Alert on actual state and outcomes: unexpected vdev topology, property drift, scrub errors, capacity thresholds, or unapproved changes from an authoritative control plane. Retain the history snapshot with a change ID to provide context, but do not make it the sole enforcement mechanism.

Incident workflow for an unexpected pool change

Stop any automated repair or cleanup that could further change the pool.
Capture zpool status -v, zpool list, zpool get, and history output before attempting remediation.
Confirm device identities using stable serials and controller paths, not only daN names.
Correlate timestamps with system logs, maintenance records, and the relevant automation job.
Determine current topology and health independently of the history entry.
Plan a reversible correction where possible, with backup and console access.
Re-query history and status after the change, and document the final state.

Do not export, destroy, clear labels, replace vdevs, or run recovery flags merely to make history “look clean.” Those actions change pool state and can reduce recovery options. If the pool is degraded or unavailable, follow the appropriate import or recovery procedure and preserve the devices before experimentation.

Protect and retain the evidence

History output can contain operational details that should not be public. Store it with least-privilege access and retention consistent with the incident or change policy. For stronger auditability, ship periodic snapshots to an append-only or otherwise protected logging system and record hashes and collection metadata. A hash proves a later copy matches the captured bytes; it does not prove that the original host’s history was complete or untampered before capture.

Use a controlled shell command with explicit output destination for evidence collection. Ensure the destination is not on the pool being investigated if that pool is unstable. Avoid redirecting to a full or read-only filesystem without checking capacity and mount state. Capture exit status and errors as well as standard output.

Acceptance criteria

An investigation using zpool history is complete when the pool is identified unambiguously, current state is independently recorded, relevant history entries are correlated to logs and change records, missing entries are not misinterpreted as proof of absence, and any remediation has a verified outcome. Pool history is an excellent clue and a limited record; reliable operations combine it with live state, independent logs, and tested recovery procedures.

Related:

Sources: