FreeBSD ZFS Quotas and Reservations: Enforce Dataset Capacity Predictably

ZFS quotas and reservations are often set with similar-looking numbers but solve opposite problems. A quota limits how much space a dataset and, depending on the property, its descendants may consume. A reservation commits space for a dataset so siblings cannot consume it first. Neither changes the physical size of a pool, and neither is a substitute for monitoring free capacity.

This guide focuses on dataset-level properties on FreeBSD. User, group, and project quotas are different controls and may be more appropriate when several teams share one dataset. Before changing a limit, map the parent-child tree, snapshot policy, replication targets, and the capacity required for recovery.

Model the dataset tree first

Start with a tree and current accounting rather than applying a quota to a guessed path:

zfs list -r -o name,used,refer,available,mountpoint tank
zfs get -r quota,refquota,reservation,refreservation tank

The output should be treated as a shared-pool view. ZFS space is allocated from the pool and attributed across datasets, snapshots, reservations, and descendants. “Available” is not a promise that this dataset alone can consume that much; ancestor limits, other datasets, reservations, snapshots, pool health, and activity can change the result. Record the output before making a policy change.

Use distinct child datasets for distinct capacity boundaries. For example, a home directory stored as files inside one large dataset cannot have an independent dataset quota unless it is moved into its own dataset. A dataset boundary also gives administrators a natural place to apply mountpoint, snapshot, replication, and retention policy.

Choose quota versus reference quota deliberately

The quota property limits the total space charge associated with a dataset and its descendants, including snapshots. This is the more complete boundary when the policy is “this tenant’s entire dataset subtree must remain under this amount.” An ancestor’s quota can constrain consumption by descendants, and a child can also have its own explicit stricter limit. Do not assume the quota property itself is automatically inherited; set and inspect it at the dataset where the policy belongs.

Set a sample subtree limit:

zfs set quota=200G tank/tenants/acme
zfs get quota,used,available tank/tenants/acme

The refquota property limits the referenced space of that dataset itself and excludes child datasets and snapshots. It is useful when the limit should constrain the live data in one filesystem while snapshot retention or descendants are accounted for separately. It is not a way to make snapshot storage free: snapshots still consume pool space, and the parent quota or pool capacity can still constrain the system.

zfs set refquota=80G tank/tenants/acme/home
zfs get refquota,refer,used,available tank/tenants/acme/home

Do not assume that refquota is a safer or more generous quota. It moves the accounting boundary. If you intend to cap the combined live data, snapshots, and child datasets, use quota and verify the hierarchy. If you intend to cap just the dataset’s referenced data, use refquota while separately budgeting snapshots and children.

To remove a limit, use none only after confirming that no parent quota will still enforce a boundary:

zfs set quota=none tank/tenants/acme

Removing a quota does not release used blocks. It merely changes the enforcement ceiling, so alerting and capacity planning remain necessary.

Use reservations as commitments, not extra capacity

A reservation guarantees a minimum amount of pool space to a dataset and its descendants for accounting purposes. It is charged to parent datasets and counts against parent quotas and reservations. A reservation can therefore prevent siblings from consuming all space needed by a critical service, but excessive reservations make the pool appear constrained even when the reserved data has not yet been written.

zfs set reservation=40G tank/services/database
zfs get reservation,used,available tank/services/database

The refreservation property reserves for the dataset itself, excluding descendants. Reference reservations are commonly useful for zvol provisioning semantics, and ZFS volume properties can impose their own allocation rules. Check zfsprops(7) for the exact behavior in the local release before applying a reservation to a volume. Dataset quotas cannot be set on zvols because volsize acts as an implicit quota.

Reservation is not replication, redundancy, or physical preallocation in every configuration. It is an accounting commitment within the pool. A reservation on a child consumes capacity at its parent and can interact with parent quotas. Before setting one, calculate the total of all sibling commitments and leave a separate operational margin for metadata, snapshots, replacement or resilver activity, and normal growth.

Understand snapshots and descendants

Snapshots share blocks with their source dataset until writes or destruction change which blocks are referenced. Deleting a file from the live filesystem may not free the pool space if a retained snapshot still references those blocks. That is why a quota that includes snapshots can stop new writes even when the current filesystem tree appears small. Conversely, destroying a snapshot may free less space than its own “used” estimate if blocks are shared with another snapshot.

Before tightening a limit, inspect snapshots and descendants:

zfs list -r -t filesystem,volume,snapshot -o name,used,refer,available tank/tenants/acme
zfs get -r quota,refquota,reservation,refreservation tank/tenants/acme

Do not run a recursive destroy as a capacity shortcut. Snapshot deletion can remove rollback points, retention evidence, and replication bases. Identify owners, retention windows, and downstream replicas first. If a snapshot policy is automated, test the effect of a quota threshold on both new writes and scheduled snapshot creation.

Plan thresholds and failure behavior

Hard quota enforcement can cause writes to fail when the limit is reached. Applications may report a generic “no space” error even though zpool list shows unused pool capacity, because the dataset or an ancestor is the actual constraint. A reservation can make other datasets see less available space before the reserved dataset uses those blocks. Monitoring only pool-wide free bytes will miss both conditions.

Set alerts at a threshold that leaves time for an operator to respond. For each dataset, monitor used, referenced, available, effective limits, snapshot usage, and parent constraints. Use zfs get to inspect properties at every relevant level. Avoid alerting only on percentage: a small dataset at 90 percent and a very large dataset at 90 percent have different absolute operational impact.

An expansion procedure should identify who owns the data, who approves a policy increase, what sibling commitments will be affected, and what the new pool headroom will be. A successful zfs set command proves only that the property changed; it does not show that the capacity plan remains safe. Recheck all affected ancestors and descendants after the change.

Validate a policy change safely

For a new tenant, create a dedicated child dataset, set its mountpoint and intended quota, then verify inheritance and effective accounting before placing production data there. Test writes in a disposable dataset on a non-production pool or a dedicated test environment. Deliberately reaching a hard limit is disruptive by design, so do not perform that test against a live service.

For a production change, capture the current values and a rollback command before editing:

zfs get quota,refquota,reservation,refreservation tank/tenants/acme
zfs set quota=250G tank/tenants/acme
zfs get quota,used,available tank/tenants/acme
zfs list -r -o name,used,refer,available tank/tenants/acme

Use a maintenance record to explain why a limit changed and what monitoring will verify afterward. If you reduce a quota, measure current accounted use and consider future snapshot growth first. Setting a quota below current use does not delete data to make the dataset fit; it constrains additional consumption and can cause immediate write failures.

Capacity acceptance criteria

An operational quota policy is complete when its boundary is explicit, properties are visible at the right levels, snapshot and child dataset behavior is understood, alerts identify the dataset-level cause, and there is a documented approval path for changes. Reservations must be summed against parent capacity and treated as commitments in capacity dashboards.

Keep ZFS quotas separate from pool health work. A full pool may require snapshot retention review, tenant growth control, or pool expansion; a degraded pool may require hardware recovery first. Do not destroy snapshots or lower reservations merely to quiet an alert without checking restore and service commitments.

Related:

Sources: