Linux Filesystem DAX: fsdax, S_DAX, and What mmap() Actually Bypasses

Direct Access (DAX) is a Linux memory-mapping path for storage that the system can expose as memory-like and addressable by the CPU. For a DAX-enabled file, the kernel can map storage directly into a process instead of using page-cache pages as an intermediate copy. That is a specific storage contract, not a generic “make this SSD faster” option, and it is not synonymous with O_DIRECT.

The useful operational question is not simply whether a filesystem supports DAX. It is whether the backing device supports it, whether the filesystem and mount policy permit it, and whether the particular file’s active state has DAX enabled. Filesystem DAX also does not make a multi-step application update atomic or automatically make every store durable. Those are separate properties that must be designed and tested.

Two DAX modes solve different mapping problems

The NVDIMM subsystem’s namespace modes make a useful first distinction. Filesystem DAX, commonly called fsdax, exposes a block device on which a filesystem can place files. An application can then mmap() a file and access mapped data through CPU loads and stores without the ordinary page-cache copy. Device DAX, or devdax, exposes a character device intended for applications that map an address range directly; it does not provide a filesystem namespace or normal file paths.

That distinction affects application architecture. A file-oriented database or store can use fsdax if the device, filesystem, kernel, and file state qualify. A specialized memory allocator or application that owns an entire mapped region may be designed around a devdax device instead. A normal block device does not become a DAX device merely because it is fast or because the application uses mmap().

Start with read-only inventory on systems that provide the relevant tooling:

ndctl list --human
lsblk -o NAME,TYPE,FSTYPE,SIZE,MOUNTPOINTS
findmnt -T /mnt/pmem -o TARGET,SOURCE,FSTYPE,OPTIONS

ndctl is specific to supported persistent-memory configurations and may not be installed or return devices on an ordinary workstation. lsblk and findmnt show block and mount context, but neither proves that a particular file currently has active DAX semantics. Do not copy commands that create namespaces, format filesystems, or change mount state into a production host without checking device ownership and maintaining a recoverable backup; those operations can destroy existing data.

The active state is not the same as the stored hint

For ext4 and XFS, the kernel documentation describes two related per-file flags. FS_XFLAG_DAX is a persistent inode setting, while S_DAX indicates the active DAX state in the in-memory inode. The persistent setting can be inherited by files and subdirectories created under a directory. Changing a parent does not retroactively rewrite the state of existing children. An active state is selected when the inode is instantiated, based on the storage’s DAX support, the persistent flag, and the mount option.

The mount policy is another input. With supported storage, dax=inode follows the inode flag and is the documented default for ext4/XFS. dax=never prevents S_DAX from being set; dax=always ignores the per-inode hint and enables the active mode where the device supports it. The older shorthand -o dax aliases dax=always; the kernel documentation marks it as legacy and recommends spelling out dax=always. Do not infer current file state from a flag alone: a mount override can affect active behavior, and toggling an inode hint does not necessarily change an already-instantiated inode until all users close it.

The kernel exposes STATX_ATTR_DAX through statx(2) so software can query the active file attribute. This is an output-attribute bit, not an input request-mask flag: request ordinary fields with STATX_* flags, then inspect stx_attributes_mask to learn whether the DAX attribute is reported. If STATX_ATTR_DAX is absent from that attribute mask, a program must report “not available” rather than silently interpreting the missing bit as “DAX is off.” A minimal inspection utility can express that distinction:

#define _GNU_SOURCE
#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <sys/stat.h>

int main(int argc, char **argv) {
    struct statx stx = {0};
    if (argc != 2) {
        fprintf(stderr, "usage: %s PATH\n", argv[0]);
        return 2;
    }
    if (statx(AT_FDCWD, argv[1], 0, STATX_BASIC_STATS, &stx) < 0) {
        perror("statx");
        return 1;
    }
    if (!S_ISREG(stx.stx_mode)) {
        fputs("DAX file state is meaningful here only for a regular file\n", stderr);
        return 3;
    }
    if ((stx.stx_attributes_mask & STATX_ATTR_DAX) == 0) {
        puts("kernel/filesystem did not report STATX_ATTR_DAX");
        return 3;
    }
    puts((stx.stx_attributes & STATX_ATTR_DAX) ? "DAX active" : "DAX inactive");
    return 0;
}

Compile this only with headers that define STATX_ATTR_DAX, and treat a syscall or unsupported-mask result as an evidence gap rather than proof about the storage hardware. A filesystem setting is a requested policy; the active query is the runtime evidence for a regular file.

What bypassing the page cache does, and does not, mean

The key performance path is fewer data copies between a memory-like block device and userspace. It can make mmap() a natural access model for workloads that benefit from load/store access to persistent memory. The kernel still manages mappings, file metadata, allocation, permissions, page faults, and synchronization. DAX does not erase filesystem semantics, and a process still needs normal synchronization when multiple threads or processes share mutable data.

Do not conflate this with direct I/O. O_DIRECT concerns a particular I/O operation’s interaction with the page cache and may have alignment and fallback rules. Filesystem DAX is an active file-access mode for compatible backing storage and mapping behavior. Conversely, an ordinary mmap() is not evidence of DAX: files on conventional block storage are normally backed through the page cache.

Nor should a DAX mapping be treated as an application transaction. Persistent-memory programming libraries describe explicit flush/drain behavior for making stores durable, and document that persistence calls do not provide atomic or transactional updates. A crash-consistent data structure still needs an ordering protocol, recovery design, and appropriate persistence primitives for its platform. A successful memcpy() into a mapping is not a commit record. If using PMDK or another persistence library, follow the versioned library’s documentation for the actual mapping type rather than transplanting CPU cache-flush instructions into a program.

This is why DAX should not be marketed as an automatic durability upgrade. A workload may see a different copy path without receiving atomic records, write ordering, checksums, replication, or recovery. Define what “committed” means for the application and test crash points around every transition that matters.

Configure only after measuring eligibility

On ext4/XFS, file and mount policy can be changed, but the order is important. A directory’s persistent DAX attribute can establish an inheritance default for future descendants. A per-file setting can affect an existing file, subject to inode lifetime constraints. A mount override can deliberately ignore per-inode policy. On filesystems with a filesystem-wide dax option, the option’s reach can be broader than one file. Use the exact filesystem’s current documentation and record the pre-change mount options.

Before a controlled change, capture:

Kernel release, architecture, filesystem type, and mount options.
The device and namespace mode, where the platform exposes them.
The active statx attribute for the actual test file.
The application’s mapping mode, workload, and durability protocol.
A known-good backup and a restoration test for the data involved.

Then test on a disposable file and a non-production workload. Compare the ordinary filesystem path with the proposed DAX path under the same input size, access pattern, concurrency, cache state, and CPU affinity. Record tail latency and CPU consumption as well as throughput. A microbenchmark that fits entirely in a cache or measures allocation rather than steady-state I/O can exaggerate benefit. Keep the workload realistic enough to include metadata and fault behavior.

On deployment, verify each layer independently. findmnt confirms the mount in use; ndctl may show the namespace mode on supported NVDIMM systems; and a statx-based check provides the per-file active attribute. If a mount says dax=always but the active flag is unavailable, resolve that uncertainty rather than declaring the rollout successful. If a file is not DAX-active, determine whether the cause is unsupported media, unsupported filesystem, an overriding option, stale inode state, or a kernel/tooling boundary.

Failure, compatibility, and rollback boundaries

DAX can change the set of operations that work on a mapping. The kernel documentation warns about limitations where DAX-backed memory lacks struct page metadata, including interactions with interfaces such as RDMA and splice() in those cases. Applications that pass mapped addresses to other subsystems must test those exact paths. A test that only reads its own mapping says little about consumers that pin, export, or transmit those pages.

Plan for media errors and recovery too. A mapping fault against a known bad persistent-memory range can be reported as SIGBUS; a process that treats it as a normal transient error can corrupt its own state or loop. Log the device, file identity, offset when available, kernel and filesystem versions, and the exact operation. Restore from a trusted backup and revalidate the application’s data invariants. Do not “repair” a failed persistent-memory structure by assuming that bytes already visible in a process are durable or correctly ordered.

Finally, keep the topic in perspective: Linux DAX is a specialized direct-mapping capability for devices that support it. It is valuable precisely because it has explicit requirements and constraints. The reliable procedure is to inventory the hardware, understand fsdax versus devdax, inspect active per-file state, measure the application workload, and design persistence independently. If those conditions are not present, ordinary buffered I/O or a separately justified direct-I/O design is the right baseline, not a failed DAX configuration to force into place.

Related:

Sources: