NTFS Internals: the MFT, Journaling, and Alternate Data Streams

NTFS has been Windows’ primary filesystem since Windows NT 3.1 (1993), and unlike FAT, it was designed from the start around a database-like internal structure — the Master File Table — plus transactional journaling for metadata consistency. Both ideas were ahead of their time in 1993 and still shape how NTFS behaves today.

The Master File Table: everything is a record

Every file and directory on an NTFS volume has a corresponding record in the Master File Table (MFT), itself stored as a file ($MFT) at a well-known location. An MFT record is normally 1KB and contains a series of attributes — the file’s name, its security descriptor, its timestamps, and its data.

fsutil file queryfileid C:\Windows\notepad.exe

For small files, the data itself can fit entirely inside the MFT record — called a resident file — avoiding a separate disk read entirely. Larger files store their data in separate clusters, with the MFT record instead holding extents: pointers to where those clusters live.

fsutil fsinfo ntfsinfo C:\

The journal: $LogFile and crash consistency

NTFS maintains a transaction log, $LogFile, recording metadata changes before they’re committed to the MFT itself — the same “write-ahead log” idea behind database transactions and behind gjournal on FreeBSD. If the system crashes mid-write, NTFS replays or rolls back incomplete transactions from $LogFile on the next mount, which is why an NTFS volume rarely needs a full chkdsk scan after an unclean shutdown the way older FAT volumes did.

chkdsk C: /scan

chkdsk /scan runs an online, non-disruptive verification pass; the classic offline chkdsk /f (which requires dismounting or a reboot) is reserved for when actual corruption — not just a pending journal replay — is suspected.

Alternate Data Streams: files within files

A lesser-known NTFS feature: every file can have, in addition to its primary (unnamed) data stream, any number of alternate data streams (ADS), addressed with a filename:streamname syntax.

Set-Content -Path notes.txt -Value "primary content"
Set-Content -Path notes.txt:secret.txt -Value "hidden stream"
Get-Content -Path notes.txt:secret.txt

dir /r notes.txt
Get-Item -Path notes.txt -Stream *

Windows itself uses this mechanism for the “unblock” checkbox on downloaded files — a Zone.Identifier alternate stream records that a file came from the internet, which is exactly the metadata Gatekeeper-style download-warning systems (and Windows SmartScreen) check.

Get-Content notes.txt -Stream Zone.Identifier
Unblock-File notes.txt

Compression and sparse files

NTFS supports per-file transparent compression and sparse files — files with large logical “holes” of zeros that consume no actual disk space — both configured via fsutil and both transparent to any application reading the file normally:

compact /c C:\Data\logs
fsutil sparse setflag C:\Data\bigfile.dat
fsutil sparse queryrange C:\Data\bigfile.dat

Hard links, junctions, and symbolic links

NTFS supports the same family of filesystem-level indirection Unix systems do, under different names: hard links (multiple directory entries pointing at the same MFT record), junctions (directory-level reparse points, resolved entirely by the local filesystem), and symbolic links (a more general reparse point that can point anywhere, including across volumes or to a network path).

New-Item -ItemType HardLink -Path link.txt -Target original.txt
New-Item -ItemType Junction -Path C:\Shortcut -Target D:\RealFolder
New-Item -ItemType SymbolicLink -Path C:\Link -Target C:\Target

Security descriptors: ACLs live in the MFT too

An NTFS file’s permissions (its security descriptor) are themselves just another attribute in its MFT record, referencing a shared entry in $Secure when multiple files share identical permissions — an optimization avoiding a full descriptor copy per file, conceptually similar to how SELinux’s labels reference a shared policy rather than embedding the full rule set per file.

icacls C:\Data\report.docx

NTFS’s own version history

NTFS didn’t arrive in 1993 in the form it exists today — it has gone through several distinct on-disk format revisions since Windows NT 3.1. NTFS v1.1, shipped with Windows NT 3.51 in 1995, added compressed files and named (alternate data) streams. NTFS v1.2, with Windows NT 4.0 in 1996, added proper security descriptor support. The bigger jump came with Windows 2000’s NTFS v3.0: disk quotas, the Encrypting File System (EFS), sparse file support, reparse points (the mechanism junctions and symbolic links are both built on), and the USN journal all arrived in that single release — enough new functionality that most of what a “modern NTFS feature” list today actually traces back to Windows 2000 specifically, not the original 1993 format, with comparatively modest revisions since.

The USN Journal: a second journal, for a different purpose

$LogFile (covered above) and the USN Journal ($Extend\$UsnJrnl, also introduced with NTFS v3.0) solve genuinely different problems despite both being described as “NTFS journaling.” $LogFile exists purely for crash consistency — a short-lived, internal record of in-flight metadata transactions, replayed or rolled back at mount time and not meant for any application to read directly, and it wraps and discards old records automatically as new transactions arrive. The USN Journal, by contrast, is a persistent, application-readable log of which files changed, without necessarily recording the exact bytes that changed — every create, rename, delete, and modification to the volume gets a monotonically increasing Update Sequence Number, letting a consuming application ask “what’s changed since USN N” instead of re-scanning an entire volume from scratch. This is the actual mechanism behind incremental-scan tools that need to notice filesystem changes efficiently — Windows Search’s indexer, robocopy /MIR’s ability to sync large trees quickly on repeat runs, and file-replication services have all relied on reading the USN Journal directly rather than polling the filesystem or relying on real-time change notifications alone, which don’t survive a period when the consuming application wasn’t running to receive them.

fsutil usn readjournal C: csv

Why the MFT-centric design has held up

Treating every file as a structured record with typed attributes — rather than a name plus a pointer to a data block, as FAT does — is what makes NTFS features like ADS, compression, sparse files, and rich ACLs all fall out of the same underlying mechanism instead of requiring bolted-on special cases. It’s also why NTFS volumes remain usable at far larger scale and far higher file counts than FAT ever could: the MFT itself is indexed and can grow, rather than relying on a fixed-size, linearly-scanned table the way FAT’s own allocation table does.

Related:

Sources: