Unified Logging: How os_log Replaced syslog on macOS

macOS moved away from traditional plain-text syslog years ago in favor of the unified logging system (os_log), and the reasons behind that shift explain a lot about why modern macOS log output looks and behaves so differently from the flat text files a long-time Unix user might expect.

What was actually wrong with plain-text syslog

Traditional syslog writes human-readable text lines to flat files, which sounds simple and is simple — but that simplicity comes with real costs at the scale a modern operating system actually operates at. Every log message written as fully-formatted text has real, continuous performance overhead (string formatting happens whether or not anyone ever reads that particular line), log files grow without bound unless actively rotated and pruned, and — critically for actual diagnosis — plain text carries no structured metadata beyond whatever a given piece of software chose to embed in its message string, making it hard to reliably filter, correlate, or query logs from different subsystems in any consistent way.

What unified logging does differently

The unified logging system defers the expensive part — formatting a human-readable message string — until a log is actually being viewed or extracted, rather than doing it at write time regardless of whether anyone will ever read that message. What actually gets written at log time is compact, structured, binary data: a reference to the log format string (not the fully rendered string itself), the actual variable values being logged, and structured metadata like subsystem, category, and severity level. This alone accounts for a meaningful reduction in the overhead of routine logging, since the expensive string-formatting work only happens on-demand for messages someone is actually viewing.

Log levels and their actual retention behavior

Unified logging defines several severity levels — from debug (most verbose, essentially never persisted, only visible when actively streaming logs from a running process) up through default, info, error, and fault — with meaningfully different persistence behavior per level baked into the system itself, rather than left entirely to per-application configuration the way traditional syslog largely was. debug and info level messages, in particular, generally aren’t persisted to disk at all by default — they’re only visible if you’re actively streaming logs in real time — which is a deliberate trade-off: extremely verbose, high-volume diagnostic detail is available when you’re actively debugging something live, without that same volume of data being permanently retained and consuming disk space indefinitely for messages that are only useful in the moment.

Structured subsystems and categories

Every unified log message can carry a subsystem identifier (typically a reverse-DNS-style string identifying the responsible component or application) and a category within that subsystem, which is what makes the modern log show/log stream command-line tools’ filtering actually useful in practice — you can query specifically for a given subsystem’s error-and-above messages across the entire system’s unified log, correlated by time against every other subsystem’s messages, in a way that grepping through a collection of separate flat-text log files never made straightforward.

Activity tracing: following one logical operation across processes

Beyond individual log messages, unified logging supports activities — a mechanism for tagging a logical unit of work (say, handling one specific user-initiated action) with an identifier that follows it across thread and even process boundaries, so log messages related to that same logical operation can be correlated together even if the actual work involved several different processes handing off to each other along the way. This directly addresses a real, common pain point in diagnosing modern macOS behavior, where a single user-facing action frequently touches multiple separate system processes and services (exactly the kind of XPC-based, privilege-separated architecture common throughout modern macOS) — without activity tracing, correlating log messages across that many involved processes by timestamp alone is unreliable and tedious.

Why this trade-off was the right one for a modern OS

The core trade-off unified logging makes — genuinely more complex to inspect casually with basic text tools like cat or grep against a raw log file, in exchange for dramatically better performance at write time and dramatically better filtering, correlation, and cross-process tracing capability when actually diagnosing something using the purpose-built tools — reflects a reasonable bet that structured, purpose-built logging tooling would pay for itself as the operating system’s own internal complexity (and the number of separate, cooperating processes involved in any given user action) continued to grow, which is exactly the direction macOS’s XPC-heavy, privilege-separated internal architecture has continued moving in the years since unified logging was introduced.

Where the actual bytes live on disk

Unlike syslog’s readable flat text files, unified log data is stored in a binary .tracev3 format across two separate directory trees — /var/db/diagnostics/ for the actual timestamped log entries, and /var/db/uuidtext/ for the format strings and metadata those entries reference by pointer rather than embedding directly. This split-storage design is a direct consequence of the “defer formatting until read time” strategy covered above: a raw log entry on disk is largely just a compact reference into the separately-stored format-string database plus the actual variable values, not a self-contained formatted message, which is exactly why the binary files aren’t meaningfully readable with cat or a plain text editor the way an old-style syslog file always was.

Why this format resists casual tampering better than flat text

A meaningful side effect of moving to structured binary storage, referenced against a separate format-string database, is that it’s considerably harder to casually forge or silently edit a specific unified log entry compared to opening a plain-text syslog file in any text editor and altering a line — an attacker or a careless script editing raw .tracev3 bytes by hand risks producing entries that fail to parse correctly against the expected binary structure at all, rather than producing a plausible-looking but falsified plain-text line indistinguishable from a genuine one. This wasn’t unified logging’s primary design goal (performance and structured filtering were the stated motivations), but it’s a genuine, if secondary, integrity benefit that came along with the architectural shift away from directly-editable flat text.

Correlating unified logs with sysdiagnose bug reports

The unified logging system was explicitly designed with Apple’s own sysdiagnose tool in mind — the comprehensive diagnostic bundle generation invoked via a specific hardware key combination or the Shortcuts app, used when filing detailed bug reports with Apple. sysdiagnose pulls directly from the same unified log store this deep-dive describes, packaging a relevant window of structured log data alongside other system state into one bundle — meaning understanding unified logging’s subsystem/category structure and activity tracing directly helps in interpreting a sysdiagnose bundle’s log contents too, since it’s the identical underlying data format and querying model, just pre-collected into a single archive rather than queried live.

Related:

Sources: