BFS: How Haiku's File System Doubles as a Database

The Be File System gives Haiku files more than names and byte streams. A node can carry typed, named attributes; a BFS volume can maintain indexes over selected attributes; and applications can issue predicates that return matching entries without scanning every directory themselves. This is why BFS is often described as database-like. The comparison is useful as long as it is not mistaken for a claim that BFS is a relational database or supports SQL transactions.

Haiku implements BFS as a native file-system add-on under src/add-ons/kernel/file_systems/bfs. The source includes inode, B+tree, query, index, journal, and volume code. Applications ordinarily reach those capabilities through the Storage Kit—especially BNode, BQuery, BEntry, BDirectory, and BVolume—rather than depending on the on-disk layout.

Attributes are typed streams attached to a node

A BFS attribute has a name, a type code, and data. It belongs to the file-system node independently of the file’s primary byte stream. An audio file can carry artist and album attributes; a mail file can carry sender and status; an application can attach its own domain-specific metadata. The type distinguishes a string from an integer, time, boolean, or another representation recognized by the application.

The Storage Kit’s BNode API can read, write, remove, rename, and enumerate attributes. GetAttrInfo() reports an attribute’s type and size before the caller allocates or interprets a buffer. Robust code checks both. An attribute is not trustworthy merely because it is stored by the file system: another application may write the wrong type, truncate data, or use a newer schema.

Attributes travel with their node during a rename or move within the same BFS volume because the directory name changes while the node remains. Copying to a file system that lacks compatible attributes can discard them, and archive or transfer tools vary in whether they preserve them. Applications must not keep irreplaceable information only in an attribute unless their backup and interchange paths are verified.

Haiku uses conventional attributes for system integration. For example, MIME-related metadata helps Tracker and the MIME database associate files with types and preferred applications. The mechanism is general, however: BFS does not need to understand what an application’s attribute means in order to store its typed bytes.

Indexes turn selected attributes into lookup keys

An attribute does not become fast to query merely by existing. A BFS volume maintains indexes for explicitly indexed names and types. The BFS implementation uses B+tree structures for directories and indexes, allowing ordered lookup without a complete volume scan. Index maintenance is part of file-system mutation: when an indexed value changes, the corresponding index entry must change as well.

Indexes are per volume. A query attached to one BVolume searches that volume’s index state, not every mounted disk. The same attribute name may be indexed on one BFS volume and absent from another. Software that depends on a custom index should detect its absence and provide a clear setup or fallback path instead of returning an unexplained empty result.

Type agreement matters. A string index and an integer index have different ordering and comparison semantics. Writing an attribute under the expected name with an incompatible type does not create a valid match. Schema documentation should therefore specify attribute name, type code, encoding, and whether the index is required.

Indexes consume space and write work, so indexing every possible attribute is not free. Choose fields that users actually search or sort across the volume. Data used only after opening one known file can remain an ordinary attribute. A stale or damaged index is a file-system consistency issue distinct from damaged file contents.

BQuery expresses predicates over a volume

BQuery is the Storage Kit interface for BFS queries. An application selects a volume, supplies a predicate, calls Fetch(), and iterates matching entries with methods such as GetNextRef(). Predicate construction also has helper methods that push attribute names, values, and operators, avoiding some manual string assembly.

Queries combine comparisons and logical operations over indexed fields. A media browser can request entries whose artist equals a value; a mail client can combine unread status with a sender; Tracker can expose saved query results as a user-facing search. The returned entry_ref identifies an entry, after which the application can open it and validate the current attributes.

A query result is not a permanent authorization or identity token. Files can be renamed or deleted after enumeration, and attributes can change between the index lookup and use. Open the entry, check errors, and revalidate critical conditions before destructive or security-sensitive actions.

Query syntax must treat user input as data. Escaping and type-correct predicate construction prevent a search term from changing the intended expression. Expensive broad predicates should be bounded in interactive applications, and result enumeration belongs off a window thread when it can take noticeable time.

Live queries convert changes into messages

BQuery can target a BMessenger for live updates. After the initial result set, relevant file-system changes generate update messages so the application can add, remove, or change displayed matches. This is the mechanism behind interfaces that behave like continuously maintained views rather than one-time search reports.

The live notification is a change event, not a replacement for local state management. Events can arrive while the UI is processing earlier work. The handler should identify the affected node or entry, re-read necessary attributes, and update its model idempotently. A full refresh remains a useful recovery mechanism after overflow, restart, or an event the application cannot reconcile.

Live queries depend on indexes and the selected volume. They do not magically monitor non-BFS file systems with equivalent semantics. Applications that support several file systems should expose the difference or provide an alternative watcher and scan implementation rather than promising BFS behavior everywhere.

Message delivery also means thread rules apply. Use a looper-owned handler, keep event processing short, and coalesce repeated updates before expensive sorting or thumbnail generation. A live query that causes a complete UI rebuild for every attribute write wastes the incremental information the file system provides.

On-disk structure and journaling

BFS organizes a volume around a superblock, allocation groups, block runs, inodes, directories, attributes, indexes, and a journal. The source’s Volume, Inode, BPlusTree, and Journal components make those responsibilities inspectable. Applications should not parse these private structures directly; the kernel file-system API and Storage Kit isolate software from implementation details and validation rules.

The journal records file-system metadata operations so recovery after interruption can restore a consistent structural state. Journaling is not the same as versioning, checksumming every user byte, or keeping a second copy of deleted data. It reduces a class of corruption from incomplete metadata updates; it cannot protect against a failing drive, accidental deletion committed normally, malicious writes, or missing backups.

Synchronization affects durability. A successful buffered application write does not necessarily mean every relevant block has reached stable storage. Software that promises durable transactions must use the available flush/sync semantics and account for storage hardware behavior. Closing a file and seeing the attribute in Tracker is not a complete power-loss test.

File-system repair tools operate below the Storage Kit abstraction and must match the implementation version. Before repairing, preserve an image when data matters. Rebuilding an index may restore query behavior without recovering an attribute whose underlying node is damaged.

Schema design for application metadata

Use namespaced, stable attribute names so unrelated applications do not collide. Specify types explicitly and plan migrations. If a value changes from integer to string, a new attribute or controlled migration is safer than silently writing mixed types under an existing indexed name.

Keep authoritative content and derived metadata distinct. A thumbnail, normalized artist name, or search token can be regenerated; a user’s only annotation cannot. Derived fields are excellent attribute candidates because indexing makes them discoverable and loss is recoverable. Essential data needs verified backup and export behavior.

Batch related changes thoughtfully. Readers can observe intermediate attribute states unless the application defines its own higher-level commit marker or replacement strategy. Live-query consumers should tolerate an entry that is temporarily incomplete while a producer writes several fields.

Test on an actual BFS volume, not only a host directory used during cross-development. Verify index presence, correct types, query matches, rename and copy behavior, live updates, reboot persistence, and transfer through the backup format you support. Also test missing indexes and malformed attributes; database-like lookup does not remove the need for defensive parsing.

What “database-like” accurately means

BFS provides persistent typed metadata, ordered indexes, predicates, and live result updates inside a file system namespace. Those are genuinely database-like capabilities, and they allow independent Haiku applications to share searchable metadata without maintaining a detached catalog that loses track of renamed files.

It does not provide joins, relational constraints, arbitrary query planning, or application-level multi-record transactions. Indexes cover declared fields on one volume, and file-system permissions remain the access-control boundary. For complex relationships or transactional business data, a database is still the appropriate tool.

The best design uses each layer for its strength: primary file data in the file, discoverable per-node metadata in typed attributes, high-value search keys in BFS indexes, and complex relational state in an application database when necessary. Haiku’s Storage Kit makes those choices accessible without exposing on-disk B+tree code to every application.

Related:

Sources: