Live Queries: Searching Haiku's File System Like a Database

A normal file search returns a snapshot. A Haiku live query begins with the same indexed BFS predicate but also delivers change notifications to a BMessenger target while the query remains open. An application can maintain a view of matching entries as attributes, names, and nodes change without rescanning an entire volume on a timer.

The feature is powerful because it combines file-system state with Haiku’s message model. It still requires careful application state: notifications can race with file deletion, values must be re-read, custom indexes may be absent, and a query is scoped to one volume. “Live” means changes are reported, not that every consumer automatically stays correct.

BQuery builds an indexed predicate

BQuery belongs to the Storage Kit and implements the BEntryList interface. Code selects a BVolume, builds a predicate, calls Fetch(), then iterates results with GetNextRef(), GetNextEntry(), or directory-entry methods. Rewind() restarts enumeration and CountEntries() exposes a count where supported.

A predicate can be supplied as an expression string or constructed through the stack API. PushAttr() adds an attribute name, typed value methods add operands, and PushOp() combines them with equality, ordering, containment, prefix/suffix, and logical operators declared in the public header. Typed construction helps keep numeric and string comparisons distinct.

The search depends on indexes on the target BFS volume. A file having Audio:Artist does not imply that the volume has an index for that name and type. Applications that define a schema should check/setup requirements and explain a missing index. A query over one volume never silently includes identically named attributes on another mounted volume.

Predicate input needs escaping and limits. If a user term is inserted into a raw expression, quotes or operators can alter its meaning. Prefer the builder API or a tested escaping function, preserve the expected type, and reject an expression too broad for an interactive operation when no useful index can constrain it.

SetTarget turns the query live

The current Query.h API exposes SetTarget(BMessenger messenger). A valid messenger directs live update messages to a handler owned by a looper. IsLive() reports whether the query is configured as live. Set the volume, predicate, and target before Fetch() so initial enumeration and subsequent change handling refer to one defined query.

BQuery query;
query.SetVolume(&volume);
query.PushAttr("Audio:Artist");
query.PushString("Miles Davis");
query.PushOp(B_EQ);
query.SetTarget(BMessenger(handler));
status_t status = query.Fetch();

Production code checks every returned status, validates the messenger, and keeps the BQuery alive as long as updates are required. Destroying or clearing the object ends that subscription. A stack-local query that goes out of scope immediately after collecting results is not a persistent live query.

The target receives messages in its looper context. This provides serialization with the handler’s other messages but does not make expensive processing safe on a UI thread. Read necessary attributes, update a small model, and schedule thumbnailing or heavy sorting elsewhere.

Initial enumeration and updates form one state machine

The first result set and later notifications can overlap in time. A file may change after it is enumerated but before the UI inserts it. The application needs one idempotent model keyed by a stable node/entry identity, not an append-only list that assumes all update messages arrive after enumeration.

For an addition/change event, open the referenced entry, re-read the attributes needed by the predicate and display, and insert or update it only if it still matches. For removal, delete the key if present; receiving removal twice should remain harmless. If the file vanished before it could be opened, treat that as an expected race.

Renames complicate path-based identity. A node can remain the same while its directory/name changes. Store a node reference or other stable identity alongside the current entry_ref, and update presentation when the name changes. Do not treat the original path as immutable proof of the object.

After restart, missed notification, or unrecoverable parse error, rebuild from a fresh query. Incremental state should be disposable because BFS remains authoritative. A periodic full rescan should not be necessary for normal operation, but a controlled reconciliation path makes the consumer robust.

Index changes drive integrated notification work

BFS already updates per-volume indexes when indexed attribute values or indexed names change. The file-system query implementation tracks open queries and identifies changes relevant to their predicates, then communicates them through the kernel/userland query interface. This is integrated machinery, not “free” behavior with no tracking cost.

The design avoids an application repeatedly walking every directory and reading every file’s metadata. It also avoids a detached crawler database becoming stale after a rename outside the application. The index belongs to the volume and the file-system mutation updates it at the source.

There is still overhead: index maintenance adds write work, every live query consumes kernel/application resources, and high-frequency attribute changes can generate many messages. Close queries that are no longer visible or useful. Coalesce model updates before re-sorting a large view.

The query mechanism cannot manufacture equivalent support on a file system without BFS indexes. A cross-volume search service may combine one BQuery per compatible volume and a scan/watcher strategy elsewhere, but it must label their consistency and latency differences honestly.

Saved queries and Tracker

Tracker exposes BFS queries as user-facing searches that can be saved and reopened. A saved query stores the predicate and presentation-related information rather than copying matching files into a physical folder. Opening it reconstructs a dynamic result view over the relevant volume.

That makes a query feel like a virtual folder: “unread mail,” “images modified recently,” or “audio by this artist” can collect entries regardless of directory. Files remain in their real locations, so deleting an item from the query view acts on the actual file, not a duplicate search result.

Saved queries can become invalid when a volume is unavailable, an index is removed, or an attribute schema changes. Software creating saved predicates should use stable attribute names/types and expose failures clearly. A query authored for one application’s private metadata is not automatically meaningful to another.

Date-relative searches deserve special care. A literal timestamp saved today does not automatically mean “the last seven days” forever unless the saved-query format or application explicitly re-evaluates that relative expression. Test the exact predicate instead of inferring semantics from the displayed label.

Live queries versus node monitoring

Haiku also provides node monitoring. A node monitor watches specified nodes/directories or mount events, while a live query watches membership in a predicate result set across a BFS volume. They answer different questions.

Use node monitoring when the application owns a known directory tree or specific file and needs raw entry/attribute/stat changes. Use a live query when location is secondary and indexed metadata defines membership. Combining them can be valid, but duplicate events and ordering must be reconciled.

A live query does not report every change to every matching file as an application-specific audit log. It reports what the file-system query protocol defines for maintaining results. If detailed content history is required, the application needs its own versioning or journal rather than assuming query messages are durable records.

Neither mechanism replaces opening and validating the file at use time. Notifications describe that something changed; access permissions, current type, and current content still determine whether an action can proceed.

Performance and correctness checklist

Create only indexes that provide real query value, with correct types. Build predicates through typed methods where possible. Keep one BQuery per volume/predicate lifecycle, check Fetch(), and retain it while live delivery is expected.

Route updates to a looper-owned handler. Key results by stable identity, make operations idempotent, re-read authoritative attributes, and tolerate deletion/rename races. Batch UI updates and move expensive derived work off the handler thread.

Test creation, attribute mutation into and out of the predicate, rename, move within the volume, deletion, index absence, volume unmount, target shutdown, and high-frequency updates. Close/reopen the query and compare its rebuilt state with the incremental model.

The distinctive feature is not simply fast search. BFS indexes and BQuery let a Haiku application treat file-system metadata as a maintained, message-producing view. Used defensively, that turns saved searches and metadata-driven browsers into lightweight system capabilities rather than private polling databases.

Related:

Sources: