Haiku Mail: mail_daemon, BFS Message Files, and Queries

Haiku’s mail workflow is built around system services and ordinary filesystem objects rather than a monolithic mail database hidden behind one application. mail_daemon coordinates account-related work; the Mail Kit exposes message and account APIs; messages are stored as files; and BFS attributes make selected message properties searchable through Tracker and query tools. These layers are related, but they are not interchangeable: an attribute index is not the canonical body, a message file is not necessarily ready to send, and an account’s outgoing SMTP settings do not determine how an incoming mailbox is synchronized.

This architecture is useful for integration and recovery because it exposes more of the state to the user. It also creates common diagnostic mistakes. Deleting a metadata attribute to “fix search” can leave the message itself untouched but unindexed. Moving a message file outside a managed mailbox may remove it from the Mail app’s normal view. Editing a file’s raw RFC 822 representation while the daemon or application is using it can produce inconsistent state.

The daemon is the coordinator, not the whole mail client

Haiku’s User Guide describes mail_daemon as a service used by mail applications to send and receive messages and to coordinate mailbox operations. The Mail preferences configure accounts, incoming retrieval (including POP or IMAP where supported by the release), outgoing delivery through SMTP, filters, and destinations. The Mail application provides user-facing composition and mailbox workflows, while the daemon handles background work and communicates through Haiku’s application/message infrastructure.

Keep those roles separate in a bug report. If an account cannot connect, record the protocol, server, port, TLS/authentication selection, and exact error without posting credentials. If a message is present on disk but not displayed, check mailbox placement, attributes, and the relevant query/index behavior. If composition succeeds but sending fails, distinguish the outgoing account and SMTP path from the incoming mailbox path.

Daemon state is dynamic. Restarting it can interrupt or reschedule work; it does not repair a malformed account or restore a deleted message. Before changing account settings, write down whether the affected message is local-only, remote, or part of a synchronized folder. POP-style retrieval, IMAP synchronization, and local folders have different recovery properties, and the server may not retain an offline copy in every configuration.

A message file contains more than one view of a message

Haiku stores individual mail messages as files. The user guide explains that important header and status values are also represented as BFS attributes, allowing Tracker queries and the mail application to find messages efficiently. The file content carries the message representation; attributes provide indexed metadata such as sender, subject, status, or other mail properties used by the system. Exact attribute names and indexing behavior can vary with software version and configuration, so use current Haiku documentation and inspect a sample message rather than hard-coding assumptions from an old BeOS guide.

This design makes searches feel integrated with the filesystem, but it is important to distinguish authoritative content from derived metadata. A search query can miss a file whose attribute is absent or whose index is stale even when the body is readable. Conversely, an attribute can exist on an incomplete, malformed, or manually copied file. A query result is a discovery aid, not proof that the message parses correctly or has been successfully delivered.

The Mail Kit’s BEmailMessage represents an email and its components. It can read from a position-I/O stream or a message reference, parse components, render RFC 822 data, attach files, and send through an account. That API boundary is preferable to hand-editing storage files: it knows how to interpret headers, bodies, and attachments as a message. Applications should check InitCheck(), every returned status, and the result of rendering or sending instead of assuming a file was accepted because it was created.

BFS attributes make mail discoverable, not self-healing

Attributes support queries such as “all mail from this sender” or “all messages with a given status.” Live queries can update a Tracker window when indexed attributes change. This works because the filesystem and query services provide the mechanisms; the Mail application supplies conventions and user experience. If a filter changes a status attribute, that can affect what a query shows without changing the message body.

If a message disappears from a saved query, first inspect whether the underlying file remains in the intended mailbox and whether the attribute required by the query exists. Then check the query predicate and volume index. Do not immediately rebuild every BFS index or rewrite the mail directory. A focused test with one known message can distinguish a missing attribute from a path, permissions, or application-state problem.

Attributes are not a backup. A disk image or file-level backup that omits filesystem metadata may preserve raw message bytes while losing the attributes that make mail search and status workflows convenient. The official backup article should be followed for the chosen tool and filesystem. After a restore, validate both message parsing and the ability to query messages; do not infer one from the other.

Account configuration and privacy boundaries

Incoming and outgoing protocols are separately configured. An IMAP account can synchronize server folders while SMTP submits outgoing mail; a receive failure does not prove SMTP is down. Verify hostnames, ports, TLS mode, authentication requirements, and certificate/time errors independently. Never paste an account password, OAuth token, or full private message into a public bug report. Redact addresses and subjects when they identify people or organizations.

Filters can move or otherwise process messages. When debugging unexpected placement, disable or narrow one filter at a time and test with a non-sensitive message. Record the original mailbox and the filter order before changing rules. A message “missing from Inbox” may have been moved successfully rather than deleted, and a query based on status may hide it even when the file remains present.

Safe recovery workflow

Start with a backup of the affected mail tree using a method that preserves the relevant filesystem data and metadata. Do not copy or rename message files while the Mail application is actively modifying them. Quit the UI and allow daemon activity to settle before a controlled offline inspection. Keep an untouched copy and perform experiments on a duplicate.

Inspect one known message through the Mail application or a small program using BEmailMessage. If parsing fails, preserve the file and capture the exact error. If parsing succeeds but search misses it, compare its attributes with a message that does appear and verify the attribute indexes and query predicates. If the remote account and local store disagree, do not delete local data until the server-side state and synchronization behavior are understood.

Reconstructing an index and reconstructing message content are separate operations. An index rebuild may make existing data discoverable but cannot restore a body or attachment that was lost. A message re-download may restore the body but duplicate entries or lose locally edited metadata. Choose a repair only after identifying which layer is defective.

Application integration without filesystem coupling

An application that creates mail should use the Mail Kit and supported account APIs rather than writing private filenames and attributes into a user’s mail hierarchy. Build a BEmailMessage, validate its body and attachments, render or send with checked status, and let the system manage storage conventions. If a feature intentionally imports a file as mail, use the documented message format and leave user-controlled mail folders untouched until the import is committed.

For a mailbox viewer, do not scan every volume for files with an email-looking suffix and present them as valid mail. Use known folders, supported message APIs, and a clear user-selected import path. Filesystem discovery should respect volume permissions and symlink behavior. Large mailbox scans belong on a worker thread, with progress and cancellation, not in a window message handler.

Verification matrix

Test receiving and sending independently with a disposable account. Confirm a newly received message can be opened, searched by an expected metadata field, moved by a filter, and recovered from backup. Test offline folders and a temporarily unavailable server without deleting local state. After restoring a sample, verify message body, attachments, attributes, status, and Tracker query results separately.

When reporting a defect, include Haiku revision, mail application/daemon version, account protocol (not credentials), incoming/outgoing stage, mailbox path class, whether the raw message file is present, and whether BEmailMessage parses it. This layered evidence makes it possible to tell a server issue from a query/index issue or an application bug.

The practical model is a service coordinating mail operations, files holding messages, BFS metadata accelerating discovery, and applications presenting those layers. Recovery is safest when it preserves original files, checks metadata separately, and avoids treating a query result as the message itself.

Related:

Sources: