Haiku Media Producers and Consumers: Format Negotiation and Buffer Ownership

Haiku’s Media Kit moves media between nodes using negotiated connections and shared buffers. BBufferProducer describes outputs and sends buffers; BBufferConsumer accepts inputs and receives them. The API is asynchronous and graph-oriented: a connection is not a function call that copies a frame into another object’s vector. A robust node must negotiate an exact format, respect buffer-group ownership, preserve timing metadata, and return storage to the graph when it is done.

This guide focuses on the boundary between producer and consumer. It is not a full tutorial for implementing a codec, a camera driver, or a Media Kit add-on. The headers define abstract callbacks and ownership hooks; real-time behavior and supported formats depend on the node and the connected graph.

The graph connection is a negotiation protocol

The roster does not simply join arbitrary endpoints. During BMediaRoster::Connect(), the producer and consumer participate in checking a proposed media_format; the selected format is returned in the actual media_output and media_input. Producer callbacks include FormatProposal(), PrepareToConnect(), and Connect(). The consumer implements AcceptFormat() and Connected(). A node should validate fields it actually understands and either accept a supported format or return a status that causes negotiation to fail.

Treat wildcard fields as requests for negotiation, not as usable runtime values. If a producer receives a partially specified format, it should specialize it to a concrete format it can emit. The producer header says that FormatProposal() must put a supported format into the in/out structure when the proposal is not acceptable; a successful result means the requesting consumer is expected to like the proposed format. Do not silently accept a format and then emit buffers with different channel count, sample rate, color space, dimensions, or encoded representation.

Connection callbacks should be transactional. Prepare any state needed for the connection, but publish it as active only when the callback reports success. If a later callback fails, undo allocations and reservations. On disconnect, release format-specific state and stop sending on that endpoint. Test partial failure at every callback boundary rather than only exercising a successful desktop audio route.

`BBuffer` is a shared payload plus metadata

BBuffer describes a chunk of memory that can be shared between nodes. Data() points at its payload, SizeAvailable() reports the capacity, and SizeUsed() reports how much content is meaningful. A producer should set the used size after writing. A consumer must use the used length and negotiated media_format; reading all available capacity can consume uninitialized padding or bytes from an earlier use of the pool.

Media buffers also carry a media_header, including timing and format context. Preserve timestamps and relevant header fields when transforming or forwarding data. A timestamp is expressed in the Media Kit performance-time model, not an arbitrary wall-clock value. Do not replace it with the time at which a consumer happened to run; scheduling delay and processing latency make those values different.

The buffer’s existence does not make its data permanently owned by the callback. The graph reuses buffers. A consumer should process only while it has the buffer, avoid retaining Data() after the callback contract ends, and recycle the buffer according to the API. If data must outlive the callback or cross into a worker queue, copy it into storage the application owns or use a supported buffer-cloning/forwarding design that preserves lifetime.

Buffer groups are capacity plans

BBufferGroup manages a pool of media buffers. The group must be sized for the concurrency and latency of the connected pipeline; a group that is too small can stall producers, while an oversized pool consumes memory without automatically improving throughput. Estimate the bytes per buffer from the negotiated format, then include multiple buffers for producer in-flight work, consumer processing, and any documented graph latency. Measure rather than treating one fixed count as universal.

The SetBufferGroup() contract is especially important. A producer callback may need to pass the group upstream, keep the supplied group, or replace an old group. The header warns that failing to delete a group that is no longer owned or passed onward can leak memory and prevent reclamation. Track the current group explicitly, define who owns it on each path, and make a replacement atomic with respect to buffer production. Do not free a group while a node can still submit one of its buffers.

The consumer-side SetOutputBuffersFor() call includes a willReclaim argument. Read the current header contract and use it intentionally; it changes whether the sender expects to reclaim the group. Never infer ownership merely from the fact that a pointer was passed. Document the contract in the node’s connection state and test the teardown path under both reclaim and non-reclaim behavior.

Keep `BufferReceived()` bounded

BBufferConsumer::BufferReceived() runs when a buffer arrives. Treat it as a latency-sensitive callback even if a particular test machine has spare CPU. Avoid filesystem I/O, network requests, blocking locks, unbounded allocation, or waiting for a UI reply in the callback. The safe pattern is to validate the header and size, copy or enqueue the minimal work needed, recycle the original buffer promptly, and let an ordinary worker thread handle slower tasks.

That pattern has a queueing tradeoff: if the worker is slower than the producer, a copied queue can grow without bound. Use a bounded queue with an explicit policy such as drop-oldest for a preview or stop-recording with a visible error for archival capture. The right policy is application-specific. It should be visible in metrics, not hidden as silent memory growth.

If a consumer transforms a buffer and sends one downstream, keep the producer/consumer responsibilities separate. The output buffer must belong to an appropriate group and satisfy the negotiated downstream format. Preserve or deliberately rewrite timing metadata according to the transformation. If the operation fails, return/recycle the input and do not send a malformed output. Error handling must not leave the graph waiting forever for a buffer that was never submitted.

Format changes and latency are runtime events

A connection’s format can be renegotiated. A producer can ask a consumer to accept a format change; while the change is pending, the producer header explicitly forbids sending buffers through SendBuffer(). Pause output, complete the handshake, switch internal state at the agreed boundary, and resume only after the accepted format is known. Tag concurrent requests so a stale completion cannot overwrite a newer format decision.

Latency is part of the graph contract. Producers provide latency information and can receive late notices; consumers report their own downstream processing latency. A late notice is diagnostic and coordination input, not permission to corrupt timestamps or skip required frames silently. Record the amount late, performance time, buffer identity, and queue depth. Then decide whether to drop, resynchronize, or notify the user based on the media product’s requirements.

A defensive consumer sketch

void Consumer::BufferReceived(BBuffer* buffer)
{
    const media_header* header = buffer->Header();
    const size_t used = buffer->SizeUsed();
    if (header == NULL || used > buffer->SizeAvailable()) {
        buffer->Recycle();
        return;
    }

    // Copy/enqueue only bounded work that outlives this callback.
    if (!queue.TryCopy(buffer->Data(), used, *header))
        RecordDroppedBuffer(*header);
    buffer->Recycle();
}

The sketch illustrates validation and lifetime discipline, not a complete subclass. Confirm the exact buffer-header accessor and recycling behavior against the target Haiku headers. Do not hold a raw pointer in queue; the example assumes it copies both the payload and the metadata. Some consumers should transfer or forward a buffer instead, but only if the negotiated group and ownership rules allow it.

Verification plan

Test both audio and video formats if the node claims both. Exercise no free buffer, undersized payload, invalid SizeUsed(), malformed metadata, consumer slowdown, format rejection, format change while data is flowing, disconnect during callback, and producer removal. Track pool capacity, buffers outstanding, callback duration, late notices, dropped buffers, negotiated format, and timestamp deltas. Confirm the pool returns to its baseline after repeated connect/disconnect cycles.

The central rule is simple: a media buffer is a temporary graph resource, not an application-owned byte array. Negotiate before sending, keep each buffer’s meaningful length and performance-time metadata correct, bound callback work, and make buffer-group ownership explicit from connection through teardown.

Related:

Sources: