Haiku BNetBuffer: Typed Serialization Without Inventing a Protocol

BNetBuffer is a Network Kit utility for appending and removing typed values from a byte buffer. It helps turn a protocol’s fields into a sequence of bytes and back, but it is not itself a network transport, a self-describing serialization format, or a complete wire protocol. The application must still define field order, message boundaries, maximum sizes, versioning, error behavior, and how untrusted input is rejected.

This distinction prevents a common integration mistake: believing that a “network buffer” automatically provides framing and interoperability. BNetBuffer exposes operations such as AppendUint32(), AppendString(), RemoveData(), and Size(). The caller decides what those bytes mean and how a receiver knows where one message ends and another begins.

Typed methods encode different types differently

The Haiku implementation converts 16-, 32-, and 64-bit integer values to big-endian form before appending them and converts them back when removed. This is useful for an explicit integer wire representation across architectures. Eight-bit values are copied directly. The current implementation appends float and double storage as bytes without the same byte-order conversion, so applications should not assume that those values are a portable network representation across architectures or ABIs.

AppendString() appends the string’s bytes, including its terminating NUL. The method does not add a length prefix. A parser must already know how many bytes to remove or define a bounded string convention. RemoveString() accepts a destination buffer and size and removes exactly that many bytes; it does not independently prove that the bytes contain a NUL terminator or valid text encoding.

The API also returns status codes. Check every append and remove. A failed append can leave a partially constructed logical message; a failed remove means the input did not contain enough bytes for that field. Do not continue parsing as if the field existed.

Define explicit framing before using a buffer

For a stream transport, a receiver may get fewer bytes than requested or several messages in one read. BNetBuffer does not solve that framing problem. A simple protocol can define a fixed header with a version, type, and payload length, then validate the declared length before reading the body. More sophisticated designs may use delimiters, checksums, or authenticated envelopes, but each adds rules that both parties must implement consistently.

BNetBuffer out;
if (out.InitCheck() != B_OK)
    return B_ERROR;

if (out.AppendUint16(kVersion) != B_OK
    || out.AppendUint16(kMessageType) != B_OK
    || out.AppendUint32(payloadLength) != B_OK)
    return B_ERROR;

// Append bounded, application-defined payload bytes only after validation.

On input, first read a complete header into a bounded buffer, decode each integer, reject unsupported versions or message types, and confirm that the payload length is below an application limit. Only then allocate or read the body. Never allocate an attacker-controlled size directly from the wire without a cap and overflow-safe arithmetic.

For a datagram transport, the datagram boundary already separates messages, but the application still needs size limits and schema validation. A truncated or malformed datagram should fail as one message. Do not treat untrusted leftover bytes as an additional valid message unless the protocol explicitly permits multiple messages per datagram.

BNetBuffer is a byte container, not a security parser

The class does not authenticate a sender, encrypt content, validate domain values, or define a safe message schema. Even a successfully decoded integer can represent an invalid command or out-of-range index. The receiver should validate every enum, length, count, and string before using it. A string should be bounded and decoded under a defined encoding, typically UTF-8 if the protocol requires text.

Avoid casting Data() to a struct pointer and trusting compiler padding or alignment. C++ structure layout can differ across compilers, ABIs, and versions. Prefer explicit typed append/remove calls for the supported primitive types and explicit byte order for any type whose representation the library does not normalize. If the protocol requires IEEE floating-point bytes, define that requirement and encode/decode it explicitly.

Appending a BMessage with AppendMessage() should not be assumed to create an authenticated or universally versioned protocol. Flattened messages are Haiku-specific serialization, and cross-version or cross-platform compatibility must be evaluated against the documented BMessage contract. For external protocols, specify the exact wire schema instead of treating a local object representation as a standard.

Partial reads and cursor state matter

BNetBuffer removes bytes from its internal buffer as fields are consumed. A parser that removes a header and then discovers the body is incomplete must decide whether to retain the partially consumed object, roll back, or discard the entire input. The correct choice depends on whether the buffer represents a stream accumulator or one already-complete frame.

One robust architecture is to read complete frames into a bounded temporary buffer, then parse that buffer from the start. Another is to keep explicit parser state and a retained byte queue across reads. Avoid mixing these models. If you mutate the cursor and then silently retry from the beginning, fields can become misaligned; if you discard a partial stream frame, the connection may need to be closed or resynchronized.

Before parsing, inspect Size() and ensure the expected header is available. After parsing a complete frame, verify that the number of consumed bytes matches the declared frame length. Extra bytes may indicate a concatenated frame only if that is part of the protocol; otherwise treat them as malformed trailing data.

Strings and arrays need an application convention

Because AppendString() appends NUL-terminated bytes without a prefix, it fits internal protocols whose receiver already knows the field boundaries. It is a poor fit for arbitrary user text that may contain NUL bytes or for records where the string length is not known in advance. Define a length-prefixed string explicitly: write a bounded byte count, append exactly that many bytes, and validate both the length and character encoding when reading.

Arrays follow the same rule. A count must be validated against both the protocol maximum and the remaining bytes before allocating storage. A count-times-element-size calculation can overflow; perform checked arithmetic before multiplying. If every field is mandatory, a missing field invalidates the message rather than turning into a default zero.

Initialize, reuse, and bound allocation

Construct BNetBuffer with a capacity appropriate for expected messages when known, then check InitCheck(). Repeated appends may grow internal storage; unbounded input can create memory pressure. Set maximum frame sizes at the transport boundary, not only after the buffer has already expanded. For large payloads, stream or chunk the data under explicit limits rather than assembling an arbitrarily large in-memory packet.

There is also a difference between buffer size and allocated capacity. Size() reports bytes currently stored according to the class contract; it should not be interpreted as an admission policy. A network service should enforce its own limits before adding data and before accepting peer-supplied payloads.

Tests should target the wire contract

Write encode/decode tests for minimum and maximum values, signed integer boundaries, empty strings, non-ASCII UTF-8 text, short input, unsupported versions, oversized lengths, and trailing bytes. Add cross-architecture golden byte vectors for multibyte integer fields. The implementation’s integer methods use big-endian conversions, so golden vectors can catch accidental changes. Do not use host-native float byte vectors unless the protocol intentionally specifies that representation.

Fuzz the parser independently from the socket code. Feed truncated headers, inconsistent counts, embedded NULs, invalid encodings, and random bytes. Confirm that failures return explicit status and do not produce partial state changes. A socket test should separately cover stream fragmentation, coalesced frames, connection close mid-frame, and datagram truncation.

BNetBuffer is most useful when its scope remains small: checked primitive serialization over an application-defined byte sequence. It can remove repetitive byte-buffer code and provide predictable integer byte order. It cannot replace a wire-protocol design. Production reliability comes from explicit framing, bounded sizes, exact validation, and tests that treat every received byte as untrusted.

Related:

Sources: