Haiku HPKR Repository Indexes: Hashes, Mirrors, and Refresh Lifecycles

Haiku’s package index is not another .hpkg package. A repository publishes an HPKR binary catalog that describes repository metadata and package attributes; package files remain separate artifacts. That distinction matters when debugging a stale catalog, a checksum mismatch, a mirror that serves inconsistent files, or a HaikuDepotServer import that has been accepted but is not yet visible.

This article focuses on that repository lifecycle rather than the HPKG container covered by the package-format deep dive. Format fields and implementation behavior below are checked against the Haiku source snapshot at commit 388ca6823493ae937134949782be1eac9f5343b9 and HaikuDepotServer snapshot 299f5b911a6ac7bf27a9133afffb46cce1ec0c33. Treat implementation details as specific to those snapshots, not as a promise that every older Haiku release behaves identically.

HPKR is a catalog, not a bundle of package payloads

The pinned Haiku file-format specification defines an HPKR file as a compact container with three conceptual parts: a header, a heap, and structured metadata stored at the end of the heap. The repository-info section is an archived BMessage representing BRepositoryInfo; the package-attributes section contains the metadata for packages in that repository. The specification explicitly says the repository file contains no package files.

At this pinned source revision, the documented HPKR header has the hpkr magic, a header size and total file size, format version and minor version, heap compression and size fields, a repository-info length, and package-attribute and string-table lengths/counts. The same specification identifies format version 2 and minor version 1 at that revision. The minor-version guidance says readers should ignore unknown attributes when encountering a greater minor version, but that is not permission to ignore malformed lengths, invalid encodings, or a format version the reader cannot parse.

The online repository layout has separate well-known resources. The pinned HaikuDepotServer repository guide describes repo as the binary HPKR index and repo.info as a plain-text description. Its wording for the packages/ path is inconsistent about the package format, so do not use that sentence as evidence that package payloads are HPKR. The pinned Haiku repository build script shows .hpkg artifacts being copied into that directory. Do not rename or mentally merge these objects: the index answers “what packages does this catalog advertise?” while a package URL answers “can I retrieve this artifact?”

Repository identity is not a mirror URL

Haiku has two related values that are easy to confuse. BRepositoryInfo::BaseURL identifies the URL for one mirror, while BRepositoryInfo::Identifier is a URI-form identity that applies across mirrors. The public header states that distinction directly. A mirror can change without necessarily changing the logical repository identity.

The external repo.info is another boundary: it is a text file alongside repo, not the HPKR’s embedded BMessage. HaikuDepotServer’s guide says the identifier value should remain consistent across mirrors and models its own Repository as a collection spanning architectures, with RepositorySource entries for architectures and one or more RepositorySourceMirror URLs per source. In other words, HDS’s administrative repository object, an on-net repository source, its mirror URL, and the repo bytes are related but not interchangeable identifiers.

For an operator, compare the configured base URL and stable identifier separately. pkgman list-repos -v prints the configured base URL, identifier, priority, and, when it can open the cached index, additional fields such as architecture and package count. A changed mirror URL is not by itself proof that the repository identity changed; an unexpected identifier or architecture is a reason to stop and verify the repository configuration rather than silently treating it as the same source.

Keep the two SHA-256 checksums separate

Haiku uses a package-level checksum attribute and a repository-file sidecar for different comparisons:

Each package record in HPKR can carry package:checksum, documented as the SHA-256 checksum of that .hpkg artifact in hexadecimal form. Haiku’s package download request accepts this expected checksum and schedules validation after fetching the file.
The repository endpoint can publish repo.sha256, a separate text file containing the SHA-256 digest of the repo index itself. The official repository build script creates repo and then writes this digest sidecar.

They do not hash the same object. A package’s package:checksum does not validate the HPKR file, and repo.sha256 does not validate every .hpkg payload. The sidecar is not a signature embedded in the HPKR format.

The pinned pkgman refresh path requests repo.sha256 from the configured base URL and compares it with the local cache digest. If it differs, the client downloads repo, hashes those bytes, checks them against the fetched 64-character checksum, and schedules cache activation with checksum validation as a prerequisite. The checksum accessor implementation reads the checksum text; the refresh implementation and cache-fetch path show the comparison and activation ordering.

For a controlled endpoint check from a host with curl and sha256sum, fetch both files from the same base URL and compare them:

set -eu

base='https://repository.example.invalid/haikuports/master/x86_64/current'
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT

curl -fsSL "$base/repo.sha256" -o "$tmp/expected"
curl -fsSL "$base/repo" -o "$tmp/repo"
expected=$(tr -d '\r\n' < "$tmp/expected")
sha256sum "$tmp/repo" > "$tmp/actual-line"
actual=$(awk 'NR == 1 { print $1 }' "$tmp/actual-line")

valid_sha256() {
    [ "${#1}" -eq 64 ] || return 1
    case "$1" in
        *[!0123456789abcdefABCDEF]*) return 1 ;;
    esac
}

if ! valid_sha256 "$expected" || ! valid_sha256 "$actual"; then
    printf '%s\n' 'invalid SHA-256 text' >&2
    exit 1
fi

expected=$(printf '%s' "$expected" | tr 'A-F' 'a-f')
actual=$(printf '%s' "$actual" | tr 'A-F' 'a-f')

printf 'expected: %s\nactual:   %s\n' "$expected" "$actual"
test "$expected" = "$actual"

This is a consistency check for one endpoint at one moment. Inference and security boundary: because the shown pkgman flow fetches the expected digest and the index from the configured base URL, matching values prove that the downloaded repo matches the checksum that endpoint supplied. By itself, the comparison does not identify a publisher or provide an independent trust anchor; an attacker able to replace both responses could make them agree. Do not describe repo.sha256 as publisher authentication. Use the repository’s actual trust and transport policy, and treat any separately documented signature-verification mechanism as a distinct control.

The .invalid host above is a deliberately non-routable placeholder; replace it with the repository’s actual base URL before running the commands.

`pkgman refresh` and HDS import are different pipelines

On a client, pkgman refresh <repository-name> tests the configured repository’s checksum-and-index path without installing packages. The source for the refresh command accepts all repositories or selected names, and pkgman list-repos -v can then expose whether a cached index was opened and what metadata it reports. If the checksum matches an existing cache, the current implementation need not download the index again. A successful digest comparison is not a full package-download test, and a checksum match alone does not prove that the HPKR structure is usable; inspect the resulting cache state as well.

HaikuDepotServer has its own import pipeline. Its repository documentation describes periodic imports and a manual import API. The ingress runner in the pinned HDS snapshot fetches and parses repo.info, then downloads HPKR to a temporary file and iterates package attributes. The inspected ingress methods do not reference repo.sha256; do not assume that HDS uses the same checksum validation sequence as pkgman. The runner commits each permitted package import through its own context and handles package-version deactivation separately when a package is absent from the refreshed index. This implementation does not establish an all-or-nothing publication or availability guarantee for the entire repository. The helper used for URL fetching may have its own behavior, but that is not documented as an HPKR checksum check in these runner methods.

An HDS import request is asynchronous. The API guide says the call starts work in a background thread, coalesces overlapping requests, and returns HTTP 200 when the job is accepted. If a repository password is configured, the request uses HTTP Basic authentication; the documented example ignores the username and uses the configured password. A 200 response therefore means “accepted,” not “all package records are already queryable.”

The repository guide gives a daily HPKR import for the HaikuPorts x86_64 source as a specific example, while describing periodic processing generally. Do not turn that example into a universal refresh SLA. Administrators can use the source-specific import endpoint when a timely refresh is needed, then verify HDS’s indexed package state after the background job runs.

Diagnose the layer that failed

Start with the client-side inventory and a single explicit refresh:

pkgman list-repos -v
pkgman refresh HaikuPorts
pkgman list-repos -v

Record the base URL, identifier, architecture, package count, exact refresh error, and time of the attempt. Do not delete the local repository cache just because one refresh failed; first establish whether the endpoint is reachable and whether its repo.sha256 and repo bytes match. If an independently fetched pair disagrees, capture both from the same mirror before switching to another one. Possible-cause inference: a mismatch can be consistent with an incompletely published pair or a stale intermediary cache, but the digest mismatch alone does not identify which component is wrong.

If pkgman refreshes and the catalog opens but an install later cannot retrieve a package, investigate the artifact URL separately. The repository format’s package checksum lets the client detect bytes that do not match the advertised package digest; it does not make an absent or unreachable artifact available. For HDS, distinguish three observations: an import request accepted, the background ingress job completed, and the expected package appears in HDS query results. They are separate checkpoints.

Operational acceptance criteria

Treat a repository update as verified only when the correct logical identifier and architecture are configured, the selected endpoint serves a consistent repo.sha256/repo pair, the client can open the refreshed cache, and the package artifact path required by the operation is independently reachable and matches its catalog digest. For HDS, additionally distinguish request acceptance from completed ingestion and observed package records. None of these checksum steps, alone, authenticates the publisher; keep provenance, endpoint security, mirror operations, and package-byte integrity as separate review items.

Related:

Sources: