Skip to content
RetrogamingDeep Dive Published Updated 9 min readViews unavailable

Doom WAD Internals: Lump Tables, Search Order, and Defensive Parsing

Read Doom's WAD directory as an indexed archive, understand classic patch precedence, and build a bounded inspector without trusting old loader code.

A Doom WAD is best understood as a small archive whose directory points to named byte ranges called lumps. The word does not promise a particular asset type: a lump may hold a level structure, image, sound, music data, text, or another engine-specific object. The container supplies names, offsets, lengths, and ordering; the meaning of each payload comes from the engine and the conventions of the content loaded with it.

This article follows the original id Software Linux Doom 1.10 loader as a historically important implementation, then derives a safer inspection strategy. The old code is useful evidence for how Doom interpreted archives, not a security reference implementation. Work only with files you own or are authorized to inspect. The examples list directory metadata; they do not extract, redistribute, or provide copyrighted game data.

The directory is an index, not a sequence of embedded files

The familiar classic WAD framing begins with a 12-byte header: a four-byte marker (IWAD or PWAD), a 32-bit lump count, and a 32-bit offset to the directory. Each directory record is 16 bytes: a 32-bit data offset, a 32-bit byte length, and an eight-byte name field. In the classic Doom implementation, the integer fields are converted through the engine’s byte-order helper, and the directory is read from the offset in the header. Do not infer that lump data immediately follows the header or that the directory is at the end of the file. The index is explicitly located by the header.

IWAD and PWAD describe roles in the original engine’s loading model, not a cryptographic trust level. Doom’s source checks that a WAD identifier is one of these markers, but a matching four-byte signature says nothing about authorship, legality, completeness, or safety. A WAD with a familiar name may still contain malformed offsets or payloads that trigger bugs in an engine or in a third-party tool.

The eight name bytes are a fixed-width field, not necessarily a C string with a terminator. The original lookup routine builds a nine-byte search buffer, uppercases it, compares eight bytes, and scans the combined lump list from the end toward the beginning. That is an implementation detail with consequences: lookup is case-insensitive in this code, and a later duplicate name is selected first. A safe inspector should preserve the original eight bytes for display and comparison instead of assuming every name is valid UTF-8 or zero-terminated.

Archive load order becomes content precedence

The W_InitMultipleFiles comments and lookup code in Linux Doom 1.10 document a deliberately simple patch model. Files are added to a shared lump table in the order supplied to the loader. Name lookup scans that table backwards, so if a later archive contains a lump with the same name as an earlier lump, the later occurrence wins for a name-based request. This lets a patch archive replace a resource without rewriting the base IWAD.

That rule is not equivalent to “the last file on disk wins.” It concerns the order the application passes files to its loader and the lookup semantics of this particular code. A different source port can add namespaces, hashes, namespaces-aware lookup, compatibility exceptions, or different command-line precedence. Record the engine, version, command line, and ordered archive list when a reproducibility report matters. A screenshot of a WAD directory alone does not prove which duplicate resource the running engine selected.

Directory order and physical byte order answer different questions. A directory can point to data ranges wherever they reside in the file, while the directory entries themselves define the logical list consumed by this loader. A tool that sorts entries by data offset may create a more visually tidy report while silently changing duplicate resolution if that report is later treated as an archive rebuild plan. Keep the source order as an explicit field.

Markers and naming conventions organize content but are not a schema validator. Map lumps, for example, rely on a family of names and expected neighboring records. Other asset types have their own layouts. A generic WAD parser can safely report that a named range exists; it cannot conclude that the payload is a valid map or image without a second parser for that resource type. Those parsers should each have their own length, count, and offset checks.

Why copying the 1997 loader is not defensive parsing

The source is valuable for behavior, but its archive reader predates modern untrusted-input practices. It reads the header and directory into fixed C structures, computes a directory allocation from the file-provided count, seeks to the supplied offset, and reads records. The shown code does not establish the full set of checks a contemporary tool should apply before using those values. Historical code should therefore be cited to explain the original contract, not presented as proof that every malicious or truncated input is handled safely.

A defensive reader should first obtain the file length from the operating system, then validate the header before allocating memory. Check that the directory offset is nonnegative, that the count is below an application-defined resource limit, and that count * 16 cannot overflow the integer type used for arithmetic. Validate the complete directory interval using subtraction-based bounds checks: directory_offset <= file_size and directory_size <= file_size - directory_offset. For every lump, reject negative offsets or sizes, then ensure offset <= file_size and size <= file_size - offset. The subtraction form avoids an overflow-prone offset + size test.

Do not reject every overlap or duplicate automatically. The classic container and engine semantics permit multiple names and a directory is an index, so diagnostics should distinguish a suspicious range from a format rule the tool has actually established. Zero-length entries may be meaningful. A strict validation mode can report unusual structures while a compatibility mode preserves them, but neither mode should read outside the file or allocate without limits.

Here is a compact Python metadata-only example. Its count cap is a local resource policy, not a WAD-format maximum. It deliberately reports names as raw bytes and never follows a lump into a large allocation:

from pathlib import Path
import struct

MAX_LUMPS = 1_000_000  # Local safety policy; tune for the inspection workload.

def inspect_wad(path: Path):
    size = path.stat().st_size
    with path.open("rb") as stream:
        header = stream.read(12)
        if len(header) != 12:
            raise ValueError("truncated WAD header")
        magic, count, directory = struct.unpack("<4sii", header)
        if magic not in (b"IWAD", b"PWAD"):
            raise ValueError("unrecognized classic WAD marker")
        if count < 0 or count > MAX_LUMPS:
            raise ValueError("lump count outside inspection policy")
        if directory < 0 or directory > size:
            raise ValueError("directory offset outside file")
        directory_size = count * 16
        if directory_size > size - directory:
            raise ValueError("directory extends beyond file")

        stream.seek(directory)
        for index in range(count):
            record = stream.read(16)
            if len(record) != 16:
                raise ValueError("truncated directory record")
            offset, length, raw_name = struct.unpack("<ii8s", record)
            if offset < 0 or length < 0 or offset > size or length > size - offset:
                raise ValueError(f"lump {index} range is outside the file")
            print(index, raw_name, offset, length)

inspect_wad(Path("authorized-test.wad"))

This sample is intentionally not a replacement for a source port’s compatibility rules. It recognizes only the classic IWAD/PWAD marker and fixed-size directory layout shown by the cited source. A forensic or preservation tool should keep the original file read-only, hash it before analysis, log the parser version and warnings, and write any report to a separate destination. Do not mount or launch an unknown archive merely to identify it.

Build useful diagnostics around the format

A good report keeps raw and interpreted values side by side. Show the archive’s cryptographic digest, file length, marker, declared count, directory interval, each entry’s index, raw name bytes, offset, size, and any anomaly. If decoding names for a user interface, escape non-printable bytes rather than dropping them. A collision report should show every duplicate and the effective winner under the selected engine’s load list, not just mark duplicates as errors.

Validation should be layered. The container pass checks bounds and resource caps. A resource-type pass checks payload-specific structures. An engine pass, performed only with an authorized compatible IWAD and known source-port version, checks how the target runtime resolves namespaces and patches. Keep these results separate: “directory range is in bounds” does not mean “texture is valid,” and “engine starts” does not mean the file is a correct preservation master.

For regression tests, create tiny synthetic archives in a temporary test directory. Cover truncated headers, negative signed values, a directory offset beyond end-of-file, a count above the parser’s configured one-million-entry cap, a directory cut off by one byte, a lump whose last byte lands exactly at end-of-file, a range one byte too long, an empty lump, a duplicate name, and a non-ASCII name. Python integers have arbitrary precision, so multiplication overflow is not a meaningful regression case for this Python example; fixed-width ports should test their own checked-arithmetic boundary separately. The expected behavior should be deterministic and should not depend on any commercial asset. Fuzzing a metadata parser with generated byte strings is also appropriate when the parser’s resource limits are finite.

Keep compatibility claims scoped

The central lesson is not that every WAD consumer uses one exact policy. It is that a stable container idea can coexist with loader-specific resolution behavior. Linux Doom 1.10 documents backward name search and later-file precedence; other ports should be checked in their own sources. A command-line list, autoload directory, DeHackEd patch stage, or namespace rule may affect the final resource set even when the raw WAD files have not changed.

When documenting a mod setup, preserve an ordered manifest with filenames, hashes, engine name and version, launch arguments, and any automatic patches. That turns “works on my machine” into a testable configuration. Never use a filename alone as identity, never silently reorder lump records during conversion, and never treat the original loader’s acceptance path as a safe validation boundary.

Related:

Sources:

Comments