How CPU Emulation Works: Interpretation vs. Dynamic Recompilation

A game console’s CPU executes a specific instruction set — the exact binary encoding a 65816, a MIPS R4300, or a Zilog Z80 understands. Your PC’s CPU understands none of that natively. Making an original game binary run means translating between two completely different instruction sets, at some layer, without changing a single byte of the original program. There are two fundamentally different ways to do that translation, and the choice between them is the single biggest lever on an emulator’s speed.

Interpretation: the simple, honest approach

An interpreter does exactly what the phrase “fetch-decode-execute” describes, one guest instruction at a time: read the byte(s) at the guest program counter, decode which instruction it represents, execute a host-native function that reproduces its effect, advance the guest program counter, repeat.

for (;;) {
    uint8_t opcode = read_byte(pc++);
    switch (opcode) {
        case 0xA9: /* LDA #imm */ a = read_byte(pc++); set_nz_flags(a); break;
        case 0x8D: /* STA abs  */ write_byte(read_word(pc), a); pc += 2; break;
        /* ... one case per opcode ... */
    }
    cycles += cycle_table[opcode];
}

This structure is comparatively easy to inspect and can make instruction boundaries and cycle accounting explicit, which helps with cycle-accurate emulation. It is not automatically accurate: flags, undocumented opcodes, bus accesses, interrupt sampling, prefetch behavior, and instruction timing still have to match the target processor. The performance cost is that every guest instruction repeatedly pays fetch, decode, dispatch, and state-management overhead.

Dynamic recompilation: translate once, run many times

Dynamic recompilation (also called a JIT — just-in-time compiler) takes a different approach entirely: instead of interpreting each instruction every time it runs, it translates a whole block of guest instructions into equivalent host-native machine code once, caches that translated block, and jumps directly into it on every subsequent execution — no further decode overhead until the guest program counter reaches code that hasn’t been translated yet.

guest block: LDA #5 ; STA $2000 ; INC $2000 ; BNE loop
                    │
                    ▼ translate once
host code:   mov al, 5 ; mov [ram+0x2000], al ; inc byte [ram+0x2000] ; jnz loop_addr
                    │
                    ▼ cached — every future hit executes this directly

This is dramatically faster for anything that loops or calls the same code repeatedly — which is most of a real program’s execution time — because the translation cost is paid once, not on every pass. It’s also considerably harder to implement correctly: the translator has to handle guest self-modifying code (invalidating a cached block if the underlying guest memory changes), precise exception/interrupt boundaries (a host CPU might have already sped ahead of the point where a guest interrupt needs to land), and per-target-architecture code generation for every host platform the emulator supports.

Real recompilers rarely map each guest instruction to one obvious host instruction as the diagram suggests. They usually decode a translation block or basic block into an intermediate representation, apply safe optimizations, allocate host registers, emit native code, and record enough metadata to reconstruct guest state at exits and exceptions. QEMU’s Tiny Code Generator, for example, records CPU state assumptions with each translation block and maintains mappings needed to recover the guest program counter when a host exception occurs.

Memory accesses are another major boundary. An emulated MMU may require guest virtual-to-physical translation, permissions, device-mapped I/O, alignment behavior, and endianness conversion. A fast path can cache ordinary RAM translations, but accesses to hardware registers must preserve ordering and side effects. Treating every address like a host pointer would be fast and wrong.

Why many emulators use both

Some emulators offer a hybrid or multiple CPU cores: interpret cold or diagnostically sensitive code and recompile paths where translation cost will be amortized. Others translate a block on first execution because their architecture and workload make that policy simpler. This differs by emulator; it is not a requirement that every serious emulator use both. Debuggers and accuracy modes may deliberately select the interpreter even when a recompiler is available.

A middle ground: threaded interpretation

Between naive switch dispatch and native-code generation sit several interpreter designs. Threaded code dispatches directly or indirectly to the next opcode handler rather than returning through one central switch; a separate predecode stage may cache decoded operations or micro-operations. These techniques can reduce decode and branch-dispatch cost without emitting executable host code. Their portability depends on the implementation language and compiler, and the speed benefit must be measured rather than assumed.

Correctness boundaries a recompiler must preserve

A recompiler cannot optimize only for the final register values. It must expose interrupts at legal guest boundaries, raise precise exceptions, maintain delay-slot or condition-code semantics, and stop before time-sensitive device interactions. If guest code writes into a page containing translated instructions, the corresponding cached blocks must be invalidated or guarded. Page protection and write tracking can make invalidation efficient, but they introduce their own platform-specific edge cases.

Floating-point translation is especially subtle. Host and guest processors can differ in precision, rounding modes, NaN payloads, denormal handling, exception flags, and fused operations. A mathematically similar host sequence can still produce bits a game or operating system does not expect. Recompilers often need slower helper paths for unusual cases.

How emulator teams verify a CPU core

Unit tests exercise each opcode across boundary values, addressing modes, exceptions, and flags. Differential tests run the same instruction stream through an interpreter, a recompiler, and—where practical—real hardware, then compare architectural state and memory effects. Randomized instruction generation can find interactions that hand-written tests miss.

Performance validation needs separate measurements for translation time, code-cache lookup, generated-code quality, invalidation frequency, and time spent in device helpers. A benchmark that stays inside one hot arithmetic loop can flatter a JIT while saying little about a game that crosses MMU and I/O boundaries constantly. Accuracy and speed are properties of the whole emulated machine, not merely its opcode switch.

Why the choice matters more for some systems than others

A Game Boy’s 4MHz Z80-derived CPU is trivial to interpret at full speed on essentially any modern hardware — there’s no practical reason to build a JIT for it. A PlayStation 2’s dual-issue MIPS core running at nearly 300MHz, or a GameCube’s PowerPC at 486MHz, is a completely different story: interpretation alone often can’t keep up with real-time playback speed on host hardware that isn’t dramatically faster, which is exactly why sixth-generation-and-later console emulators (PCSX2, Dolphin) lean heavily on dynamic recompilation, while emulators for 8-bit and 16-bit systems frequently get away with straightforward interpretation and spend their engineering effort on accuracy instead of speed.

That comparison is a rule of thumb, not a law. A simple target can still benefit from a recompiler on constrained hardware, while a complex emulator may retain an interpreter for testing and fallback. Host architecture, compiler, timing model, peripherals, accuracy target, power budget, and maintenance cost determine the useful design.

Related:

Sources: