How CPU Emulation Works: Interpretation vs. Dynamic Recompilation

A game console’s CPU executes a specific instruction set — the exact binary encoding a 65816, a MIPS R4300, or a Zilog Z80 understands. Your PC’s CPU understands none of that natively. Making an original game binary run means translating between two completely different instruction sets, at some layer, without changing a single byte of the original program. There are two fundamentally different ways to do that translation, and the choice between them is the single biggest lever on an emulator’s speed.

Interpretation: the simple, honest approach

An interpreter does exactly what the phrase “fetch-decode-execute” describes, one guest instruction at a time: read the byte(s) at the guest program counter, decode which instruction it represents, execute a host-native function that reproduces its effect, advance the guest program counter, repeat.

for (;;) {
    uint8_t opcode = read_byte(pc++);
    switch (opcode) {
        case 0xA9: /* LDA #imm */ a = read_byte(pc++); set_nz_flags(a); break;
        case 0x8D: /* STA abs  */ write_byte(read_word(pc), a); pc += 2; break;
        /* ... one case per opcode ... */
    }
    cycles += cycle_table[opcode];
}

This is easy to write, easy to debug, and trivially accurate — each guest instruction genuinely executes as one discrete, inspectable step, which makes it straightforward to track exact cycle counts (more on why that matters in cycle-accurate emulation). The cost is speed: every single guest instruction pays the full overhead of a decode-and-dispatch, even a trivial one like “add register to register” that a native CPU would execute in a single cycle.

Dynamic recompilation: translate once, run many times

Dynamic recompilation (also called a JIT — just-in-time compiler) takes a different approach entirely: instead of interpreting each instruction every time it runs, it translates a whole block of guest instructions into equivalent host-native machine code once, caches that translated block, and jumps directly into it on every subsequent execution — no further decode overhead until the guest program counter reaches code that hasn’t been translated yet.

guest block: LDA #5 ; STA $2000 ; INC $2000 ; BNE loop
                    │
                    ▼ translate once
host code:   mov al, 5 ; mov [ram+0x2000], al ; inc byte [ram+0x2000] ; jnz loop_addr
                    │
                    ▼ cached — every future hit executes this directly

This is dramatically faster for anything that loops or calls the same code repeatedly — which is most of a real program’s execution time — because the translation cost is paid once, not on every pass. It’s also considerably harder to implement correctly: the translator has to handle guest self-modifying code (invalidating a cached block if the underlying guest memory changes), precise exception/interrupt boundaries (a host CPU might have already sped ahead of the point where a guest interrupt needs to land), and per-target-architecture code generation for every host platform the emulator supports.

Why many emulators use both

A common pattern is a hybrid: interpret cold code (run once or a handful of times) and only pay the recompilation cost for code that’s actually hot — loops, frequently-called subroutines, the inner workings of a game’s main loop. This mirrors how modern language JITs (V8, the JVM’s HotSpot) work for exactly the same reason: recompilation only pays for itself once code runs often enough to amortize the translation cost.

A middle ground: threaded interpretation

Between naive switch-dispatch interpretation and full recompilation sits threaded interpretation — instead of a giant switch statement re-decoding an opcode’s type on every iteration, each instruction is pre-decoded once into a small sequence of function pointers (“threaded” together), and the interpreter loop just calls through the chain. It’s still fundamentally interpretation (no native code generation, no code cache to invalidate), but it removes a meaningful chunk of dispatch overhead compared to a raw switch statement, at much lower implementation complexity than a full JIT.

Why the choice matters more for some systems than others

A Game Boy’s 4MHz Z80-derived CPU is trivial to interpret at full speed on essentially any modern hardware — there’s no practical reason to build a JIT for it. A PlayStation 2’s dual-issue MIPS core running at nearly 300MHz, or a GameCube’s PowerPC at 486MHz, is a completely different story: interpretation alone often can’t keep up with real-time playback speed on host hardware that isn’t dramatically faster, which is exactly why sixth-generation-and-later console emulators (PCSX2, Dolphin) lean heavily on dynamic recompilation, while emulators for 8-bit and 16-bit systems frequently get away with straightforward interpretation and spend their engineering effort on accuracy instead of speed.