Preserving x87 State Across DOS Libraries and Interrupt Boundaries

On a DOS machine, floating-point state is shared processor state. A library, game engine, numeric routine, or interrupt handler that changes the x87 unit can silently corrupt a caller that expected its control word, status, stack registers, or pending exceptions to remain intact. The bug may appear only when two components interleave, making it look like random arithmetic drift rather than a calling-convention violation.

The important distinction is between saving the x87 environment and saving the complete x87 state. Intel’s Software Developer’s Manual specifies that FSTENV/FNSTENV store the environment, including control and status information and instruction/data pointers, but not the x87 data-register stack. They also mask x87 exceptions as part of their defined operation. FSAVE/FNSAVE, by contrast, store the environment and register stack and then initialize the x87 unit. FRSTOR restores a previously saved full state. These are not interchangeable shortcuts.

Decide what your interface promises

A function that uses floating point must define whether it preserves the caller’s x87 state. Many legacy compiler conventions assume the x87 stack is empty at call boundaries and allow a routine to change control state; that is a compiler/runtime contract, not a universal hardware guarantee. Hand-written assembly, plug-ins, callbacks, and mixed-compiler libraries need an explicit rule. Document whether the routine may leave values on the x87 stack, alter rounding or exception masks, or leave exceptions pending.

If a routine must be transparent to an arbitrary caller, a full save/restore pair is the relevant concept. Reserve storage in a format and size appropriate to the execution mode and operand-size attributes, execute a full save before using the unit, then restore before returning. FSAVE and its no-wait form have different waiting behavior; the latter is not a general license to ignore pending exceptions. Confirm the precise instruction semantics and state image in the processor manual for the target CPU.

An environment-only save is appropriate when the contract concerns control/status configuration but the routine does not need to preserve the caller’s numeric register contents. It cannot restore values that were on the x87 stack. Using FNSTENV as if it were a full context save is a common corruption bug: after the routine returns, the caller’s stack values are still replaced by the routine’s calculations.

The save instructions have side effects

FSTENV stores the environment and then masks all x87 exceptions. Code that expects the prior exception-mask state must restore it or use a full-state strategy. FSAVE saves the full x87 state and initializes the unit; following it with new arithmetic begins from the initialized state, not from the caller’s previous stack. FRSTOR is needed to put that complete saved state back.

The F versus FN forms matter too. The waiting forms synchronize with prior x87 operations and may report an unmasked exception before completing the store. The no-wait forms omit that synchronization. A routine must choose based on its exception and synchronization contract, not by assuming the shorter mnemonic is always safer. Consult the Intel instruction reference for the exact exception behavior and operand image rather than depending on an assembler’s undocumented pseudoinstruction expansion.

Context preservation in callbacks and interrupts

The x87 state is not automatically saved on a real-mode interrupt. If a hardware interrupt handler uses the FPU, it can interrupt code between two floating-point instructions and replace the interrupted program’s register stack or status. Many DOS interrupt handlers correctly avoid x87 operations entirely. A periodic handler should set a flag or update integer-only state, then leave floating-point work to the foreground path.

The same rule applies to callbacks that can occur during a library call. If a callback invokes a second component that uses x87, define which layer owns preservation. Saving in every nested layer can be expensive, while saving nowhere can corrupt state. A documented boundary is essential: the caller saves the complete state around a foreign routine, or the callee promises to preserve it. Do not make an undocumented assumption from how a single compiler-generated function behaves.

For task switching, DOS extenders and multitaskers may manage floating-point context as part of their own API. A protected-mode application should use the extender’s documented floating-point or task-state contract; it should not assume that a real-mode TSR’s save area is valid in protected mode. Likewise, a DPMI host’s virtualization of hardware does not remove the requirement to obey its client ABI.

Reserve and validate the image correctly

The saved image is not a generic byte string. Its layout and size depend on instruction form, operand-size attributes, and whether the target uses legacy x87 state or an extended save mechanism. Allocate a correctly aligned, sufficiently large buffer according to the specific instruction definition and assembler mode. Do not hard-code a size copied from a 32-bit example into a 16-bit program without confirming the format.

An x87 save is also not a complete modern processor context. It does not, by itself, preserve general-purpose registers, segment registers, flags, or every SIMD/vector register file that later code might use. On processors and runtimes that use SSE state, the operating environment has a broader context-management contract. A DOS-era routine should state its actual scope as “x87 state” rather than “CPU state,” and should not assume that saving the x87 stack also snapshots an unrelated vector unit.

MMX adds another historical edge: its registers alias the x87 data-register storage on implementations that support MMX. Code that mixes MMX and x87 operations must follow the instruction-set transition rules, including the required state cleanup before subsequent x87 work. A routine that promises to preserve an arbitrary caller must consider every execution unit it uses, not just the mnemonic family used in its main loop. Verify the exact CPU manual when an implementation introduces MMX or SIMD instructions.

Keep each state image private to the invocation or task. Reusing one global buffer from nested code lets an inner call overwrite the outer caller’s snapshot. On error paths, make sure every path that has saved state reaches the matching restore. If a routine can terminate the process, long-jump, or transfer control through a callback, ordinary linear cleanup may not happen; design those exits explicitly.

A safe implementation pattern

The exact assembly depends on memory model and target CPU, but the contract can be reviewed as a sequence:

if transparent-call contract:
    allocate a private full x87 state image
    save complete caller state
perform bounded floating-point work
discard or account for the routine's own x87 results
restore the caller's complete state on every return path

For an environment-only contract, state clearly that register-stack values are not preserved. Never mix those two descriptions. Add assertions or debug instrumentation where practical: record stack depth at entry/exit, test non-default rounding modes, test masked and unmasked exception cases, and verify that values already on the x87 stack survive a call that claims transparency.

Compatibility and testing

The original 8087, later x87-compatible units, and emulators have instruction-level compatibility differences and timing behavior. A DOS program may also run under a DPMI extender or virtual machine. Test on each supported environment, especially if the code uses no-wait forms or relies on exception delivery. Do not infer that testing only integer code exercises the x87 context path.

Useful test cases include a caller with two live x87 stack values, a non-default rounding mode, a changed exception mask, and a pending exception where the test harness can safely observe it. Call the routine repeatedly and through a callback path. Inspect the saved image only with a decoder that matches its format; printing raw bytes as if offsets were universal is not a reliable diagnostic.

Operational rule

Treat the x87 as shared mutable context. Prefer integer-only work in interrupt handlers. At ordinary function boundaries, follow a clearly documented compiler or library convention. At foreign-code boundaries where transparency is required, use the complete save/restore mechanism supported by the target and guarantee cleanup on every path. The key is not to save more state indiscriminately; it is to save exactly the state your interface promises to preserve.

Related:

Sources: