x87 Exception Reporting in DOS: Masks, Sticky Flags, and Deferred Faults

x87 floating-point failures can appear far from the instruction that caused them. The FPU records exception conditions in its status word, and mask bits in the control word determine whether an exception is handled in a masked form or reported to software. On classic DOS hardware, the reporting path also depends on processor generation, board wiring, BIOS, runtime, and emulator behavior. A diagnostic that prints only a numeric result can therefore miss the state that explains a later trap.

This article focuses on observing and reasoning about x87 exception state. It complements full x87 context preservation, but it does not describe every processor-specific exception route. Use the Intel Software Developer’s Manual for the exact target CPU and verify any handler assumptions with the compiler runtime and DOS extender in use.

Six exception classes and two words

The x87 status word has sticky flag bits for invalid operation, denormal operand, divide by zero, overflow, underflow, and precision/inexact result. The control word contains corresponding mask bits. When an exception condition is detected, the processor records its flag; the control-word mask determines the defined masked response or whether the condition is unmasked and eligible for reporting. Multiple conditions can be recorded during a computation, so a status word may contain more than one flag.

“Sticky” means a flag remains set until software clears it or resets the FPU state. A later calculation can therefore appear to be the source even though an earlier operation raised the condition. At a well-defined diagnostic boundary, capture the status and control words before clearing anything. The instruction pointers and other environment fields may also help identify the operation on processors and save formats that report them.

Do not interpret the status word as a direct boolean “the last operation failed.” A precision flag can be expected for ordinary rounded arithmetic; an invalid operation or divide-by-zero may be masked and produce a special value; and an unmasked exception may remain pending until the execution environment reports it. The application must define which classes are expected, which are fatal, and what result policy applies.

Masks are a policy, not a fix

An exception mask allows the processor to continue with its documented masked response. Changing the control word to mask all exceptions can make legacy software continue, but it does not make an invalid calculation correct. Likewise, unmasking every exception globally can make a runtime or library fail in code that was written assuming the default masked environment.

If a component changes the control word, it should define whether that state is private to the component or shared with its caller. Save the original control word, install only the intended mask changes, and restore the original on every normal exit. When multiple libraries or callbacks can use the FPU, make the ownership rule explicit. The full register-stack and environment save concerns are covered separately; a control-word snapshot alone does not preserve numeric register contents.

Be careful when loading a control word that unmasks an exception whose flag is already set. A subsequent synchronizing FPU instruction can surface the pending condition. Capture and understand the prior status before changing masks. Do not clear all flags blindly just to suppress a fault, because that destroys evidence and can conceal an earlier arithmetic defect.

Deferred reporting and synchronization

x87 arithmetic is historically asynchronous relative to the integer instruction stream. An unmasked exception can be recorded by the FPU and become visible to software at a later synchronization point, such as a waiting instruction. The precise interaction among FWAIT, waiting and no-wait instruction forms, and CPU exception delivery is architecture- and configuration-dependent. On later processors, the numeric-error configuration affects whether reporting follows an internal exception path; older PC/AT designs used an external coprocessor error route.

The practical consequence for a DOS debugger is simple: mark the program’s synchronization points and inspect state close to each floating-point region. A failure at a later store or wait instruction may reflect an earlier arithmetic operation. Conversely, an instruction that does not wait may allow the integer program to continue until another instruction synchronizes with the FPU.

Do not install a hardware or software exception handler by copying a 386-or-later example into an 8087/80287/80387 environment. The vector, processor control bits, board logic, DOS runtime, and extender may change the reporting path. A DPMI host may own protected-mode exception vectors while real-mode DOS uses a separate interface. Follow the environment’s documented contract.

A diagnostic snapshot pattern

The following assembly illustrates the observation goal on a target where the instructions and execution mode are supported:

; Store raw x87 status and control words for later reporting.
fnstsw  [saved_status]
fnstcw  [saved_control]
; Decode flags and masks after leaving the sensitive numeric path.

These no-wait stores do not themselves constitute a full context save or prove that no exception is pending. Choose waiting versus no-wait forms according to the target’s synchronization requirement, and consult the instruction reference for the exact behavior. If the diagnostic needs to attribute a fault to the preceding arithmetic, deliberately place a documented synchronization point after capturing the relevant operands and intermediate result, in a controlled test build.

The diagnostic record should include CPU or emulator, FPU presence or emulation mode, control word, status word, operation sequence, operands, result, rounding mode, exception masks, and whether a wait occurred. Preserve the raw words in hexadecimal as well as decoded labels. Do not decode a saved environment image using offsets from a different operand-size or processor format.

Common failure patterns

An exception reported far from the source often indicates missing synchronization in the test or an earlier unmasked condition. A handler that immediately executes more floating-point instructions can recursively trigger the same pending exception. A library that changes masks without restoration can cause later code to fail. An error that appears only with a different optimization level may result from a different instruction sequence or synchronization point, not necessarily a different mathematical input.

If a program receives an unexpected infinity, NaN-like value, denormal behavior, or inaccurate result, record the masks and sticky flags before changing them. Check whether the compiler or runtime established a non-default control word. Reproduce with a minimal arithmetic sequence and known operands, then add calls, callbacks, and interrupt-adjacent activity. Keep hardware interrupt handlers integer-only unless the execution environment explicitly defines FPU use and context preservation.

Testing on DOS and emulators

Build tests for one masked and one intentionally unmasked exception in an isolated program. Use known operands for invalid operation and division by zero; add overflow and underflow cases only when the chosen precision and rounding mode make the expected result clear. Verify the status flags before and after clearing them using the documented instruction sequence. Keep a process-level timeout because a bad handler can hang the guest.

Repeat across each supported CPU or emulator and identify the runtime, extender, and FPU model. A successful run under one emulator does not prove external IRQ13 behavior on physical AT hardware. This authoring host does not provide a DOS x87 environment, so the instruction excerpt is source-checked but not executed here. Treat real hardware exception-vector behavior as unverified unless tested on the declared target.

Acceptance checklist

Before declaring an x87 failure understood, capture the control and status words before clearing them; identify the first arithmetic instruction that sets a flag; document waiting/no-wait boundaries; check the compiler’s control-word assumptions; define handler reentrancy; and verify that save/restore boundaries preserve the state contract. If only the final result is captured, the diagnosis is incomplete.

The useful distinction is between recording an exception condition and receiving an exception notification. The status/control words describe x87 arithmetic state; CPU and operating-environment configuration determine how software learns about unmasked conditions. Keep both layers visible, preserve evidence, and avoid treating global mask changes as a correctness fix.

Related:

Sources: