The DOS InDOS Flag Is Not a Reentrancy License for TSRs

A Terminate-and-Stay-Resident (TSR) program can be activated by a timer, keyboard, or other interrupt while DOS is halfway through a file operation. If that handler immediately calls INT 21h to print text or open a log, it can re-enter DOS while kernel or driver state is already in use. The result may be corruption, a hang, or a failure that appears only with a particular device driver.

The InDOS flag is a useful diagnostic signal, not a general permission token. INT 21h, AH=34h returns a pointer to a byte that reflects DOS function processing, but historical DOS documentation warns that this internal interface is not guaranteed for future versions. A zero value alone does not prove the entire DOS stack is reentrant, and it does not make an arbitrary hardware interrupt a safe place to perform filesystem work.

Why DOS calls are not freely reentrant

Classic DOS uses mutable kernel structures and device-driver call paths that were designed around one active foreground program. File operations may update shared tables, traverse a filesystem driver, wait on a device, invoke a network redirector, or enter an error path. If an interrupt handler invokes another DOS function in the middle of that sequence, both calls can try to use the same state or stacks.

The dangerous case is not limited to the kernel. Installable drivers, redirectors, console handlers, and resident utilities may have their own non-reentrant paths. A call that is safe at the DOS API level can still trigger a driver that is unsafe to re-enter. “DOS is idle” therefore needs a narrow, version-aware meaning and a test plan for the drivers actually installed.

What AH=34h tells you, and what it does not

For DOS 2.0 and later, INT 21h, AH=34h returns ES:BX pointing to the InDOS flag. A TSR that uses this internal mechanism traditionally obtains the address during initialization and stores it, rather than repeatedly invoking DOS from an interrupt handler just to ask if DOS is busy. The Microsoft reference notes that the pointer’s validity is not guaranteed for future DOS versions.

The flag reports whether DOS function processing is active. It does not answer every question about current kernel state, pending device-driver work, critical-error handling, or whether a particular interrupt context may safely call DOS. Historical implementations also have a critical-error flag associated with the InDOS state, with layout varying across DOS versions and vendors. Do not hard-code an adjacent byte offset as a universal ABI.

That last point is especially important for FreeDOS and DOS-compatible kernels. Compatibility with an MS-DOS API does not mean every private kernel data layout is stable. If a resident program depends on undocumented internal state, tie the implementation to explicitly tested kernel versions and fail safely when its assumptions are not satisfied.

Defer work from hard interrupt handlers

The safest architecture is to do as little as possible in a hardware interrupt handler. Record a small request in resident memory, preserve required registers and flags, chain to the previous handler as required, and return. Let a context that is documented as safe perform the substantial work later.

For example, a timer hook that wants to log an event can set a byte-sized pending flag rather than opening a file in the timer handler:

; Interrupt-side pseudocode only. No DOS call is made here.
timer_hook:
    cmp     byte ptr [log_request_pending], 0
    jne     already_pending
    mov     byte ptr [log_request_pending], 1
already_pending:
    ; Restore saved state and chain/return according to the hook design.

This is a design sketch, not a complete interrupt handler. A real TSR must follow the processor interrupt calling convention, preserve the state required by the interrupted program, avoid unbounded work, and chain or acknowledge hardware interrupts correctly. The point is to separate “notice the event” from “call DOS to persist it.”

INT 28h is a special idle hook, not a blanket escape hatch

DOS invokes INT 28h while waiting in certain character-input loops, providing a conventional opportunity for resident software to perform limited background work. It is not a universal callback that runs whenever the machine has spare CPU time. Applications and drivers may not reach it, and its safety contract is narrower than “all DOS calls are now fine.”

Historical DOS programming references distinguish DOS functions that may be used while the idle interrupt is active from calls that share the active input stack. The permitted behavior varies by DOS version and call. Preserve and chain the previous INT 28h handler correctly, keep work bounded, and follow the target system’s documentation instead of copying an unqualified TSR sample from another DOS release.

Do not call INT 28h yourself as a way to trick the kernel into appearing idle. The interrupt is meaningful because DOS reaches it from its own input path. Artificially invoking it from a timer hook does not establish that DOS or its drivers are reentrant.

Critical errors complicate the simple flag test

When DOS handles a critical device error, a transient error handler may be active. Classic documentation and the DOS interrupt list describe cases where the InDOS byte alone is not sufficient to infer safety. The associated critical-error state has varied, and applications sometimes relied on undocumented relative offsets. That is historical implementation knowledge, not a portable extension point.

If an operation can produce a critical error, design the resident workflow so it does not try to perform a second DOS operation from inside the error path. Keep error handlers minimal, record a small status, and defer recovery or logging until execution returns to a documented safe context. Use INT 24h behavior and the Abort, Retry, Ignore, or Fail decisions according to the handler’s documented contract; do not recursively invoke the same failing disk path.

Separate installation-time setup from interrupt-time work

An installation routine runs as an ordinary program and can allocate resident memory, inspect the target DOS version, save prior interrupt vectors, and establish the TSR’s small state block. That is the right phase to perform setup calls such as obtaining the InDOS pointer when a compatible implementation requires it. It is not a reason to leave a fragile pointer unchecked forever; compatibility-sensitive code should document supported kernels and test them.

After installation, interrupt-time code should avoid file I/O, dynamic allocation, general-purpose DOS services, and lengthy display updates. A resident program that needs persistent logs can queue fixed-size records and flush them later from an explicitly safe path. Bound the queue and define what happens when it fills; silently overwriting memory is worse than dropping a diagnostic event with a counter.

Validate on the actual DOS stack

Test a TSR against the DOS version, memory manager, filesystem, device drivers, network redirector, and resident utilities that users will run. Exercise interrupts during disk activity, critical errors, shell input, and shutdown. Verify vector chaining and removal behavior. A DOSBox-like emulator is useful for development, but different kernels and drivers can expose distinct reentrancy behavior.

Use the InDOS flag as one piece of evidence only when the target explicitly supports that internal convention. Never infer that a nonzero flag can be ignored because a request is urgent, or that zero authorizes arbitrary INT 21h calls from a timer or keyboard handler. Defer work, use documented idle opportunities narrowly, and keep undocumented state behind a compatibility boundary.

Related:

Sources: