FreeDOS INT 21h/AH=5Ch: Locking Byte Ranges on Open Files

INT 21h function AH=5Ch asks DOS to lock or unlock a byte range in a file that is already open. The call is small, but its behavior depends on more than the register setup: a local FreeDOS file goes through the resident SHARE component, while a file on a redirected drive is handed to that redirector. A program must therefore check both the DOS result and the behavior of the storage path it actually uses.

This is the register-level contract, not a general guide to configuring SHARE or a claim that every DOS clone and network server implements identical locking. It is distinct from the higher-level file-sharing modes selected when opening a file: open sharing governs access to the file as a whole; AH=5Ch requests a narrower interval.

Register contract

The Microsoft MS-DOS reference documents the function for DOS 3.0 and later. Its inputs are:

Register	Meaning
`AH`	`5Ch`
`AL`	`00h` to lock; `01h` to unlock
`BX`	Handle of the already-open file
`CX:DX`	32-bit byte offset from the beginning of the file; `CX` is high word and `DX` low word
`SI:DI`	32-bit byte length; `SI` is high word and `DI` low word

On success, carry is clear. On failure, carry is set and AX contains the DOS error. The documented common codes are 01h for an invalid function or missing sharing service, 06h for an invalid handle, 21h for a lock violation, and 24h for an exhausted sharing buffer. Network paths can return redirector-specific outcomes, so do not assume every error will be one of those four.

The word ordering is easy to reverse accidentally. For example, offset 00008000h and length 00001000h mean:

CX = 0000h    DX = 8000h
SI = 0000h    DI = 1000h

The call does not take a pointer to an offset or a record number. It takes two 32-bit values, each split high-word first across the named register pair. These are byte units, even if an application organizes its file as fixed-size records.

Lock, work, then unlock the exact same interval

Here is an 8086-style example for a file handle already returned by a successful DOS open or create call. It locks bytes 8000h through 8FFFh, a length of 1000h bytes, performs the application’s record work, and then issues the matching unlock.

; file_handle is a word containing the open DOS handle.
; Lock offset 00008000h, length 00001000h.
        mov     ax, 5C00h       ; AH=5Ch, AL=00h: lock
        mov     bx, file_handle
        xor     cx, cx          ; high word of offset
        mov     dx, 8000h       ; low word of offset
        xor     si, si          ; high word of length
        mov     di, 1000h       ; low word of length
        int     21h
        jc      lock_failed    ; inspect AX before another DOS call

; Read/update/write the record using the application's file protocol.
; Every success and error path after the lock must reach the unlock.

        mov     ax, 5C01h       ; AH=5Ch, AL=01h: unlock
        mov     bx, file_handle
        xor     cx, cx
        mov     dx, 8000h
        xor     si, si
        mov     di, 1000h
        int     21h
        jc      unlock_failed

The example keeps the lock parameters literal so the matching requirement is visible. Real code should save the chosen offset and length in its own state and use those exact values for the unlock; do not reconstruct them from a possibly changed current file position. Test carry immediately after INT 21h, before calling another DOS service or altering flags. On a failed call, preserve AX for the application’s error path.

The DOS reference says that an unlock must match the previously locked offset and length. An unlock for only part of a locked interval, or for an adjacent range, is not equivalent. A lock that extends beyond the current end of file is permitted and does not itself enlarge the file. Later writes, not the lock call, change file contents or length.

Process and handle lifetime matters

Acquire the lock only after opening the file successfully. Keep the critical section short: lock the agreed record range, read and validate the current record, make the bounded update, write it, and release the exact range. A lock is not a transaction log, does not make a multi-write update crash-atomic, and cannot restore a partly written database after power loss.

The MS-DOS documentation warns that closing a file with active locks, or terminating while such a file remains open, leaves the file in an undefined state. Every exit path, including error, Control-C, and critical-error handling, needs a plan to unlock before closing or terminating. If recovery cannot safely continue after an interrupted write, use the database’s journal or recovery protocol rather than treating a released lock as proof that the data is sound.

Lock ownership also is not interchangeable with handle identity. The reference describes duplicate handles in the same process differently from handles inherited by a child through EXEC: duplicates in the current process can access that process’s locked range, while a child with an inherited handle does not gain that same bypass. Design and test around the process model documented for the target DOS; a copied handle value is not evidence that the lock has been released.

The FreeDOS kernel 2046 source provides a concrete implementation boundary. Its INT 21h dispatcher assembles CX:DX and SI:DI into two long integers and treats nonzero AL as an unlock request. The kernel then validates BX against the current process’s handle table. For a shared/remote SFT it calls the remote lock routine; for a local file it returns invalid-function when SHARE is not installed, checks that the SFT has a sharing entry, and passes the current PSP and sharing-table index to SHARE.

That is why a working local test does not establish remote behavior. A redirector, server, cache, and filesystem must agree on what a lock means and how a conflict is reported. DOS byte-range locks protect accesses made through participating DOS/redirector services; they cannot stop an application that writes disk sectors directly or a separate tool that bypasses the file API. The FreeDOS SHARE project describes itself as an implementation for the FreeDOS kernel; do not infer drop-in compatibility with other DOS kernels from that project scope.

Test conflicts on the actual path

Use a disposable file and two independent processes, first on a local FreeDOS volume and then, if relevant, through the production redirector and server:

Have process A open the file using the application’s real access and share mode, then lock one known byte interval.
Have process B request a conflicting lock and confirm it receives a failure instead of proceeding as if it owns the record.
Test a non-overlapping range separately; do not infer interval behavior from one conflict case.
Have A unlock with the exact original offset and length, then confirm B can acquire the intended range.
Repeat through the actual network/cache configuration and record the exact DOS or redirector error.

Do not run this experiment against the only copy of valuable data. Include abnormal termination only inside a disposable VM or emulator snapshot, and verify both the lock outcome and the application’s own data-recovery behavior. A reset that clears volatile state can still leave a partially updated file.

Troubleshoot by stage

If AX=01h is returned on a local FreeDOS file, verify that compatible SHARE is loaded; the kernel source uses that error when the local sharing service is absent. AX=06h points first to handle lifetime, not offset arithmetic. AX=21h means the requested range could not be locked or unlocked as requested; confirm that the other process has released its conflicting lock and that the unlock interval is exact. AX=24h indicates a sharing-buffer limit, not a reason to spin indefinitely.

If a lock succeeds but a second client still appears to read or write the region, capture the actual path, handles, process identities, and operation sequence. Check whether both clients use the same redirector and server semantics, whether a cache or direct-I/O layer is involved, and whether the second access is really from another process. Do not silently retry every failure as though it were transient: retry only the documented conflict case under a bounded policy and expose other errors to the caller.

AH=5Ch is useful only when the full stack honors the protocol. Correct register pairs, exact-range unlocks, explicit error handling, and a two-process test provide evidence; merely seeing a successful interrupt return does not prove that every machine or redirector will coordinate the same file.

Related:

Sources:

Register contract

Lock, work, then unlock the exact same interval

Process and handle lifetime matters

What FreeDOS routes to SHARE or a redirector

Test conflicts on the actual path

Troubleshoot by stage

Comments