Skip to content
WindowsDeep Dive Published Updated 7 min readViews unavailable

CancelIoEx: Correct Cancellation and Completion for Windows Overlapped I/O

Use CancelIoEx without freeing buffers too early: target one OVERLAPPED request, await its terminal result, and handle completion races explicitly.

CancelIoEx requests cancellation of outstanding I/O for a file handle. It is not a synchronous “stop now” operation, and it does not free the request’s buffer or OVERLAPPED structure. A request can complete normally before the cancellation reaches it, complete with ERROR_OPERATION_ABORTED, or fail for another reason. Correct code keeps the request state alive until its ordinary completion mechanism reports one of those terminal outcomes.

The most important difference from CancelIo is thread scope. CancelIo affects outstanding requests issued by the calling thread; CancelIoEx can target requests from other threads in the same process. Pass a specific OVERLAPPED pointer to cancel one operation, or pass NULL to attempt cancellation of all outstanding I/O for the handle. Broad cancellation is easy to race with producers that can submit more I/O, so a per-operation target plus explicit application state is usually easier to reason about.

Model the request, not just the handle

An overlapped request has at least four pieces of lifetime-sensitive state: the file handle, the OVERLAPPED object, the buffer, and the completion consumer. The kernel and I/O subsystem may still reference the request after CancelIoEx returns. A timeout or user pressing Cancel changes the desired outcome; it does not prove that the request has stopped using memory.

Represent each operation with a stable object whose lifetime extends from submission through completion. Keep a state such as Created, Pending, CancelRequested, and Completed, but let the completion path determine the final result. Protect state transitions with an atomic or lock. Do not recycle a pool slot or reuse an OVERLAPPED address while a prior request at that address might still complete; otherwise a late completion can be mistaken for a newer operation.

For a single operation, targeted cancellation avoids affecting unrelated reads or writes on a shared handle:

#include <windows.h>
#include <array>
#include <atomic>
#include <cstddef>

void ReportCancellationFailure(DWORD error); // Provide an application-specific logging implementation.

struct PendingRead {
    OVERLAPPED overlapped{};
    std::array<std::byte, 4096> buffer{};
    HANDLE file = INVALID_HANDLE_VALUE;
    std::atomic<bool> cancelRequested{false};
};

bool RequestReadCancellation(PendingRead& request)
{
    request.cancelRequested.store(true, std::memory_order_release);
    if (CancelIoEx(request.file, &request.overlapped)) {
        return true; // Request marked; completion still must be observed.
    }

    const DWORD error = GetLastError();
    if (error == ERROR_NOT_FOUND) {
        // It may already have completed or no longer be pending.
        // The normal completion path still owns final cleanup.
        return false;
    }

    ReportCancellationFailure(error);
    return false;
}

The fragment assumes that the caller has already submitted the request with overlapped I/O and that the PendingRead object remains alive until completion. ERROR_NOT_FOUND is not a reason to free the request; it can mean there was nothing left to cancel because completion won the race. The completion path remains authoritative.

Submission and cancellation race

Suppose a producer starts an I/O request while another thread calls CancelIoEx(handle, nullptr). Windows processes outstanding requests for the handle, but producer threads can start new requests while the broad cancellation operation is walking threads. Microsoft documents this race. If the application needs a bounded set of canceled requests, stop new submissions under an application-level state lock, collect the exact request objects, issue targeted cancellation, and then drain completions.

The order should be designed as a protocol:

  1. Mark the component as stopping and reject new submissions.
  2. Snapshot or otherwise identify each in-flight request owned by that component.
  3. Call CancelIoEx for each specific OVERLAPPED request where appropriate.
  4. Continue processing the configured completion mechanism until every request reaches a terminal result.
  5. Release buffers, OVERLAPPED storage, request contexts, and finally the handle according to their ownership rules.

If a completion port or thread-pool I/O object is in use, its completion notification remains part of the lifecycle. A cancellation request does not mean that the completion port has already received and dequeued the completion. A worker must avoid using the request context after it has transferred ownership to a cleanup path. Exactly one path should own final reclamation.

Completion can win

Cancellation is best understood as “please stop if this request is still cancelable,” not as a result code. The operation can win the race and complete normally even if the caller successfully asked for cancellation. Some drivers or operations may not support cancellation at the current stage. The final status must therefore be read from the same mechanism used for normal completion, such as GetOverlappedResult, an I/O completion port packet, or a registered callback.

For an event-based overlapped request, wait for its completion signal and inspect the result. For an I/O completion port, drain the queued packet and inspect the transferred byte count and error status. If the operation completed with ERROR_OPERATION_ABORTED, it was canceled; a different error is a distinct failure. A successful byte transfer after a cancel request is also possible and should be handled according to the application’s consistency policy.

Do not call GetOverlappedResult(..., TRUE) from a thread that must remain responsive without understanding that it can block until the underlying operation completes or cancellation is acknowledged. A device or driver problem may make this wait much longer than the user-facing timeout. If bounded shutdown is mandatory, the system design needs a recoverable policy for a non-completing operation, not an assumption that CancelIoEx enforces the timeout.

Handle and structure ownership

The file handle must refer to the handle used for the operation. Keep it open until the completion and cancellation policy is complete; handle reuse can make diagnostics confusing and complicate shared ownership. If multiple components use one handle, define which component may cancel which request. A handle-wide cancellation can disrupt unrelated consumers.

Do not reinitialize an OVERLAPPED object while its operation is pending. Do not reuse its event handle or buffer for another request. For targeted cancellation, keep the exact structure address alive until the operation has completed, whether normally or by cancellation. Closing a handle is not a substitute for a completion protocol: it can affect all users of the handle and does not give application code permission to free memory the I/O system still references.

The I/O completion path should also be idempotent with respect to shutdown. For example, a UI may display “cancel requested,” but it should not destroy the request view-model until the worker reports final completion. A service can stop accepting requests, ask in-flight operations to cancel, and still emit one terminal result per accepted request. These rules keep user-visible state aligned with the actual completion state.

Synchronous I/O is a separate case

CancelIoEx is for outstanding I/O requests represented by the file-handle operation model. Windows also provides CancelSynchronousIo for pending synchronous I/O performed by a particular thread. That API has its own race: if a shared thread finishes one synchronous operation and starts another before the cancellation call is processed, the wrong call can be affected. Thread-pool threads make this especially subtle because unrelated work can reuse a worker.

Do not mix synchronous and asynchronous cancellation assumptions. Prefer overlapped I/O when the application needs explicit per-request ownership and cancellation, but verify that the device and filesystem support the requested mode. Use higher-level APIs with progress/cancel callbacks when the operation is implemented as a compound action, such as file copy, rather than assuming a single generic I/O cancellation API covers it.

Test the terminal states

Build tests where cancellation is requested before the device starts, during active transfer, just as completion arrives, and after completion has already been dequeued. Verify normal success, ERROR_OPERATION_ABORTED, other error results, ERROR_NOT_FOUND, and cancellation API failure. Repeatedly reuse the request pool under stress and use sanitizers or Application Verifier where available to expose use-after-free and double completion cleanup.

Instrument request identifiers, submission time, cancel-request time, final status, bytes transferred, and completion-consumer identity. Never log buffer contents merely to diagnose a cancellation race. A useful operational trace shows whether a request was still pending when cancellation ran and how long it took to reach final completion. With those states explicit, CancelIoEx becomes a controlled lifecycle signal rather than a dangerous attempt to pretend asynchronous work has vanished.

Related:

Sources:

Comments