Skip to content
WindowsDeep Dive Published Updated 6 min readViews unavailable

Windows WaitOnAddress: State Predicates, Wake Races, and Shutdown Protocols

Use Windows address-based waits with atomic state publication, predicate loops, bounded deadlines, stable memory lifetimes, and explicit shutdown behavior.

WaitOnAddress lets a thread sleep while a small memory location still contains an undesired value. Another thread changes that value and calls WakeByAddressSingle or WakeByAddressAll. This can avoid polling loops and a separate event handle when the application already has a suitable state variable.

The wait is not the state protocol itself. Correctness comes from atomically published state, a predicate that the waiter rechecks, and memory that remains alive until every waiter has stopped using it. Windows supports the API starting with Windows 8 and Windows Server 2012. It is process-local; placing the state in shared memory does not turn it into a supported cross-process wake mechanism.

A notification is not an ownership transfer

The wait compares 1, 2, 4, or 8 bytes at the watched address with the value supplied by the caller. If the values already differ, it returns without needing a future wake. If they match, the thread can wait for notification or timeout. It is also permitted to wake early for reasons other than a useful state transition.

Therefore, a return from the wait means “inspect the state again,” not “consume a result.” A wake carries no payload, reserves no queue item, and does not count how many events happened. Two waiters may both observe a readiness state. If only one may consume the associated resource, they need a separate atomic claim or lock-protected ownership decision.

For a reusable queue, start by writing its invariant: which state protects each item, how empty differs from shutting down, and which operation makes an item visible. Only then choose how threads sleep. A conventional SRW-lock/condition-variable monitor is often clearer than a custom address protocol when several fields define the predicate.

Publish data before waking

The producer must change the state before calling the wake function. Waking first can let the consumer see the old value and return to waiting without another notification. Use an atomic operation with a defined memory-ordering contract for the state; volatile alone is not a complete synchronization strategy.

This deliberately narrow example publishes one immutable integer once. It uses Win32 interlocked operations rather than relying on the representation of a C++ std::atomic object as raw bytes. Every access to the shared readiness state uses that interlocked discipline, except the comparison performed internally by the Windows wait.

#define _WIN32_WINNT 0x0602
#include <windows.h>

struct OneShotResult {
    alignas(4) volatile LONG state = 0;
    int value = 0;
};

// Exactly one producer; no reset and no writes after publication.
void Publish(OneShotResult& result, int value) {
    result.value = value;
    InterlockedExchange(&result.state, 1);
    WakeByAddressAll(const_cast<LONG*>(&result.state));
}

bool Await(OneShotResult& result, int& value, DWORD& error) {
    const LONG waiting = 0;
    for (;;) {
        const LONG observed = InterlockedCompareExchange(&result.state, 0, 0);
        if (observed == 1) {
            value = result.value;
            error = ERROR_SUCCESS;
            return true;
        }

        LONG comparison = waiting;
        if (!WaitOnAddress(&result.state, &comparison,
                           sizeof(comparison), INFINITE)) {
            error = GetLastError();
            return false;
        }
    }
}

The sample requires the producer and consumers to keep result alive until all consumers have returned. Link a Windows SDK build against the synchronization library required by the documented wait API. There is no reset, cancellation, queue management, or multiple-producer arbitration here. Adding any of those changes the state machine and needs another correctness argument.

Ordinary interlocked operations provide atomicity and ordering under their Windows contract. The initial state load uses compare-exchange with an unchanged zero value, so it observes state atomically. The producer writes the payload before publishing readiness; consumers read that immutable payload only after observing readiness. A reusable payload cannot simply be overwritten while existing readers still access it.

Understand the check-to-sleep race

A consumer may observe zero, then the producer publishes one and wakes before the consumer reaches WaitOnAddress. The comparison value is still zero, so the wait observes that the address has changed and returns. This is why the watched state is essential; a standalone notification without retained state would not communicate that the work is ready.

Another case is a wake with no state change. The consumer must loop because the original predicate is still false. Conversely, if state changes from zero to one and back to zero before the consumer checks again, a binary flag cannot describe the intermediate event. Use a monotonic generation or queue state when the application must distinguish repeated transitions. Address waiting does not solve that ABA-style ambiguity automatically.

Choose one or all according to the predicate

WakeByAddressSingle wakes one waiting thread, while WakeByAddressAll wakes all waiters at that address. The sample uses all because every reader is allowed to observe the same immutable result. For a queue with one new item, waking one may reduce contention, but the consumer still must claim the item safely.

Shutdown usually changes a predicate observed by every worker, so an all-waiter notification is commonly appropriate. The wake cannot force workers to stop while they are doing unrelated work. They must check the shutdown state at defined points, and the owner must join them before destroying watched memory.

Do not recycle an allocation at the same address while old waiters remain. The address identifies the wait target; allocation reuse can make an old waiter observe unrelated state. A destructor that merely stores “stopping” and wakes threads is incomplete unless it also waits for those threads to relinquish access.

Bound total time rather than every individual wait

With a finite timeout, WaitOnAddress returns false and GetLastError reports ERROR_TIMEOUT when the wait expires. Capture the error immediately. Repeating a full timeout after every early wake can make a nominal five-second operation last much longer.

For an overall deadline, calculate remaining time from a monotonic clock before each wait and stop when the deadline expires. Decide what timeout means for the caller: abandoning the consumer does not cancel the producer automatically. Preserve the result owner until the producer is also finished, or use another ownership model that keeps the shared object alive.

Validate transitions, not just throughput

Test publication before any consumer starts, publication between the consumer’s check and wait, many consumers, a wake without state change, timeout, and shutdown during a wait. Add stress runs under different CPU counts and architectures supported by the application. Timing delays alone rarely prove race correctness; instrument state transitions and inject synchronization barriers at the difficult windows.

If a thread hangs, inspect the watched address, current state, comparison value, producer completion, and last wake site. A state that changed without a wake points to a producer protocol defect; a state that never changed may indicate a failed producer or missing shutdown path. A corrupted or freed address is a lifetime bug, not a timeout-tuning problem.

Address waits are useful when a small, durable predicate already describes readiness. Keep the state model explicit, publish it atomically, recheck after every wake, and finish all access before releasing the storage.

Related:

Sources:

Comments