NSCache on macOS: Eviction Semantics, Cost Accounting, and Miss Recovery

NSCache is a temporary store for values that an application can recreate. Its main value is that entries are eligible for automatic eviction when memory pressure makes them expensive to retain, and its operations can be used from multiple threads without adding an external lock around the cache itself. Those properties do not make it a durable database, a strict memory quota, an LRU contract, or a substitute for coordinating the expensive work that creates a value.

Use it for decoded thumbnails, derived render data, or other recomputable objects. Do not make a cache hit the only record that a user owns something, that a file was saved, or that a background task completed. If eviction changes correctness rather than performance, the data belongs in a persistent store and needs a different design.

Define a cache key as a complete computation identity

A key must include every input that can change the computed value. For an image thumbnail, that may include the source identity and revision, target pixel dimensions, display scale, crop mode, color treatment, and decoder options. If one of those values is omitted, the cache can return a technically valid object for the wrong request. Avoid mutable keys: NSCache does not copy key objects, so a key whose hash changes while stored can become impossible to look up reliably.

import Foundation

final class ThumbnailCache {
    private let storage = NSCache<NSString, NSData>()

    init() {
        storage.countLimit = 400
        storage.totalCostLimit = 96 * 1024 * 1024
    }

    func value(for key: String) -> Data? {
        storage.object(forKey: key as NSString) as Data?
    }

    func insert(_ data: Data, for key: String) {
        storage.setObject(data as NSData, forKey: key as NSString, cost: data.count)
    }
}

This example uses byte count as an estimated cost because the stored payload is Data. For an object graph, the real retained memory may differ from the encoded byte count. Treat cost as a relative accounting signal, not a measurement of exact process footprint. Pick namespaced keys or a structured key encoder so unrelated features cannot collide.

Limits are pressure hints, not guarantees

countLimit expresses the maximum number of objects the cache should hold, and totalCostLimit describes the total cost before eviction begins. Apple explicitly documents these as non-strict. An object may be evicted immediately, later, or possibly never depending on implementation details, and the order of eviction is not guaranteed. Do not write a test that assumes the least-recently-used entry is always the one removed.

Choose cost units consistently. If one component uses bytes, another uses pixel count, and a third uses a constant cost of one, the total-cost threshold has no coherent interpretation. Cost can help compare approximate memory pressure, but it does not cap actual resident memory. The operating system, object overhead, autorelease behavior, intermediate decode buffers, and simultaneous in-flight computations all affect real memory usage.

Also bound active work. A cache limit does not prevent ten callers from concurrently decoding the same 80-megabyte image after a miss. Use an in-flight request table, task coalescing by key, and cancellation ownership to prevent duplicate work. A cache is only one layer in a memory plan that includes source size, decoded output, temporary buffers, and consumer retention.

Cache misses are normal control flow

Every lookup must have a correct miss path. Read the authoritative source, compute the value, validate it, then insert the result if it remains useful. If the source changed during computation, avoid publishing under a key for the old revision. A cache entry is valid only for the inputs captured by its key and computation generation.

Do not expose NSCache’s delegate eviction callback as the only place that releases business state. Entries can be removed because of memory pressure or explicit application policy, and callbacks should not be used to infer that the data was persisted elsewhere. Use cache misses to reload or recompute. If recomputation can fail, model failure separately from a miss so a corrupt source does not trigger an endless expensive retry loop.

NSCache operations themselves are thread-safe, but the cache’s associated loading logic may not be. Two concurrent calls can both observe a miss and duplicate decoding. That may be harmless for small pure computations, but it can saturate CPU or memory for large work. A keyed task registry can make callers await one shared decode; remove its entry after completion and ensure cancellation by one waiter does not cancel work still needed by others.

Choose object ownership carefully

The cache retains inserted values until removal or eviction, while callers can retain returned objects independently. Eviction does not reclaim an object that a view model or renderer still strongly references. If the goal is to lower peak memory, track consumer lifetimes and release old references as well as trimming the cache. A cache budget should be measured against the application’s actual object graph, not just totalCostLimit.

For discardable object contents, NSDiscardableContent can let an object release subcomponents when content is no longer in use. Apple documents that cached discardable-content objects are removed when their content is discarded by default, though the policy can be changed. This is an advanced protocol: callers must obey its begin/end access discipline, and code must tolerate discarded content. Do not adopt it casually for immutable values that are simpler to rebuild as a whole.

Prefer immutable cache values after insertion. Mutating a shared cached image or render result from multiple consumers makes the cache’s thread-safety guarantee irrelevant to the object’s own invariants. If an object must be mutable, synchronize its mutation separately or store immutable snapshots keyed by version.

Invalidation needs an explicit dependency map

Eviction is opportunistic; invalidation is an application action when inputs change. Track which source revision and transformation options went into a cached value. On a document edit, account switch, style change, or data deletion, remove entries by a bounded key prefix or advance a revision token used by new keys. A global removeAllObjects() is simple but can create a cache stampede if many views immediately request the same data again.

When invalidating a user-specific value, make sure all relevant stores or processes are covered. In a multi-window app, one window may retain a value after another window changes the source. Notify or version the shared model so consumers stop displaying stale derived data. NSCache does not define cross-process synchronization or persistent invalidation semantics.

Never cache secrets or sensitive content merely because the API is convenient. NSCache is an in-memory object cache, not an encryption or access-control layer. Account transitions should clear user-scoped objects and prevent an old request from inserting stale values after sign-out. Tag in-flight work with an account generation and reject completion when the current identity no longer matches.

Measure hit rate and memory behavior

Record lookups, hits, misses, computation duration, output size estimate, coalesced callers, and cancellation. Avoid logging raw keys if they contain paths, user identifiers, or private document names. A high hit rate is not always good if entries are stale; correctness requires the key and invalidation model to be right first. A low hit rate can mean limits are too small, keys vary unnecessarily, or consumers are not sharing a cache instance.

Test under memory pressure, rapid source revision changes, concurrent identical requests, account switch during decode, cache clearing, large values, and repeated miss/recompute cycles. Observe process memory and decode concurrency in addition to cache hit rate. Use Instruments or a representative workload to validate memory rather than treating configured cost as RSS.

Acceptance criteria should include: every eviction can be recovered by recomputation, duplicate work has a known upper bound, stale results cannot overwrite newer values, keys remain immutable, and a missing cache entry never changes durable application truth. The cache should make the app faster, not more fragile.

NSCache is effective when entries are disposable, cost is approximate, and misses are routine. Its automatic policy is deliberately not a deterministic storage contract. Build around that reality with complete keys, bounded computation, explicit invalidation, and measured recovery.

Related:

Sources: