AVCaptureSession on macOS: Device Setup, Frame Delivery, and Interruption Recovery

AVCaptureSession connects media inputs such as a camera or microphone to outputs such as photo capture, movie recording, or video data callbacks. It owns the active capture graph and coordinates access to capture devices. The session’s configuration, its running state, sample delivery, and UI preview are related but distinct concerns. A responsive macOS capture app gives each one a clear owner and handles interruptions as normal state transitions.

Before adding an input, determine that the app has a camera or microphone that matches the feature and obtain the required user authorization. Include the appropriate NSCameraUsageDescription and NSMicrophoneUsageDescription strings in the app’s information property list when those media types are used. Attempting access without a required usage description can raise an exception. Do not treat a black video frame or silent sample as successful capture; authorization may still be unresolved or denied.

Configure the graph transactionally

Discover an available device using the supported device-discovery APIs, create an AVCaptureDeviceInput, and ask the session whether it can add that input before mutating the graph. Add compatible outputs and select a session preset only if the session accepts it. Use beginConfiguration() and commitConfiguration() to group changes into one configuration update.

import AVFoundation
import Foundation

final class CaptureController: @unchecked Sendable {
    // AVCaptureSession is not Sendable. This controller confines every
    // session access to sessionQueue; do not expose the session elsewhere.
    private let session = AVCaptureSession()
    private let sessionQueue = DispatchQueue(label: "com.example.capture.session")

    func configureAndStart() {
        sessionQueue.async { [self] in
            guard let camera = AVCaptureDevice.default(for: .video),
                  let input = try? AVCaptureDeviceInput(device: camera),
                  session.canAddInput(input) else {
                return
            }

            session.beginConfiguration()
            session.addInput(input)
            if session.canSetSessionPreset(.high) {
                session.sessionPreset = .high
            }
            session.commitConfiguration()
            session.startRunning()
        }
    }

    func stop() {
        sessionQueue.async { [self] in session.stopRunning() }
    }
}

This illustrates session serialization but omits authorization, output configuration, and error reporting; without an output it does not deliver frames to an app consumer. In production, check AVCaptureDevice.authorizationStatus(for:) and request access in context before creating an input. Ensure commitConfiguration() runs on every path after beginConfiguration(); a defer in a small helper is often safer when several additions can fail.

The @unchecked Sendable conformance is justified only by the example’s private session and strict queue confinement. It is not a blanket way to silence Swift concurrency diagnostics: if another object reads or mutates the session outside sessionQueue, the invariant no longer holds. A preview integration must define a deliberate handoff boundary rather than exposing the session as a freely mutable property.

startRunning() is a blocking call that can take time. Apple’s capture-session documentation recommends starting it on a serial queue so the main queue remains responsive. Use the same queue to serialize configuration, start, stop, and recovery transitions. Do not let a button callback block while the capture graph is being prepared.

Separate session work from frame processing

For AVCaptureVideoDataOutput, assign a sample-buffer delegate and a dedicated serial callback queue. A serial queue provides ordered callback delivery for that output and helps prevent frame processing from racing itself. Do not perform expensive image analysis, disk writes, or network uploads synchronously inside a frame callback. Decide whether to process every frame or drop late frames; for a live preview, processing the newest available frame is often more useful than building an ever-growing backlog.

When a callback hands work to another queue, retain only the required sample or copy immutable data before returning if the sample’s lifetime or reuse policy requires it. Bound memory in any frame queue, measure processing latency, and attach a capture generation to results. If a new session begins while an older vision result is still running, discard the old result rather than drawing it over the new preview.

Video frames include timing and format information. Use the sample buffer’s presentation timestamp rather than wall-clock arrival time when synchronizing audio and video. Account for orientation and mirroring at the output or display transform boundary. Test front-facing and external cameras separately; their supported formats and controls may differ.

Preview layer and AppKit presentation

An AVCaptureVideoPreviewLayer can display a session’s live video. It is a rendering surface, not a separate capture graph. Tie the preview layer to the currently owned session, update its frame when the AppKit view resizes, and choose the video gravity that matches the product’s crop policy. If the app renders frames in a custom view, define color, orientation, and frame-drop behavior explicitly.

Keep view mutations on the main thread or main actor. Session configuration and sample processing should not block that thread. A preview can disappear while capture continues, or the feature can stop capture when its view closes; choose the lifecycle deliberately. Do not leave a camera session running invisibly just because a controller forgot to stop it.

Interruptions, runtime errors, and device changes

Observe session interruption and runtime-error notifications. isRunning and isInterrupted describe different conditions: a stopped session may be stopped by the app, while an interrupted session may need a recovery action after the interruption ends. Do not respond to every notification by tearing down and rebuilding the entire graph blindly. Inspect the reason, preserve user intent, and retry only when the relevant condition has ended.

Device availability can change if an external camera is unplugged or another system event takes control. Handle input removal and failed device configuration as a normal error state. Offer a device selector when multiple cameras are part of the workflow, and resolve the selected device again before restarting; a stored device name is not a guaranteed hardware identity.

If the user revokes camera or microphone access, explain the missing capability and provide a link to the appropriate system settings where supported. Do not attempt to alter privacy databases or silently switch to another input. Re-check authorization when the app returns to the foreground because settings may have changed while the process was inactive.

Recording and file finalization

Photo capture and movie recording use different output objects and completion semantics. A running session does not mean a recording has started, and a recording stop request does not mean its file is finalized. Wait for the output delegate’s completion result before publishing or moving the file. Write to a staging location, verify the result, then commit it to the user’s chosen destination.

For long recordings, define storage exhaustion behavior, duration limits, audio/video synchronization, and recovery after a device disconnect. Keep file I/O away from the sample callback queue unless the output API explicitly owns that writing path. Do not expose a partially finalized movie as a complete artifact.

If recording uses separate audio and video inputs, add them to the same session and validate both connections before presenting a “recording” state. A camera preview can run while microphone authorization is denied, so the UI must not imply that a movie contains sound unless the audio input is actually active. Store the selected media settings with the recording generation so a later completion callback cannot finalize a file into a destination the user has already replaced.

When the application enters a lifecycle state where capture should stop, stop the session on its owning queue and wait for the transition before releasing output objects. Closing a window is not the same as finishing a recording: ask whether the user wants to stop, continue in a clearly visible app-level mode, or cancel the file. Make this policy explicit rather than allowing object deallocation to decide whether the capture graph is still active.

Diagnostics and acceptance tests

Test permission not determined, granted, denied, usage description missing in a test build, camera absent, camera unplugged, configuration rejection, session interruption, runtime error, slow frame processing, app background/foreground, window close, and disk full during recording. Assert that the main thread remains responsive, frame queues stay bounded, and the session stops when the feature’s owner releases it.

Record session generation, device class, preset, input/output counts, authorization state, running/interrupted transitions, frame timestamps, dropped-frame count, and runtime errors. Avoid saving camera samples or audio in diagnostics. Measure time to first preview frame and end-to-end processing latency under realistic CPU load.

The reliable capture pipeline uses an explicitly owned session queue, permission before input creation, bounded frame delivery, and a recovery state machine. AVFoundation moves media through the graph; the application must decide when capture is active, which results are current, and when the media file is truly complete.

Related:

Sources: