Skip to content
macOSDeep Dive Published Updated 8 min readViews unavailable

ScreenCaptureKit on macOS: Stream Ownership, Frame Delivery, and Recovery

Build ScreenCaptureKit pipelines with explicit consent, content filters, bounded sample processing, live reconfiguration, audio timing, and teardown.

ScreenCaptureKit delivers selected display, app, or window content to a Mac app as media sample buffers. It is a capture pipeline, not a screenshot function with a continuous timer: the user chooses or authorizes content, the app constructs a filter and output configuration, a stream emits asynchronous buffers, and downstream work must keep pace without blocking the callback. A production design gives each transition an owner and makes the stop path as deliberate as startup.

This guide focuses on SCStream for ongoing capture. A one-frame capture can use the framework’s screenshot API; a continuous recording or analysis workflow needs stream lifecycle, output registration, and bounded processing. Screen capture is sensitive. The feature should explain what will be captured, use the system content-sharing picker when appropriate, and avoid collecting unrelated windows or audio.

Discover content and make a narrow filter

Use SCShareableContent to discover the displays, windows, and applications available for selection. The result describes what is shareable at that moment; it is not a permanent inventory. A window can close, an app can exit, and display topology can change between discovery and stream creation. Resolve selection again when the user returns to a stale capture setup rather than assuming an old object remains valid.

Build an SCContentFilter for the smallest useful source. A display filter may include or exclude selected windows; a window filter should capture only the chosen window when that matches the product. Prefer the system picker for user-driven selection rather than inventing a competing permissions dialog. Keep the selected source and filter in a capture model so the UI can show what is currently being shared and let the user stop or change it.

Configure output before starting

SCStreamConfiguration describes output dimensions, frame interval, pixel format, audio capture, and queue behavior. Choose dimensions based on the capture target and intended output, not merely the window’s point size. Retina scale and display mode affect the relationship between logical geometry and pixel dimensions. Measure resulting frame size and memory use on actual displays before setting a fixed policy.

The configured sample queue is a latency and memory tradeoff. A deeper queue can absorb brief downstream stalls but retains more frames and can make displayed analysis stale. Keep the callback cheap, choose a bounded queue depth, and drop work that is no longer useful instead of building an unlimited app-side backlog. If the product needs every frame, treat that as a throughput requirement and verify sustained processing under realistic load.

import CoreMedia
import Foundation
import ScreenCaptureKit

final class CaptureOutput: NSObject, SCStreamOutput, SCStreamDelegate {
    private var stream: SCStream?
    private var stopInFlight = false
    private let sampleQueue = DispatchQueue(label: "capture.samples")

    func start(filter: SCContentFilter, configuration: SCStreamConfiguration) throws {
        precondition(Thread.isMainThread, "Serialize capture lifecycle changes on the main queue")
        guard stream == nil, !stopInFlight else { throw CaptureLifecycleError.transitionInProgress }

        let stream = SCStream(filter: filter, configuration: configuration, delegate: self)
        try stream.addStreamOutput(self, type: .screen, sampleHandlerQueue: sampleQueue)
        self.stream = stream
        stream.startCapture { [weak self, stream] error in
            guard let error else { return }
            NSLog("Capture start failed: %@", String(describing: error))
            DispatchQueue.main.async {
                if self?.stream === stream { self?.stream = nil }
            }
        }
    }

    func stream(_ stream: SCStream, didOutputSampleBuffer sampleBuffer: CMSampleBuffer,
                of type: SCStreamOutputType) {
        guard sampleBuffer.isValid else { return }
        // Hand off bounded processing; do not block the capture callback.
    }

    func stop() {
        precondition(Thread.isMainThread, "Serialize capture lifecycle changes on the main queue")
        guard let oldStream = stream, !stopInFlight else { return }
        stopInFlight = true
        oldStream.stopCapture { [self, oldStream] error in
            if let error { NSLog("Capture stop failed: %@", String(describing: error)) }
            DispatchQueue.main.async {
                if self.stream === oldStream { self.stream = nil }
                self.stopInFlight = false
            }
        }
    }

    func stream(_ stream: SCStream, didStopWithError error: Error) {
        NSLog("Capture stream stopped: %@", String(describing: error))
        DispatchQueue.main.async {
            if self.stream === stream { self.stream = nil }
        }
    }
}

private enum CaptureLifecycleError: Error {
    case transitionInProgress
}

The coordinator must also retain the CaptureOutput object strongly for the active session. The example serializes lifecycle changes on the main queue, keeps the stream property populated until stop completion, and captures both the owner and stream in the completion closure. That prevents the output/delegate owner from disappearing while shutdown is still in progress. Start failure and delegate stop events clear only the matching stream; a full app should reconcile delegate failures with UI state, disable UI actions while transitions are pending, and make repeated start/stop calls idempotent. Do not create a new stream for every view redraw.

Treat sample buffers as timed media

The output callback identifies the stream output type and supplies a CMSampleBuffer. Validate the buffer before reading it. For video, inspect frame attachments and status rather than assuming every callback contains a complete image. Use the sample presentation timestamp for ordering and synchronization. A callback arriving does not mean that downstream encoding, display, or storage has completed.

Screen and audio outputs have different formats and processing costs. Give them queues that preserve the ordering each consumer requires. If audio and video are combined later, retain timestamps and use an explicit clocking strategy; arrival order on independent queues is not a synchronization guarantee. Avoid doing image conversion, compression, disk I/O, or UI layout directly inside the callback.

For analysis or preview, use a bounded handoff: a serial worker can process in order, while a latest-frame slot can discard superseded frames when low latency matters more than completeness. Update AppKit or SwiftUI state on the appropriate UI executor after work completes, and check that the result still belongs to the active stream generation. This prevents an old frame from a stopped stream replacing a newer capture state.

Make the overload policy visible in the architecture. A live thumbnail may prefer the newest complete frame and intentionally discard older work; an archival recording may need to preserve every sample and therefore must slow or segment its downstream writer rather than silently skipping. These are different product contracts and should not share an undocumented queue policy. Record dropped-frame counts and queue occupancy so a user report of stutter can be distinguished from a slow UI consumer.

If processing uses Core Image or Metal, retain the sample’s pixel-buffer-backed resources only as long as the GPU or encoder needs them. Copying each frame into a second full-size buffer can double memory pressure. Reuse bounded pools where the media API supports them, and keep per-frame metadata such as time, dimensions, color attachments, and stream generation beside the image. Test that a capture restart cannot cause a completion from the old frame pipeline to publish into the new one.

Reconfigure without confusing stream identity

Apple documents live updates to a stream’s configuration and content filter. Use those APIs when a person changes the selected window, output size, or audio policy, and serialize updates so two UI actions cannot race. Keep the current filter and configuration as app state. If an update fails, preserve the last known working state or stop with a visible error; do not claim the new source is active before the completion result confirms it.

On display removal, window closure, or application termination, re-evaluate whether the filter is still meaningful. Some transitions can be repaired by selecting a new source and updating the filter; others should stop and request a fresh choice. Retain a generation identifier and discard queued buffers after that generation ends. The stream object represents the capture session, while the user selection and destination file are separate model state.

Recording, privacy, and failure boundaries

Capturing samples is not the same as producing a playable movie. A recorder must define container and codec choices, append compatible timed samples, handle backpressure, finish writing, and verify the output before publishing it. Keep an incomplete temporary file distinct from the final artifact. If the user cancels, close the writer and remove only the temporary output the current task owns.

Screen recording permission, content selection, and microphone or system-audio capture are distinct concerns. Ask for only the capture mode the feature uses and communicate the active state visibly. A permitted screen stream does not authorize a product to record every available audio source or upload the result. Keep file retention and export behavior explicit, and do not log captured frame contents or sensitive window titles.

Operational acceptance checks

Exercise permission granted, denied, and changed in System Settings; picker cancellation; a selected window closing; display attach and detach; app sleep and wake; start failure; stop during callback activity; rapid source changes; audio disabled and enabled; and a slow encoder. Check that the stream stops, output handlers are removed, pending work is bounded, and the UI names the actual current source.

Measure start latency, callback interval distribution, dropped or incomplete frame count, processing queue depth, peak memory, stop latency, and audio/video timestamp drift. Test on both high-density and standard-density displays and at the target output resolution. A reliable implementation treats ScreenCaptureKit as a timed stream with system-managed capture boundaries, not as a promise that every requested frame will be processed by the app.

Related:

Sources:

Comments