Rules for the Real-Time Audio Thread

Every audio app has one piece of code that runs on a deadline: the render callback. At 48 kHz with a 128-sample buffer, it has about 2.7 milliseconds to fill that buffer — every time, without exception. Miss the deadline and the listener hears a click or a dropout.

That deadline is what makes audio code different. Average performance doesn’t matter; the worst case does. So the rules for the audio thread are really rules about avoiding anything whose cost is unbounded or unpredictable.

Don’t allocate memory

malloc and friends may take a lock, and may ask the operating system for more pages. Neither has a bounded cost. Allocate everything up front — buffers, voice pools, scratch space — and reuse it.

In Swift this is subtle, because allocation hides in ordinary-looking code. Appending to an array, creating a class instance, capturing a closure, or bridging to an Objective-C type can all allocate.

Don’t take locks

If the audio thread waits on a mutex held by the UI thread, it’s now running on the UI thread’s schedule. Use lock-free structures instead: a single-producer, single-consumer ring buffer covers most cases of passing messages or audio between threads.

Don’t do I/O

No file reads, no networking, no logging with print. If you need to stream audio from disk, read it on a background thread into a ring buffer, and have the callback read from that.

Don’t call code you don’t control

Objective-C message sends, Swift runtime calls and most framework APIs make no real-time guarantees. Some are fine in practice. Many are fine until they aren’t.

What’s left

Arithmetic, reading and writing memory you allocated earlier, and atomics. That turns out to be enough. A typical render block looks something like this:

let renderBlock: AURenderBlock = { _, _, frameCount, _, outputData, _ in
    // Pick up parameter changes sent from the UI thread.
    while let change = parameterQueue.pop() {
        synth.apply(change)
    }

    let buffers = UnsafeMutableAudioBufferListPointer(outputData)
    synth.render(into: buffers, frameCount: Int(frameCount))
    return noErr
}

Everything synth touches was allocated before the audio unit started. The queue is a fixed-size ring buffer of plain structs. Nothing in this block can wait.

Ross Bencina’s Real-time audio programming 101 is still the best introduction to all of this, and worth reading in full.