After we put out an update for Frahm a few weeks ago (and by extension, our other spatial audio plugins Transfer and Eigen) that let it support Ambisonic audio, we had a few people ask us about what Ambisonic audio actually is, what it can be used for, and how to use it.
The wikipedia article on Ambisonics is informative but not nescessarily a good guide on how to get started, so we figured, why not do a little writeup?
How Ambisonic Audio Works
Most spatial audio you run into day to day is built around speakers. Stereo assumes two of them. 5.1 and 7.1 assume a fixed ring of them around you. Even something as flexible as Dolby Atmos still ends up placing sounds relative to a known speaker layout or a fixed set of virtual spots.
Ambisonics just does its own thing: instead of describing where a sound should come from relative to speakers, it describes the entire sound field at a single point in space, speakers aren’t a concern at this stage. The actual playback system is basically an afterthought that gets bolted on at the very end.
That’s why Ambisonics ended up being the backbone of so much VR, 360 video, and game audio systems. The amount of speakers you have prior to decoding, where they’re sitting, or whether you’re even using speakers at all doesn’t matter. It’s just a way of capturing and storing a sound field, not a format tied to a specific playback rig.
Ambisonic Orders: How spatial detail gets added
The simplest flavor of this, first order Ambisonics (you’ll sometimes see it called B-format), uses four channels: one for overall pressure, plus three for front-back, left-right, and up-down. Together those four give you a rough, low-res map of the sound field. It’s enough to localize sound decently and spin the whole scene around freely, but the directional “sharpness” is limited.
Higher orders throw in more channels, each one adding a finer directional pattern on top (similar to adding higher harmonics to sharpen up a waveform). Second order is nine channels, third order is sixteen, and it increases from there. More order just means sharper spatial resolution; sources get reproduced more precisely, and the “sweet spot” where the illusion actually holds together gets bigger (and increasing the order also means you need more and more channels, which can start to choke up your computer).
In our update of our spatial plugins, the order gets automatically selected based on how many channels are available on a given track (ex. if you used Reaper to make a 16 channel track, the plugins would process Ambisonics in the 3rd order)
It’s also worth noting that higher order isn’t really about volume or channel count in the usual sense. It’s about how finely the directional info is resolved, more like bumping up the resolution on a spherical photo than adding more speakers to a room.
The way that this works is via the use of “spherical harmonics“, basically a set of polar / orthogonal functions that get more and more complicated as the “order” increases; like so:
Ambisonic Order Visualizer
Each shape is one channel of the sound field. Lobe size shows how strongly that channel picks up sound from a given direction.
This visualizer stops at an order of 3. The higher-order shapes are even more complicated, and thus give an even greater amount of spatial resolution.
Encoding and Decoding
Going in, sounds get “encoded” into Ambisonic format. If you’re dealing with a synthetic sound in a game engine or VR project, this usually just means taking a mono sound and a direction you want it to come from, then mathematically projecting that sound into the Ambisonic channels so it comes out from the right direction later.
Note that as of right now all of our spatial-oriented plugins are just for doing the encoding step of the process; external plugins would need to be used to converting the encoded audio to a decoded format.
If you’re capturing a real space instead, you would use specialized mic arrays: a handful of capsules arranged around a small sphere, each pointed a different way, whose signals get mathematically combined into the Ambisonic channels. When we worked with MNTRA on making Living Sky, they used a specialized microphone that had 64 different capsules in a sphere for capturing their Ambisonic impulse responses
Coming out, “decoding” takes those Ambisonic channels and figures out what each individual speaker (or each ear, for binaural rendering) should actually receive, based on the real playback setup. This is the step where the format is actually committed to something specific. Since decoding is just a math function applied to the stored sound field, the same Ambisonic file can get decoded differently for different rooms, speaker counts, or headphones, without ever touching the original recording or mix.
Enjoy this visualization of how basic Ambisonic encoding works (note that the W channel is always 1 in this case, as the W channel (or order 0) is just a mono representation of the signal:
Ambisonic Encoding / Panning Demo
Drag the dot to move a sound source around a full sphere and watch how it gets encoded into each ambisonic channel.
Beyond just Encoding & Decoding
Ambisonic rotators let you spin the entire sound field without re-encoding anything, which lets sound designers in games / movies easily “move” the scenes spatially without much difficulty.
Compressors and limiters built for Ambisonics tend to work differently than you’d expect. Rather than compressing each channel independently (which would distort the spatial image), they typically analyze the overall loudness of the sound field, or of a decoded reference layout, and apply gain changes in a way that preserves the relationships between channels.
Other Ambisonic tools can do a whole bunch of other unique things, a great place to get some would be the IEM Plugin Suite, along with our stuff, of course)
All Coming Together
Ambisonics solves one specific problem really well: it lets you capture or create a full spherical sound field once, then punt every decision about playback (headphones, speakers, room shape, which way the listener’s facing) down the line to a rendering step that might happen on a totally different device than whatever made the recording.
That flexibility is why it quietly became the default way of representing spatial audio in VR headsets and 360 video, even though most people never see the word “Ambisonics” anywhere in their game / video player. It’s just doing the behind-the-scenes work of turning “a field of sound” into whatever actually reaches your ears.
