Mikhail Naganov

Recording vs. IR Rendering

This post is not a digression from the Spectral Correction topic. I needed to explore the recording vs. rendering topic because it directly affects the way I’m preparing the coming earspeakers vs. speakers comparison experiments. Unexpectedly, it also helped to find a flaw in the correction DSP filters for my LXdesktop speakers, and solidified my understanding that although LXdesktop / LXmini are not as linear as Genelec 8331A objectively (from measurements), they are still linear enough for my ears.

What does Recording vs. IR Rendering mean?

Capturing the behavior of an audio system for the subsequent “offline” evaluation is possible in two distinct ways:

  1. Making a recording of it while it’s playing some test sounds and music.
  2. Capturing the system’s linear transfer function (TF) as an impulse response (IR) and then convolving with it the same “dry” material as one would use for a recording.

From a theoretical perspective, only the recording captures the complete behavior of the system. That’s because the complete behavior is usually represented as the sum of the following components:

From my experience in this experiment, I would also add the time variance factor due to room temperature change. Note that we intentionally neglect any non-linearities and noise in the capturing chain itself assuming them to be negligibly smaller compared to the issues of the system that we intend to evaluate.

I already touched on the nature of these differences in my LXdesktop Auralization with Ambisonics post. At that time I concluded that an IR rendering sounds like a “better version of the original system.” (That comparison was with the system itself, not with a recording of it. But if a recording captures the system faithfully, the same difference should show up between a rendering and a recording.)

This time I actually want to settle this question by doing some blind ABX comparisons—will a rendering via the system’s IR sound the same as a recording, or different? In strict terms, what we want to verify is whether the success rate of distinguishing a rendering from a recording is higher than that of distinguishing between two recordings. The second comparison is the control: no two recordings are actually “the same” if analyzed at microscopic levels using DSP tools, yet they may still be subjectively indistinguishable in blind testing. If a rendering is no easier to tell from a recording than another recording is, then for practical purposes they are the same. And if they can be distinguished, then what is it that makes them different?

Why is this question important?

For our explorations of the spectral correction of phantom sources I will need to compare the sound of speakers vs. binaural playback over earspeakers. Creating a binaural rendering using IRs is much more robust since one works with “parts”: the IRs (or BRIRs—binaural IRs), the HRTF, the signals; and these parts can be adjusted independently and combined as needed. Recordings are more or less set in stone—one can adjust overall EQ, but it’s much harder if not impossible to adjust the amount of room reverb, for example.

In addition, making and working with recordings is cumbersome. Making a good recording requires quiet conditions. If one wants to make two recordings for further blind comparison, there should be no intermittent environmental noises that can easily reveal one recording from another. If the experimenter does not have access to special noise isolated chambers, the consequence is that recordings must be short, and the process of recording requires multiple takes, so that at least one recording ends up being “clean.”

Also, if binaural listening is planned to be used, recordings made with traditional microphone techniques including dummy head recording can’t be used with head-tracking systems. Only Ambisonics recordings allow post-recording rotation, however in order to make a truly high fidelity Ambisonics recording one needs an expensive high order microphone array.

The Experiment

The design goal of the experiment was to ensure that every comparison has a control of the same kind, so that a difference can be attributed to distortion, noise, or time variance rather than to the procedure.

By the way, the help from Claude Code was indispensable in every part of the experiment: the design, the verification, automated manipulation of the recorded fragments, and the analysis of the results. It’s as if I had a PhD-level assistant working tirelessly with me. So when talking about the experiment I will be using “we” meaning me + Claude.

We have split the experiment into two stages that only differ in the capture instrumentation: a single microphone (Earthworks M30) vs. a binaural capture (KU-100). This is to separate potential effects of binaural listening that may provide more information to the hearing system. Both stages share the same test signals and the testing protocol. The speaker was a second variable: I started with the LXdesktop and later, for reasons explained in the results, repeated the experiment with a Genelec 8331A.

The test signals that we used (all mono, all of 10 second duration):

We also placed beginning and end markers made of exponential sweeps (ESS). Their presence helped to align the recordings and renderings precisely in the face of time variance due to room temperature fluctuation (at first I used Dirac pulses but they ended up being too weak). I created a single file containing all these samples. This is the link to it. The waveforms look like this:

The capturing protocol was as follows:

  1. Set up the speaker—I used frontal speaker placement, either LXdesktop + subwoofer, or Genelec 8331A alone. The gain level was adjusted so that -20 dBFS pink noise produced 80 dB SPL at the recording position.
  2. Capture the room environmental noise (the noise bed) + any noise from idle speakers (I can barely hear any, but it will be present in the recordings of the test sample file).
  3. Using REW, capture 3 IRs using -20 dBFS level (we also tried -3, -7, -12, and -30 dBFS). Timing by the loopback was used.
  4. Play the test sample file, in a loop. I recorded 4 takes so that we could select the capture with the least random noise from the street: planes, loud cars, etc.
  5. Play the test sample file again, also 4 takes.
  6. Capture another 3 IRs again to see the room variance.
  7. One more capture of the room noise.

The purpose of capturing the noise was to build a sample that can be mixed together with the IR rendering.

This is a quick shot of the binaural capture setup with the Genelec 8331A and Neumann KU-100 aligned using a laser level:

When exporting the IRs from REW, I used a symmetric window and placed the t = 0 at the middle index. This is to ensure that we also capture the full behavior of Acourate’s DSP filters used for LXdesktop—this has turned out to be important!

From these captures and the test sample file, Claude built 4 files for listening:

In order to achieve sample-level accuracy, the IRs had to be stretched a bit. The ESS markers were trimmed out from the files for listening.

Then a shell script made copies of the files under the names “A”, “B”, “C”, “D”, and wrote the mapping into a file—which no one reads until the listening test is complete!

I listened to the anonymized files on IEMs (Etymotic EVO) and first tried to identify the clean rendering—that was always possible, although with the KU-100 it was not so obvious. Then I ran ABX tests, using a plugin in foobar2000, between the pairs of the remaining 3 anonymized files.

Results

Running this experiment turned out not to be easy! My initial attempt revealed that the operator (me) must be out of the room for IR capture (although it was obvious to me that I should be out during recordings, I didn’t think about that for the IR capture)—otherwise the captured IRs showed significant variations, because the reflections and diffractions from my body changed as I moved. I must note that I was amazed by the ability of Claude Opus to derive that “there is someone in the room at a distance of 1.5 meters from the microphone” just by looking at a set of IRs!

I bought a long USB cable so that I could be outside of the room. The next attempt produced stable IRs, however I immediately distinguished the recording from the rendering by two factors:

  1. Rattling during ESS sweeps. Although I attempted to dampen some of the things in the room to avoid that, not everything could be damped, for example the room door has an apparent 30 Hz resonance. The door can’t be removed. Luckily, the rattles were only excited by the sweeps, so trimming the ESS markers out of the listening files removed this cue.
  2. Pre-ringing before snare hits, present on the recording but not on the rendering. As I realized, it’s also audible when listening on the speakers but I did not notice it until I actually started listening carefully.

The analysis traced the pre-ringing to the DSP correction filter. The measured speaker IR had two very narrow, deep dips (at 1730 Hz and 900 Hz) of the “excess phase” kind, and Acourate’s correction inverts the excess phase with a time-reversed filter. The inverse of such a narrow dip rings for seconds, longer than the filter itself, so the filter carried a steady tone from its very first tap, 680 ms before the peak. At about −105 dB it was invisible when looking at the IR, but convolved with a sustained snare it accumulated into audible pre-ringing.

The fix was to make the two dips minimum-phase in the speaker IR (the magnitude response changed by at most 0.35 dB) and then simply repeat the filter generation in Acourate with the repaired IR. The new filter had no pre-ringing, and the next recordings confirmed it. I think this problem deserves a post of its own: it is not the kind of thing I was expecting from Acourate, and the correct way of fixing those nulls was not obvious at the first attempt.

The next attempt finally produced really good recordings, and I realized that the noise is the only factor that can be used for distinguishing between the rendering and the recordings: once the room noise was mixed into the rendering, the ABX scores were at chance (I used 16 trials per pair). I still had some doubts though, because I’m not an expert listener, and after the first blind ABX session I had the impression that the render was a bit easier to pick out, even though the scores didn’t show it. The residual analysis found distortion in the recordings that the render lacks, which seemed to explain the impression, so we spent time locating its sources. There were 3 sources of distortion:

  1. The distortion of the full range driver in the midrange region. This is something noted both by myself and Erin. If you recall I had spent some time trying to find a driver which does not exhibit them when used in a baffleless configuration, but it seems to be nearly impossible.
  2. The distortion of the woofer at the lower end due to strong excursions.
  3. And—that’s the most interesting one—a Doppler-like distortion of the full range driver when the woofer is playing a low frequency tone (confirmed for 100–120 Hz). Although in the LXmini and LXdesktop design the radiating surfaces are orthogonal to each other, unlike in a coaxial driver arrangement, there is still some mechanism present which causes the intermodulation. Maybe it results from the reactive behavior of the air above the woofer’s surface, maybe these are reflections from the woofer cone—definitely a topic to explore!

To see why the modulation of the full range driver by the woofer was hidden in plain sight, compare these two RTA graphs (1/48 octave) with woofer being silent and playing a 100 Hz tone and the full range driver playing a 8127 Hz tone:

There is practically no difference visible except for the presence of the woofer signal and its harmonic distortions. That’s because 1/48-octave bands at 8 kHz are about 117 Hz wide, so the ±100 Hz sidebands fall inside the adjacent bands. But if instead of the RTA we use spectrum view, we can see that the spike of the full range driver becomes a bit fatter:

And if we zoom in around the full range tone:

Now you can see intermodulation tones to the left and to the right, that’s why I said it’s “Doppler-like.” They are at a benign level though: about 50 dB below the main tone.

Since none of these distortions of LXdesktop are easy to remove, I decided to repeat the experiment using Genelec 8331A which is much more linear. I tried both monaural and binaural recording (with KU-100), and I wasn’t able to distinguish between a rendering+noise vs. a recording. A second blind ABX session with the LXdesktop recordings, and the sessions on the Genelec, showed that my initial impression of the rendering sounding a bit “cleaner” was wrong: the distortion is measurable but not audible in this material. This settled the question.

If you want to listen to these recordings and renderings, use the links below:

Other Interesting Findings

With the system calibrated to produce 80 dB SPL from -20 dBFS pink noise, we checked whether it’s better to use the IR captured at -20 dBFS, -3, -7, -12 or -30 dBFS. It turned out that the IR from the -20 dBFS sweep is the best (the -30 dBFS sweep shows even less level dependence, but it is significantly noisier). The music signals drive the woofer less hard than even the -30 dBFS sweep does, which means that they are generally played in the linear region of the speaker operation.

The changes in the speed of sound due to room and equipment temperature changes are noticeable at the DSP level. IRs captured in the beginning and the end of the session allowed us to measure the drift. As a consequence, the render had to be stretched in order to match recordings with sample accuracy.

The KU-100 recordings sounded less noisy than the M30 recordings. At first I attributed that to the specs: 16 dB(A) self-noise for the KU-100 against 20 dB(A) for the M30, which matched the measured 4–5 dB A-weighted SNR difference. That match turned out to be a coincidence. Once both chains were put on a calibrated SPL scale, their noise differed by only about 1.3 dB A-weighted; what differed was the signal. The head delivers about 4–5 dB more of a frontal speaker’s sound to the ears than the M30 receives at the same position. Decorrelation of the noise between the ears was not the explanation either: a simulated pair of M30s with decorrelated noise still sounded hissier than the KU-100.

When the simulated pair’s signal was raised by the measured head gain, it sounded cleaner than the KU-100, because the dummy head picks up more room noise below 1 kHz and above 8 kHz, which A-weighting under-counts. So for a mono capture that should sound as clean as a binaural one, the cure is more signal at the microphone, not a microphone with even lower self-noise.

Again, if you want to hear these differences yourself, use the link.

Conclusions

The main conclusion of course is that for a reasonably linear speaker, a mix of a render via the system’s IR + room noise is indistinguishable from a recording made with the same setup. That holds even in the case when the system has some known non-linear behavior by measurements (LXdesktop). I suppose that when using earspeakers for listening to the IR rendering, the mixing in of noise is not needed because the same room noise is heard naturally.

Also, the exercise of comparing the IR rendering with a recording may reveal some hidden problems of the system. I suppose any kind of critical listening can help, however, by making the comparison and experimenting with the properties of the IR rendering it is possible to narrow down the possible sources of the discrepancy, and eventually fix them.