Bits is Bits? Right?Check Again.

George Blood, Safe Sound Archive

Joint Technical Symposium

Oslo, Norway

Monday, May 3, 2010

Typical Playback ChainAnalog source

Analog-to-digital converter

Digital Audio Workstation

Digital File

Physical Carrier of Files

Typical Playback ChainAnalog source

Analog-to-digital converter

Digital Audio Workstation

Digital File

Physical Carrier of Files

}

From ADC to Digital FileADC Digital Output (AES-3)

Cable/patching/switching

Digital Audio Workstation:

Digital File

Computer: CPU/Bus/etc.

Driver

Software

OS/Drivers

Storage interface/drivers: FW, USB, SATA, FC, iSCSI

Storage Device (HDD)

Interface: AES-3 to FW/USB/etc.

Disappointments

• Don’t mention brands

• Don’t tell you our solution

• Very pervasive problem

• http://www.avpreserve.com/wp-content/uploads/2010/01/Digital_Audio_Interstitial_Errors.pdf

Disappointments

• Planned to announce a software tool to automate this testing….

Tested Hardware

• Lynx AES16

• M-Audio (96 and 192)

• M-Box

• Metric Halo ULN-2

• Nagra V

• SonicStudio 302 (aka ULN2+DSP)

Tested Software

• Boom

• Metric Halo Console

• ProTools LE (Mac)

• Sonic Solutions HDSP

• Sonic Studio soundBlade

• SoundForge

• WaveLab

Tested OS

• Mac OS9

• Mac OS X

• Windows 98

• Windows Home

• Windows ME

Tested Drivers

• Stock drivers (mostly)– OS stock drivers– Hardware bundled drivers

• Dedicated Drivers– OS 9 (Sonic Solutions)– OS X (SonicStudio & MH)

Tested Storage

• Fat32• FibreChannel (2Gb, RAID5) • FireWire 400• HFS+ (SCSI Ultra160)• HFS-extended• SATA

Not Tested

• Audacity• AudioCube• [Bias Peak]• iSCSI• ProTools HD• ProTools LE (PC)• Pyramix• RME• ADC to FireWire

Testing Multiple Components Simultaneously• A b 1• A b 2• B b 1• B b 2• A a 1• A a 2

Finding errors

• 100% Listen– Inefficient– Unreliable

• Software verification– Doesn’t certify authenticity– Limited parameter testing– Unreliable

Finding Errors

• Null Test:– 2 presumed identical files– Sample-accurate alignment– Invert polarity– Add together

• If a1=a2, then a1 - a2=0

Aligning 2 stereo pairs (1)

Aligning 2 stereo pairs (2)

Aligning 2 stereo pairs (3)

Aligning 2 stereo pairs (4)

Aligning 2 stereo pairs (5)

Aligning 2 stereo pairs (6)

…then invert 1 pair

Testing for 2 parameters

• Time– Number of samples

• Amplitude– Number of bits

Null if mis-aligned by 1 sample

Error of 76 samples dropped

from: http://www.avpreserve.com/wp-content/uploads/2010/01/Digital_Audio_Interstitial_Errors.pdf

Bits vs. samples missing

• 1010101010101010

• 1010101011101010

Bits vs. samples missing

• 1010101010101010

• 1010101011101010

Null if gain changed

What happened?

• New Hardware– Failure

• Reviewed set up with engineer– Failure

• Reviewed old systems– Failure

• Simplified Test– Finally some passes– Failures continue between identically

configured systems

Qualifying a Control

• Round Robin Test– A≠B– B≠C– A=C– …B is wrong

Qualifying a Control

• Assumes only one variable at play– Host– I/O Hardware– Driver– Application

…test configuration

Hardware in hand• SonicStudio

– Great editor– Semi-proprietary file types– Non-destructive editing– No support for WAV or BWF output– No AES-31 ADL support

• Nagra V– Simple– Self-contained– Can’t change variables

• (other than resolution)

Results

• Null tests passed

• Every time

• Always

• Validates testing procedures

Talent Pool

• Head of QA with physics background

• 2 experienced audio engineers

• 2 dealers

• 2 outside technical engineers

Tech “support”

• Universally appalling• Short attention span• Don’t understand concepts• Always some other link in chain• Regional sales reps care more• Engineer curious but unable to help

Computer code “traps”

• Intractable problem buried in code• Build routine to watch for it• Applies fix• Efficient solution

Computer code “traps”

• Masking (aka “hiding”) problem!– Little crossfades at glitches– Other lies…

True Nulls

Very small error in Null

Very small error in Null

Close up

Incomplete Null

Incomplete Null

Non-zero offset

Calculating size of error

• “Normalize” difference signal– Increases gain to zero headroom

• 1 bit = ~ 6dB • Normalizing amplitude

divided by 6dB

equals # of bits (amplitude) of difference…

Calculating size of error

• 90dB / 6 dB = 15 bits

Incomplete Null

Spectrum of difference signal

Conclusion

• Not passing 24 bits

• Truncating at 16 bits

• Dithering 16th - 24th bits– More noise but Lower distortion

than truncating at 16 bits and dithering 17-24

Struggling to keep up

Different bit depth, all non-zeroExample 3 drops samples

Struggling to keep up

Example 1 & 2 Normalized -- “properly” dithered

“Well-dithered lie”

Toggling single bit

Are there no limits?

“Fatter” difference on L

Display higher resolution on R

Are there no limits?

“Fatter” difference on L

“Full” null - absurd gain change - on R

Are there no limits?

Different residual on L!

More absurdities

• Same hardware, different ages, different results

• Same configuration, different results on different days

• Same hardware, good results with one software; poor results with different software

Conclusions

• Get what you pay for– More expensive hardware & software performs better in

Null tests

• Stock drivers aren’t as good as hardware specific drivers– keep up to date, especially after OS upgrades

• Non-host-based systems perform significantly better• Mac-based systems performed better than PC-based

systems, but are far from flawless• Worst PC and Worst Mac systems equally bad• These problems are widely known by manufacturers

and blatantly hidden