Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
r&d legal direction
evolving views on HOA:from technological to pragmatic concerns
Jérôme Daniel Orange Labs
Ambisonics Symposium, Graz, 2009/06/25
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 2 r&d legal direction France Telecom Group
introduction (chronology)
main concepts and promises
focus on: 5.0 decoding and HOA microphone array
HOA tools and integration (plugins)
format, standardization, coding
HOA in "real life": learning from recording / production experiments
1
2
3
outline
4
5
6
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 3 r&d legal direction France Telecom Group
introduction / earlier study and motivations
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 4 r&d legal direction France Telecom Group
Ambisonics and HOA: a chronology of space expansion…
70's 90's 96-00 03-06 07/08
[Bamford][Poletti][Daniel][Nicol][Sontacchi…]
[Gerzon, Craven,…]
SoundField
[Daniel, Nicol, Moreau, Bertet]
EigenMike(mh-acoustics)
[Gerzon][Malham]
Head-tracked binauralPlugins VST, etc.
09…
AmbisonicsSymposium…MPEG AudioBIFS
SRP (Trinnov)
00-03
[Fazzi] [Solvang] [Adriensen] [IEM…]
[Laborie et al]
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 5 r&d legal direction France Telecom Group
higher order ambisonicsin short
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 6 r&d legal direction France Telecom Group
Higher Order Ambisonics (HOA)increase angular discrimination thanks to additional encoding directivities
spatial encoding ≈ circular Fourier Transformspatial spectrum = {ambisonic components}spatial bandwidth = highest angular frequency
1st order 2nd order 3rd order 4th order
cosθ
sinθ
cos 2θ
sin 2θ
cos3θ
sin 3θ
cos 4θ
sin 4θ
0th order
enrichedspatial
bandwidth
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 7 r&d legal direction France Telecom Group
enhancedbeamwidth
Higher Order Ambisonics (HOA)increase angular discrimination thanks to additional encoding directivities
spatial encoding ≈ circular Fourier Transformspatial spectrum = {ambisonic components}spatial bandwidth = highest angular frequency
enhance spatial separation to feed loudspeakers more selectively
synthesize directivities with enhanced beamwidthspatial decoding ≈ multi-directional beamforming
Front (X)
Back
Left(Y)
Right
+ + + +
= = = =
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 8 r&d legal direction France Telecom Group
Higher Order Ambisonics (HOA)increase angular discrimination thanks to additional encoding directivities
spatial encoding ≈ circular Fourier Transformspatial spectrum = {ambisonic components}spatial bandwidth = highest angular frequency
enhance spatial separation to feed loudspeakers more selectively
synthesize directivities with enhanced beamwidthspatial decoding ≈ multi-directional beamforming≈ inverse discrete circular Fourier Transformmore accurate sound images (reduced spread angle)
1st order 2nd order 3rd order 4th order
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 9 r&d legal direction France Telecom Group
NFC-HOA [Daniel]filter implementation improvement[Adriaensen]More workable scheme for close sources: High-passed (NFC)-HOA [Daniel, Moreau]Further connections with WFS [Nicol, Daniel] [Fazzi], …
HOA as a "holophonic" approach
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 10 r&d legal direction France Telecom Group
HOA claims and promises
claimsformat flexibility
• (wrt reproduction setups, spatial manipulation, scalability)spatial "objectivity" and predictability
• representation reproduction ⇒ spatial fidelity and transparency?high res, "true 3D" recording technology
promises for many application contextsa format for new 3D audio content generation and consumption!?
• one generic content for various terminals, transport constraints, consumption styles
immersive telecommunication: teleconferencing, ambience sharinginteractive 3D navigation, games…
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 11 r&d legal direction France Telecom Group
A way to turn dreams into reality
assess or criticize HOA claims to improve the techno
consider format standardization (also matter of maturity)
test out HOA with "real life" concernscurrent format and equipment standards (5.0), current practicesget lessons…
bring HOA techno into the hands of content creators / sound engineers
make advertising, adapt practices, facilitate the convergence between research and production worlds
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 12 r&d legal direction France Telecom Group
focus on: decoding for standard 5.0 setupspherical HOA microphones
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 13 r&d legal direction France Telecom Group
HOA decoding for standard 5.0 setup (+8.0)
concerns: image stability, consistency, homogeneity, discreteness…
= physical limit for energy vector(pair-wise pan-pot)
+ = target, ie ideal sound image)* = "energy vector" (HF prediction)□ = "velocity vector" (LF prediction)
energy vector optimized Craven [AES24] energy vector optimized
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 14 r&d legal direction France Telecom Group
Mainly these two decoders were involved in recording+production experiments reported laterMany other potential decoders, by combining criteria in different ways
HOA decoding for standard 5.0 setup (+8.0)
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 15 r&d legal direction France Telecom Group
HOA sphere microphone: basicsQ sensors on the sphere
sound field samplingQ=32 4th order, K=25 HOA components
processing = matrix + EQEQ:
• theoretically -mx6dB / oct !• one must limit bass-boost
rough( / )111
r cB+
rough( / )100
r cB+
rough( / )r cmnBσ
rough( / )10
r cmB+
rough( / )1 r cmmB+
rough( / )1 r cmmB−
( / , / )1
r c R cEQrough( / )1
11r cB−
rough( / )110
r cB+
NFC( / )111
R cB+
NFC( / )100
R cB+
NFC( / )R cmnBσ
NFC( / )10
R cmB+
NFC( / )1 R cmmB+
NFC( / )1 R cmmB−
NFC( / )111
R cB−
NFC( / )110
R cB+
( / , / )1
r c R cEQ
( / , / )1
r c R cEQ
( / , / )r c R cmEQ
( / , / )r c R cmEQ
( / , / )r c R cmEQ
( / , / )r c R cmEQ
Matrix
N x K
( / , / )0
r c R cEQ
Q HOA signals
102 103 104
0
20
40
60
80
100
120
140
160
180
200a = 5 cm
Fréquence (Hz)
Am
plitu
de (d
B)
01234
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 16 r&d legal direction France Telecom Group
HOA sphere mic: limits and tradeoff
estim
atio
n er
ror
102 103 104
01
2
3
4
Frequency [Hz]
Ord
er
-60
-50
-40
-30
-20
-10
0
10
(dB)
shift towards LF when radius increases
shift towards HF when radius decreases
reduced spatial bandwidth
spatial aliasing
correct estimation
∅7cm,32 sensors
25 components(4th order)
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 17 r&d legal direction France Telecom Group
102 103 1040
10
20
30
40
50
60
Frequency (Hz)
(dB
)
rigid sphere arrays: trade-offs
(for a given radius)
increase EQ max level…bandwidth enlarges towards LF, but…less benefits for higher ordersand increase of noise level
increase the number of sensors Q…4 x more sensors to gain 1 octave towards HFSNR and robustness improvment: +10*log10(Q) dB=> e.g. 15dB for Q = 32 sensors
+18dB 3 oct1,5 oct 1 oct
etc.
+15dB SNR
Qx4
1 oct
spat
iala
liasi
ng
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 18 r&d legal direction France Telecom Group
HOA microphones built and experimented
EigenMike™(mh-acoustics)
32 caps 4th order
FTR&D32 caps 4th order12 caps 2th order20 caps 3th order (Sennheiser mke4)
(DPA4060)(Panasonic 2€)
alternative[Epain & Daniel]
8 caps 3th order, 2Dreduced spatial aliasing
⇒ great improvement in terms of usability!
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 19 r&d legal direction France Telecom Group
Impact of imperfect encoding
Craven o4 /o1Craven o4 /o2Craven o4 /o3Craven o4 /o4
4th order 3rd ordertruncation
For instance: on Craven, 4th order 5.0 decoding
2nd ordertruncation
1st ordertruncation
to lower frequencies
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 20 r&d legal direction France Telecom Group
HOA tools, integration and demonstrators
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 21 r&d legal direction France Telecom Group
HOA VST plugins (Orange Labs)HOAEncoderHOAMicProcessorHOARotatorHOASpkDecoderHOABinDecoder
Tested host applicationsPlogueBidule, (MaxMSP)(Cubase, Nuendo), Podium
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 22 r&d legal direction France Telecom Group
format, standardization, compression
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 23 r&d legal direction France Telecom Group
HOA format: which need for standardization?
for interoperability between HOA processing unitsfor sharing HOA sound files
no or few concern with compression⇒ associated data, extended file header, new 'trunk'
for insertion in 3D interactive multimedia contents⇒ eg MPEG4 (cf AudioBIFS V3 norm)
for generalized consumption: broadcast, exchange high concern with compression issuesextended issues: format conversion, spatial audio object coding
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 24 r&d legal direction France Telecom Group
HOA in "real life" (incl. mass market concerns)learning from experimental and collaborative recording opportunities
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 25 r&d legal direction France Telecom Group
Various recording conditions and constraints
immersive ambiencemusic / theatre performance (often codified spatial organisation⇒cultural expectations)without or with video or even stereoscopic video
⇒ mic positioning constraints; ⇒ sound and visual image coherency issues
with or without spot/close microphone mixingdirect mixing of post-producedwith concurrent mutlichannel system (trees) EWOin collaboration with professional sound engineers Radio-France
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 26 r&d legal direction France Telecom Group
Fully immersive "nature ambience"
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 27 r&d legal direction France Telecom Group
classic organization: frontal scenefront loudspeakers provide the orchestra image (quite dry)rear loudspeakers for field reverberated by the room
Orchestre National de France, Studio 104, Radio-France, Juin 2008
32 DPA4060, 4th order
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 28 r&d legal direction France Telecom Group
very large front sceneWorkshop Ears Wide Open, Le Tambour, Rennes, Mars 2008
multi-microphone trees
HOA sphere20 DPA, order 3
large scene more or less contributions from rear loudspeakers
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 29 r&d legal direction France Telecom Group
panoramic sceneSoundPainting session, Chapelle des Ursulines, Lannion, November 2008
EigenMike, ordre 4
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 30 r&d legal direction France Telecom Group
Opéra de Rennes: La Trahison Orale
"with height" spatial organisation
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 31 r&d legal direction France Telecom Group
stage
orchestra pit
stalls (audience)
front
wal
l
Spatial configuration: "La Trahison Orale"
difficult trade-off between sound balance and spatial readibility3D 2D projection issues depending on the microphone positioning and "pointing direction"
tubabalcony
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 32 r&d legal direction France Telecom Group
Don Giovanni, Opéra de Rennes (2 June 2009)3D video and audio capturedirect satellite broadcast to different places
• + Mezzo TV in SD (in 39 countries)audio part: Orange Labs + Radio-France
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 33 r&d legal direction France Telecom Group
Don Giovanni
Technical aspectsHOA sphere positioning trials 5.0 provided to Radio-Francereal time mixing with 4 dozens of spot microphones artificial reverb added (even on "ambience HOA 5.0")⇒ no longer "HOA" spatial model
Resultsgreat success (artistic, technical, popular)very nice and appreciated sound ! … but not really faithful wrt the actual theatre ;-)great communication impact for HOA !
• (websearch: HOA + "Don Giovanni")
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 34 r&d legal direction France Telecom Group
Some lessons
Mistrust monitoring over another setup:eg 8.0 vs 5.0 ⇒ front-back instability+ influence of the reproduction room (and ldspk, and array size), esp. with particular sounds (applauses, broadband signals, etc.) or recorded scene (very large, etc.) : dry, spherical vs hemi-spherical, spherical vs horizontal ⇒ projection of elevated sourcebinaural vs loudspeaker presentation ⇒ wave interferences, potential coloration and phasing
Shall content creation and recording be generic, or specific to a target setup?Use HOA as end-to-end approach or intermediary toolset?
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 35 r&d legal direction France Telecom Group
Conclusion
further improvements of the technologic part still expectedsystem transparency (spatial decoding)3D scene analysis and manipulation: spatial editing tools (benefiting from new ext. dev.: mic array, et. )coding / compression, format conversion
get lessons from experimentsadapt HOA to sound engineers practice… then reciprocally!
• improve ergonomics• HOA as end-to-end techno and format? Or as a toolkit?
issues regarding: • spatial organisation, • mic positioning, etc.• generic of specific content production
Ambisonics Symposium/2009-06-25/Jérôme Daniel – p 36 r&d legal direction France Telecom Group
Thank you for your attention