D1 - 29/05/2014 France Télécom Recherche & Développement Workshop « From 5.1 to Sound Field Synthesis..." AES 120th Convention, Paris 2006 Higher Order Ambisonics: promises and reality Jérôme Daniel, France Telecom R&D

#2 Traditional 1st order Ambisonics: B-Format encoding Panoramic sound recording Coincident omni (W) and bidirectional (X,Y) microphones Front-back, Left-Right separation Directional information = amplitude relationships Description of wave propagation direction & speed localization Independent of any loudspeaker layout Front (X) Back Left (Y) Right

#3 Reproduction over loudspeakers : spatial decoding Simulate any coincident mic setup Recombine B-Format directivity patterns Decoding operation: matrix signals W,X,Y One virtual microphone per loudspeaker... as many as wanted, but... … sound image blur remains the same + - = + = B-Format Front (X) Back Left (Y) Right

#4 Reproduction over loudspeakers : spatial decoding Simulate any coincident mic setup Recombine B-Format directivity patterns Decoding operation: matrix signals W,X,Y One virtual microphone per loudspeaker... as many as wanted, but... … sound image blur remains the same Optimized decoding for localization (LF < z) Reproduce true wave propagation at the listener scale ( good ITD) HF (> Hz) Concentrate energy contributions in the expected direction ( less altered ILD, ITD) Front (X) Back Left (Y) Right minimise opposite contributions Compromise for large area [Malham] Optimize localization at the sweet spot [Gerzon]

#5 "Traditional" 1st order Ambisonics: pros & cons Pros Compact multichannel format (no redundancy) Spatial homogeneity Acoustic fidelity (regarding propagation properties) Easily extended to 3D (additional Z) Flexibility: sound field transformation; reproduction setups Commercialized B-Format microphones (eg SoundField) Cons Blurred / unstable sound images ("tiny" sweet spot) Not well adapted to irregular/unbalanced loudspeaker arrangements (esp. ITU setup) Limitations due to low directivity of usual mikes, esp. at LF... thats why non-coincident microphone approaches might be preferred

#6 Introducing Higher Order Ambisonics (HOA) Increase angular discrimination in spatial encoding add directivities with "faster" angular variation Front (X) Back Left (Y) Right 1st order 2nd order3rd order4th order

#7 Introducing Higher Order Ambisonics (HOA) Increase angular discrimination in spatial encoding add directivities with "faster" angular variation Increase angular selectivity of loudspeakers contributions selective virtual microphone directivities better use of narrowed loudspeakers Front (X) Back Left (Y) Right ++++ = = ==

#8 Introducing Higher Order Ambisonics (HOA) Increase angular discrimination in spatial encoding add directivities with "faster" angular variation Increase angular selectivity of loudspeakers contributions selective virtual microphone directivities better use of narrowed loudspeakers Front (X) Back Left (Y) Right 1st order2nd order 3rd order4th order

#9 Rendering properties of higher spatial resolution Acoustic reconstruction Enlarged sweet area "Holophony" [Nicol, Daniel] Enhanced distance encoding control of the wave curvature monochromatic plane wave (f=600Hz) 1st order 2 nd order 5th order10th order Quality of sound images: localization clues for a centred listener spherical wave (R=1m) (gaussian pulse) Order M1234 f lim 700 Hz1300 Hz1900 Hz2500 Hz E 45°30°22.5°18° good reconstruction ( good ITD) up to f lim blur angle due to HF clues alteration (ILD&ITD) above f lim

#10 Compatibility with irregular/unbalanced arrangements Synthesize directivities adapted to ITU inter-loudspeaker angles From 4th order ambisonics [Craven, 2003] Using 5th order resolution [Laborie et al]: better front channels separation Possible decoding criterion (among others): imitate pair-wise pan-pot [Craven, 2003] [Laborie et al]

#11 Compatibility with irregular/unbalanced arrangements Synthesize directivities adapted to ITU inter-loudspeaker angles From 4th order ambisonics [Craven, 2003] Using 5th order resolution [Laborie et al]: better front channels separation Possible decoding criterion (among others): imitate pair-wise pan-pot 4th order decoding over enriched ITU setup (5+2+1) C (0°), L&R(+-30°), S L &S R (+-120°) … + L&R(+-70°) … + B (180°) Demonstration on a 8-loudspeaker setup (kindly provided by Cabasse) = "energy vector" (* = target, ie ideal sound image)

#12 Extension to 3D encoding and reproduction 3D encoding and decoding Dynamic binaural reproduction Virtual loudspeakers doesnt sound so good Enhanced method: better efficiency (CPU) & rendering Sound field rotation driven by head-tracker Demo : Poster session P31, Tuesday, 14: :30 Encoding into 3D HOA Format Reproduction over a 3D rig Reproduction over headphones Spatial decoding (similar to 2D) Head-tracker Virtualization: HRTF filtering K N LdSpksignals K HOAsignals Sound Field Rotation

#13 First conclusion on Higher Order Ambisonics Pros Scalable multichannel format Spatial homogeneity Acoustic fidelity + "high spatial definition" Wave field reconstruction Easily extended to 3D – Efficient binaural spatialisation Even more flexibility: sound field transformation; reproduction setups, including irregular arrangements like ITU Cons nothing? What do we need in practice? HOA (or « high spatial resolution ») microphone systems Spatial processing tools

#14 Higher Order Ambisonics Microphone Systems Synthesis of Spherical Harmonics Extension of differential microphones: Pressure gradient and higher order derivatives using non-coincident acoustic sensors! Non concentric sensor distribution (Trinnov) Distribution over a rigid sphere (FT) [Meyer, Elko, Kubli] [Rafaely] [Ward, Abhayapala]… Trade-off on the size of the array –bigger is better to have spatial resolution at LF –smaller is better to reduce spatial aliasing (at HF) A few words on FT prototype Designed for "proof of concept" (homogeneous 3D) 32 sensors 4th order 3D (and even 5th order 2D) Objective measurements & validation [Moreau et al] Poster session P31, Tuesday, 14: :30

#15 Tools and applications Recording and mixing tools Prototypes of HOA mic (FT, Trinnov) Suite of VST plug-ins demo Use in common audio edition tools, or interactive audio progr. Applications Music, documentary, fictions Sharing of events/ambiances (eg familial use), teleconferences Interactive audio and multi-media: –A flexible multi-channel 3D audio format –Games, Virtual/Mixt Reality –News nodes for virtual scene description in MPEG4 (AudioBIFSV3) –label a multi-channel stream as a HOA content (AudioChannelConfig) –a new kind of sound object that describes a Surrounding Sound Field (SurroundingSound)

#16 Demonstrations Loudspeaker reproduction Reproduction of 4th order 3D recordings over enriched ITU setup (5 to 8 ldspk) Acknowledment: Many thanks to Cabasse and R&D manager Yvon Kernéis Head-tracked binaural reproduction [Moreau et al] Poster session P31, Tuesday, 14: :30 Could also be shown after this workshop