1. Audio Concepts in Plain English: 3D and I3DL2 Scott Selfon Xbox Advanced Technology Group

2. Overview What’s so great about “3D”? Simulating aspects of 3D sound Audio engine implementations I3DL2 Advanced 3D Challenges for Games Q & A

3. Why bother with “3D” audio? Screen = 45-90 degree visual ‘window’ –What’s happening in the other 270-315 degrees? Sound informs player of distances, obstacles, environment Competing with TV & movie ‘experience’

4. Typical 3D Sound Presentation Priorities Location Distance Speed Geometry –Sound emitter orientation –World obstacles

5. Locating a Sound in 3D Space Speaker panning –Cheap, can be effective –Doesn’t cover surround for 2-speaker scenarios HRTF – Head Related Transfer Function –Simulates 3D spatialization based on shapes of ‘generic’ head and ears –Filtering (shape of ears, sound waves not directly reaching ears) –Delays (sound reaches one ear slightly before the other

6. Distance in the Real World Sounds attenuate (get quieter) with distance –Approximately half as loud (-6 dB) for every doubling of distance Sounds “muffled” with distance –Air is a low-pass filter Sound travels slowly vs. light –~340 m/sec (vs. 300,000 km/sec for light) All values vary with humidity, altitude, temperature, etc. Dist.Vol. 10 dB 2-6 dB 4-12 dB 8-18 dB 16-24 dB 32-30 dB 64-36 dB 128-42 dB

7. Distance Simulation: Volume Typical distance model (DSound, OpenAL): –Minimum distance (where to start attenuating) –Maximum distance (where to stop attenuating) –Rolloff (how quickly to attenuate with distance) Others simplify to: –Minimum distance (where to start attenuating) –Distance to silence (where sound is silent; rolloff implied from here – linear or log)

8. Distance Attenuation Example

9. Distance and Rolloff Rolloff factor – adjust attenuation curve relative to “Real World” –.5: Less rolloff (sound louder at given distance) –2.0: More rolloff (sound quieter at given distance) Rolloff also influenced by minimum distance –Remember, half volume at doubled distance

10. Rolloff Factor and Attenuation

11. Minimum Distance and Attenuation

12. Rolloff Confusion Maximum distance isn’t silence –Sound not further attenuated –Might abruptly stop voice (PC) Distance factor unrelated to rolloff –Sets units (meters [1.0], feet [0.3048], etc.) –Used for doppler shifting

13. Rolloff Model Challenges Unintuitive calculations to determine distance to silence –spreadsheet to help Hard for sound designer to experiment without game-exposed controls Simplification options –Replace max dist/rolloff with “distance to silence” –Arbitrary rolloff curves

14. Simplifying Rolloff Distance to silence –Trivial to calculate Arbitrary user- defined point curves Custom curvature control

15. Simulating Other Distance Cues Filtering with distance –EAX supports –Programmable LPFs in other engines can be used for this Delay with distance (speed of sound) –Rarely simulated by engines Can programmatically delay sound triggers Perceived in games as ‘lag’?

16. Speed Perception in the Real World Doppler Effect –Sound/listener moving closer pitched up –Sound/listener moving apart pitched down

17. Doppler Effect Simulation Programmer updates listener/source velocities –DSound: Relative velocity used –OpenAL: Separate velocities used Distance factor –Units do matter here (m/sec) Doppler factor –Exaggerate (>1.0) or lessen (<1.0) effect

18. Sound Geometry in the Real World Few sounds are truly omnidirectional –Most volume projected in one direction Few sounds are truly “point” sources Sounds interact with the environment –Environment interaction (“reverb”) –Filtered by obstacles

19. Sound Geometry Simulation Directional sound emitters –Inside/outside sound cone angles –Outer volume attenuation –3D spatial orientation for emitter Non-point emitter solutions –Simulate with multiple points –Simulate with relative positioning –Pre-rendered multichannel –More sophisticated engines

20. Environmental Simulation Simulation of sound propagation paths –Direct-path = signal reaches listener directly from emitter –Reflected paths = ‘echoes’ off walls, obstacles, etc. Some delay versus direct-path

21. Basic Reverb Simulation Direct-path: sound presented “intact” –Distance attenuation –Low pass filtering based on obstacles/distance Reflected path: delays and filtering –Often less noticeable distance attenuation –Early reflections = discrete echoes –Late reflections = ‘reverberation’ (more dense, complex, decaying echoes)

22. Obstacle Simulation: Obstruction Objects may ‘block’ the sound’s path to the listener –Direct-path diffracted by or transmitted through obstacle –Reflected path generally not affected

23. Obstacle Simulation: Occlusion Sound is being played in another environment –Direct, reflected paths both filtered

24. An Introduction to I3DL2 Designed by Interactive Audio Special Interest Group (IASIG – www.iasig.org) –“Interactive 3D Audio Rendering Guidelines, Level 2” Provides standard for simulating: –3D positioning (attenuation, doppler) –Environmental reverberation –Occlusion/obstruction materials

25. I3DL2 Concepts Listener = describes environment –Timings for early/late reflections (“Room”) –Differences between high and low frequency behavior Source = per-emitter settings –Occlusion/obstruction –Exaggerate/reduce environment’s effect on this emitter

26. I3DL2 Environmental Reverb (“Listener”) Decay time = time for reverb to fade to -60 dB

27. I3DL2 Reverb Settings 30 pre-defined high level presets… –Generic environments (cave, bathroom, etc.) –‘music’ reverbs …or 12 low-level properties –Adjust timings: Reflections_delay, Reverb_delay, Decay_time –Set volumes: Room, Reflections, Reverb, Room_rolloff_factor –Adjust behavior of high vs. low frequencies: HF_reference, Room_HF, Decay_HF_ratio –Reverb timbre: Diffusion (“graininess”), Density (“hollowness”)

28. I3DL2 Sound Emitters (“Source”) Volume adjustment –Direct-path level: Direct, Direct_HF –Room (reverb) rolloff factor and level: Room_rolloff_factor, Room, Room_HF Occlusion/obstruction control –8 presets supplied (doors, walls, etc.) –Volume attenuation: Obstruction, Obstruction_LF_ratio, Occlusion, Occlusion_LF_ratio

29. I3DL2 Reverb and Occlusion/Obstruction Demos

30. I3DL2 Limitations Reverberation flexibility restrictions –I3DL2 doesn’t handle multiple simultaneous environments (listener only) –Can be expensive (CPU, memory) –“Morphable” reverb settings are hard No sense of game geometry –Game must test for and set occlusion/obstruction –Early reverb reflections are static, non-directional –Reverb transitions

31. Advanced 3D Challenges Multiple players, one set of speakers –Present sound relative to closest listener –Reproduce sound for all listeners Doppler issues Overlayed soundscapes Speaker biasing? (P1=left, P2=right) Audio for complex geometry –Rivers –Non-cubical environments

32. Advanced 3D Challenges 3D integration with multichannel audio –4- vs. 5- channel 3D? –Morphing 4.0/5.1 sounds to point sources? Multiple 3D sources presenting same audio (PA system?) –Sync and Doppler challenges More sophisticated sound propagation simulation –Complex, distant environments (pathfinding?) –AI response to sound propogation (enemies can hear you)

33. Q & A Resources –OpenAL: http://www.openal.org/http://www.openal.org/ –DirectSound: http://msdn.microsoft.com/http://msdn.microsoft.com/ –I3DL2, IA-SIG: http://www.iasig.org/http://www.iasig.org/ –http://www.midi.org/http://www.midi.org/ Audio physics resources –http://hyperphysics.phy- astr.gsu.edu/hbase/sound/soucon.htmlhttp://hyperphysics.phy- astr.gsu.edu/hbase/sound/soucon.html –http://www.gmi.edu/~drussell/Demos/doppler/doppler. htmlhttp://www.gmi.edu/~drussell/Demos/doppler/doppler. html Questions, comments?