Audio Concepts in Plain English: 3D and I3DL2 Scott Selfon Xbox Advanced Technology Group

Overview What’s so great about “3D”? Simulating aspects of 3D sound Audio engine implementations I3DL2 Advanced 3D Challenges for Games Q & A

Why bother with “3D” audio? Screen = degree visual ‘window’ –What’s happening in the other degrees? Sound informs player of distances, obstacles, environment Competing with TV & movie ‘experience’

Typical 3D Sound Presentation Priorities Location Distance Speed Geometry –Sound emitter orientation –World obstacles

Locating a Sound in 3D Space Speaker panning –Cheap, can be effective –Doesn’t cover surround for 2-speaker scenarios HRTF – Head Related Transfer Function –Simulates 3D spatialization based on shapes of ‘generic’ head and ears –Filtering (shape of ears, sound waves not directly reaching ears) –Delays (sound reaches one ear slightly before the other

Distance in the Real World Sounds attenuate (get quieter) with distance –Approximately half as loud (-6 dB) for every doubling of distance Sounds “muffled” with distance –Air is a low-pass filter Sound travels slowly vs. light –~340 m/sec (vs. 300,000 km/sec for light) All values vary with humidity, altitude, temperature, etc. Dist.Vol. 10 dB 2-6 dB 4-12 dB 8-18 dB dB dB dB dB

Distance Simulation: Volume Typical distance model (DSound, OpenAL): –Minimum distance (where to start attenuating) –Maximum distance (where to stop attenuating) –Rolloff (how quickly to attenuate with distance) Others simplify to: –Minimum distance (where to start attenuating) –Distance to silence (where sound is silent; rolloff implied from here – linear or log)

Distance Attenuation Example

Distance and Rolloff Rolloff factor – adjust attenuation curve relative to “Real World” –.5: Less rolloff (sound louder at given distance) –2.0: More rolloff (sound quieter at given distance) Rolloff also influenced by minimum distance –Remember, half volume at doubled distance

Rolloff Factor and Attenuation

Minimum Distance and Attenuation

Rolloff Confusion Maximum distance isn’t silence –Sound not further attenuated –Might abruptly stop voice (PC) Distance factor unrelated to rolloff –Sets units (meters [1.0], feet [0.3048], etc.) –Used for doppler shifting

Rolloff Model Challenges Unintuitive calculations to determine distance to silence –spreadsheet to help Hard for sound designer to experiment without game-exposed controls Simplification options –Replace max dist/rolloff with “distance to silence” –Arbitrary rolloff curves

Simplifying Rolloff Distance to silence –Trivial to calculate Arbitrary user- defined point curves Custom curvature control

Simulating Other Distance Cues Filtering with distance –EAX supports –Programmable LPFs in other engines can be used for this Delay with distance (speed of sound) –Rarely simulated by engines Can programmatically delay sound triggers Perceived in games as ‘lag’?

Speed Perception in the Real World Doppler Effect –Sound/listener moving closer pitched up –Sound/listener moving apart pitched down

Doppler Effect Simulation Programmer updates listener/source velocities –DSound: Relative velocity used –OpenAL: Separate velocities used Distance factor –Units do matter here (m/sec) Doppler factor –Exaggerate (>1.0) or lessen (<1.0) effect

Sound Geometry in the Real World Few sounds are truly omnidirectional –Most volume projected in one direction Few sounds are truly “point” sources Sounds interact with the environment –Environment interaction (“reverb”) –Filtered by obstacles

Sound Geometry Simulation Directional sound emitters –Inside/outside sound cone angles –Outer volume attenuation –3D spatial orientation for emitter Non-point emitter solutions –Simulate with multiple points –Simulate with relative positioning –Pre-rendered multichannel –More sophisticated engines

Environmental Simulation Simulation of sound propagation paths –Direct-path = signal reaches listener directly from emitter –Reflected paths = ‘echoes’ off walls, obstacles, etc. Some delay versus direct-path

Basic Reverb Simulation Direct-path: sound presented “intact” –Distance attenuation –Low pass filtering based on obstacles/distance Reflected path: delays and filtering –Often less noticeable distance attenuation –Early reflections = discrete echoes –Late reflections = ‘reverberation’ (more dense, complex, decaying echoes)

Obstacle Simulation: Obstruction Objects may ‘block’ the sound’s path to the listener –Direct-path diffracted by or transmitted through obstacle –Reflected path generally not affected

Obstacle Simulation: Occlusion Sound is being played in another environment –Direct, reflected paths both filtered

An Introduction to I3DL2 Designed by Interactive Audio Special Interest Group (IASIG – –“Interactive 3D Audio Rendering Guidelines, Level 2” Provides standard for simulating: –3D positioning (attenuation, doppler) –Environmental reverberation –Occlusion/obstruction materials

I3DL2 Concepts Listener = describes environment –Timings for early/late reflections (“Room”) –Differences between high and low frequency behavior Source = per-emitter settings –Occlusion/obstruction –Exaggerate/reduce environment’s effect on this emitter

I3DL2 Environmental Reverb (“Listener”) Decay time = time for reverb to fade to -60 dB

I3DL2 Reverb Settings 30 pre-defined high level presets… –Generic environments (cave, bathroom, etc.) –‘music’ reverbs …or 12 low-level properties –Adjust timings: Reflections_delay, Reverb_delay, Decay_time –Set volumes: Room, Reflections, Reverb, Room_rolloff_factor –Adjust behavior of high vs. low frequencies: HF_reference, Room_HF, Decay_HF_ratio –Reverb timbre: Diffusion (“graininess”), Density (“hollowness”)

I3DL2 Sound Emitters (“Source”) Volume adjustment –Direct-path level: Direct, Direct_HF –Room (reverb) rolloff factor and level: Room_rolloff_factor, Room, Room_HF Occlusion/obstruction control –8 presets supplied (doors, walls, etc.) –Volume attenuation: Obstruction, Obstruction_LF_ratio, Occlusion, Occlusion_LF_ratio

I3DL2 Reverb and Occlusion/Obstruction Demos

I3DL2 Limitations Reverberation flexibility restrictions –I3DL2 doesn’t handle multiple simultaneous environments (listener only) –Can be expensive (CPU, memory) –“Morphable” reverb settings are hard No sense of game geometry –Game must test for and set occlusion/obstruction –Early reverb reflections are static, non-directional –Reverb transitions

Advanced 3D Challenges Multiple players, one set of speakers –Present sound relative to closest listener –Reproduce sound for all listeners Doppler issues Overlayed soundscapes Speaker biasing? (P1=left, P2=right) Audio for complex geometry –Rivers –Non-cubical environments

Advanced 3D Challenges 3D integration with multichannel audio –4- vs. 5- channel 3D? –Morphing 4.0/5.1 sounds to point sources? Multiple 3D sources presenting same audio (PA system?) –Sync and Doppler challenges More sophisticated sound propagation simulation –Complex, distant environments (pathfinding?) –AI response to sound propogation (enemies can hear you)

Q & A Resources –OpenAL: –DirectSound: –I3DL2, IA-SIG: – Audio physics resources – astr.gsu.edu/hbase/sound/soucon.htmlhttp://hyperphysics.phy- astr.gsu.edu/hbase/sound/soucon.html – htmlhttp:// html Questions, comments?