CSE 380 – Computer Game Programming Audio Engineering

Audio Engineering For games: –Sound Effects and Music Sometimes overlooked If not done right, can ruin a game If done right, enhances experience –seamless with gameplay –not too obvious

What do you need sound effects for? Discernable Game Events –Collisions tied to collision response –Movement footfalls, rockets, tied to sprite state –Ambiance background noise tied to game state –Gameplay Events tied to victory, area triggers, power-ups, death, etc to help give your game meaningful play

Recording & Exporting Sounds Lots of tools to use Recommendation: Audacity –free –easy to learn to use –exports to many formats Ex: mp3 –easy to mix sounds to make complex sound effects –

Sound Properties Frequency –measures speed of sound vibrations –measured in cycles per second (Hz) –audible range for human ear: 20Hz – 20,000Hz Intensity –amount of air molecules pushed around by sound –pressure applied by sound wave –measured in decibels (dB), a log scale sound of whispering 5ft. Away: 20 dB sound of thunder: 120dB

Sound File Formats WAV –stores raw sound data, is the largest format –easy on CPU MP3/M4A, OGG, & WMA –compressed formats ~10:1 compression over WAV –negligible quality difference from WAV –decompression CPU hit MIDI –programmed sound

Long vs. Short sound files For short sound effects: –store as MP3/OGG/etc. –decompress data in memory at load time –play as needed in decompressed format –heavy on memory, light on CPU For long sound effects or music: –store as MP3/OGG/etc. –play in compressed format –again, heavy on CPU usage, light on memory

Music What role does music play? –emotion –recognition –themes –fill the void Sound & Music: –should compliment the gameplay, not overwhelm it

For the programmer Lots of libraries Keep game code platform independent Options: –FMod and tools (FMod Designer, etc.) –DirectSound (deprecated) –XAudio/XACT (soon to be deprecated –XAudio2/XACT3 (soon part of Windows) See DirectX Documentation for C++ –lots of other libraries as well (Miles, WWise, etc.)

Microsoft Audio Technologies

For us? XACT3 –designer controlled content –easy audio tweaking XAudio2 –API to build sound engine

What’s so great about that? The audio designer can focus on audio Easy to: –test many different sound effects & songs –choose the right sounds & music –alter/improve sounds & music All without having to bother a programmer

XBox CPU cores –each with 2 hardware threads –Common approach: run XAudio2 and XACT on their own threads Has real-time XMA decoder hardware –XMA is XBox format for WMA –Windows uses ADPCM Windows is more complicated –How many cores? GetLogicalProcessorInformation

XAudio Tools/APIs XACT XACT Command-Line Tool AdpcmEncode Wave Merge Tool XMA Command-Line Encoder XMA2 Command-Line Encoder xWMA Command-Line Encoder X3DAudio XAPO XAPOFX XMAEncoder

My Recommendation Each team pick an audio engineer for Benchmark 3 That person will become the XACT3/XAudio2 expert –start with the DirectX Audio manual The other team members can continue to concentrate on gameplay Go to DirectX install directory Go to Samples/

XACT Terminology Start by reading about: –wave –wave bank –sound –sound bank –track –events –cues

IXAudio2 Core of the engine –enumerate available audio devices –configure properties –create & destroy voices –monitor performance It’s a COM object –create after CoInitializeEx One can create multiple instances –each operates in own thread

Voices Used to process, manipulate, and play audio data –Source Voices used to stream audio data –Submix Voices manipulate audio data –Mastering Voices send data to audio hardware

Audio Graph A collection of voices

To Play a Sound 1.Initialize XAudio2 2.Load Audio Data –init a WAVEFORMATEX & XAUDIO2_BUFFER 3.Create a Source Voice 4.Pass data to the voice 5.Start the voice

For the Designer Determine all the sound effects for your game Produce such a list right now This is a to-do list of recordings to make

A very good reference to start with

References Programming with DirectX : Sound in DirectX – XAudio 2 – Game Coding Complete, 4 th Edition –McShaffry/Graham DirectX Software Development Kit Documentation