1. CSE 380 – Computer Game Programming Audio Engineering

2. Audio Engineering For games: –Sound Effects and Music Sometimes overlooked If not done right, can ruin a game If done right, enhances experience –seamless with gameplay –not too obvious

3. What do you need sound effects for? Discernable Game Events –Collisions tied to collision response –Movement footfalls, rockets, tied to sprite state –Ambiance background noise tied to game state –Gameplay Events tied to victory, area triggers, power-ups, death, etc to help give your game meaningful play

4. Recording & Exporting Sounds Lots of tools to use Recommendation: Audacity –free –easy to learn to use –exports to many formats Ex: mp3 –easy to mix sounds to make complex sound effects –http://audacity.sourceforge.net/http://audacity.sourceforge.net/

5. Sound Properties Frequency –measures speed of sound vibrations –measured in cycles per second (Hz) –audible range for human ear: 20Hz – 20,000Hz Intensity –amount of air molecules pushed around by sound –pressure applied by sound wave –measured in decibels (dB), a log scale sound of whispering 5ft. Away: 20 dB sound of thunder: 120dB

6. Sound File Formats WAV –stores raw sound data, is the largest format –easy on CPU MP3/M4A, OGG, & WMA –compressed formats ~10:1 compression over WAV –negligible quality difference from WAV –decompression CPU hit MIDI –programmed sound

7. Long vs. Short sound files For short sound effects: –store as MP3/OGG/etc. –decompress data in memory at load time –play as needed in decompressed format –heavy on memory, light on CPU For long sound effects or music: –store as MP3/OGG/etc. –play in compressed format –again, heavy on CPU usage, light on memory

8. Music What role does music play? –emotion –recognition –themes –fill the void Sound & Music: –should compliment the gameplay, not overwhelm it

9. For the programmer Lots of libraries Keep game code platform independent Options: –FMod and tools (FMod Designer, etc.) –DirectSound (deprecated) –XAudio/XACT (soon to be deprecated –XAudio2/XACT3 (soon part of Windows) See DirectX Documentation for C++ –lots of other libraries as well (Miles, WWise, etc.)

10. Microsoft Audio Technologies

11. For us? XACT3 –designer controlled content –easy audio tweaking XAudio2 –API to build sound engine

12. What’s so great about that? The audio designer can focus on audio Easy to: –test many different sound effects & songs –choose the right sounds & music –alter/improve sounds & music All without having to bother a programmer

13. XBox 360 3 CPU cores –each with 2 hardware threads –Common approach: run XAudio2 and XACT on their own threads Has real-time XMA decoder hardware –XMA is XBox format for WMA –Windows uses ADPCM Windows is more complicated –How many cores? GetLogicalProcessorInformation

14. XAudio Tools/APIs XACT XACT Command-Line Tool AdpcmEncode Wave Merge Tool XMA Command-Line Encoder XMA2 Command-Line Encoder xWMA Command-Line Encoder X3DAudio XAPO XAPOFX XMAEncoder

15. My Recommendation Each team pick an audio engineer for Benchmark 3 That person will become the XACT3/XAudio2 expert –start with the DirectX Audio manual The other team members can continue to concentrate on gameplay Go to DirectX install directory Go to Samples/

16. XACT Terminology Start by reading about: –wave –wave bank –sound –sound bank –track –events –cues

17. IXAudio2 Core of the engine –enumerate available audio devices –configure properties –create & destroy voices –monitor performance It’s a COM object –create after CoInitializeEx One can create multiple instances –each operates in own thread

18. Voices Used to process, manipulate, and play audio data –Source Voices used to stream audio data –Submix Voices manipulate audio data –Mastering Voices send data to audio hardware

19. Audio Graph A collection of voices

20. To Play a Sound 1.Initialize XAudio2 2.Load Audio Data –init a WAVEFORMATEX & XAUDIO2_BUFFER 3.Create a Source Voice 4.Pass data to the voice 5.Start the voice

21. For the Designer Determine all the sound effects for your game Produce such a list right now This is a to-do list of recordings to make

22. A very good reference to start with http://www.ultimategameprogramming.com/BeginningDirectX11%5C58958_App_B_rev01_lores.pdf

23. References Programming with DirectX : Sound in DirectX – XAudio 2 –http://programming4.us/multimedia/3830.aspxhttp://programming4.us/multimedia/3830.aspx Game Coding Complete, 4 th Edition –McShaffry/Graham DirectX Software Development Kit Documentation