Use of sound within games

Question Clinic: FAQ In lecture exploration of answers to frequently asked student questions

Sound in Games Overview of the different uses of sound within games

Use of sound in game The use of sound within games includes: Music - establishing scene mood, e.g. uplifting, urgent, peaceful, etc. Sound effects - dependent sounds tied to in-game events, e.g. collecting power-ups, jumping, etc. Dialog – recorded/generated speech used within cut-scenes (furthering plot) or in-game (improving realism, emersion, etc.).

The importance of sound Sound plays a very important role in setting the mood for the game as well as, increasingly, how engaging the game is. For example: A pinball game without any sound (ball hits, accumulating score) will appear dull and lifeless (the player will feel more detached). In a 3D game, character movement without footfall sounds will reduce the believability/emersion (consciously, or not, the player will feel movement is not ‘right’).

Sound Management Basic principles behind managing sound playback

Output channels Sound hardware typically supports a maximum number of audio output channels (simultaneous sounds). Additionally, sound APIs may support a maximum number of output channels. The number of channels defines the maximum number of simultaneously playing sounds. Aside: “The Java Sound API provides support for the capture, processing, and rendering of audio and MIDI data. Java Sound … is a 16-bit, 32-channel audio rendering and MIDI controlled sound synthesis engine.”

Output channels (a limited resource? Example 1) Constraint: You cannot have more individual sounds playing than you have channels. Actually, 32+ channels is ample for most games… Background music People talking Water sounds Foot falls

Output channels (a limited resource? Example 2) What happens if every projectile from a ship is to make a sound when it hits something? This could be a problem…

Dealing with a large number of concurrent sounds… Question: Given a finite number of channels, how can we deal with a potentially large number of simultaneous sounds? Start 2 mins 3 mins 30 sec Finished 4 mins 1 min 6 mins 10 mins 9 mins 8 mins 7 mins 5 mins

Dealing with a large number of concurrent sounds… Approach 1: Allocate a maximum number of sound channels that can be used for a particular type of sound. Develop a manager that dynamically monitors channel usage (imposing sound type channel restriction if all channels are in use) Total 32 channels Explosions (max 6 channels) Issues: When the playback limit is reached, how are new requests handled? Are they ignored? Is playback of the longest playing sound in that class cut short? If other classes have exceeded their maximum, which one will contract? Projectile hits (max 10 channels)

Aside: Approach 1 and 2 can be successfully combined together. Dealing with a large number of concurrent sounds… Approach 2: As sound waveforms are additive, software/hardware can be used to combine separate waveforms to be then played via a single output channel. This effectively provides an unlimited number of channels, but with the additional cost of combining the individual sounds. Aside: Approach 1 and 2 can be successfully combined together.

Sound in Java Overview of sound usage in Java

Sound in Java Sound playback is achieved using the Java sound API, using: SourceDataLine – referring to a source of data for a mixer (i.e. output from a program) TargetDataLine – referring to a source of output from a mixer Clip – a loaded (not streamed) sound file. LineListener –attached to lines to monitor LineEvents (i.e. audio opened, closed, started, or stopped)

To do: Explore this Sound in Java See the following classes within the core code repository: ● SoundAsset ● SoundAssetClip ● SoundAssetAssembly

Sound in XNA Overview of sound usage in XNA

Sound in XNA (XAct) XNA audio is created using the Cross- Platform Audio Creation Tool (XACT). XACT supports WAV, AIFF, XMA formats and stero or 5.1 output XACT provides excellent audio integration features and supports streaming support and audio event notification Aside: To play MP3s (and similar formats) it will be necessary to convert them into a WAV format). This applies to XNA and Java

Sound in XNA (XAct) Audio within XACT is structured as follows: To do: Explore this Sound in XNA (XAct) Audio within XACT is structured as follows: Wave Banks – a collection of wave files Sound Banks – instructions and cues for the playback of sounds in a wave bank Cues – triggering audio playback from a sound bank. A cue can contain a sequence of ordered sounds to play, or a list from which to select a random sound to play. audioEngine = new AudioEngine(audioParameters); waveBank = new WaveBank(audioEngine, @“Assets\WaveBank.xwb"); soundBank = new SoundBank(audioEngine, @“Assets\SoundBank.xsb"); soundBank.PlayCue(Name); Cue cue = soundBank.GetCue(Name); cue.Play();

Summary Today we explored: Overview of the use of sound within games and how sounds can be managed Overview of Java/XNA sound classes To do: Think about the role of sound in your game (how important do you want it to be?) Continue developing visual/aural design of game Continue writing code to display graphics or play sound