178. Game Design, Development, and Technology
CS 382
Game Design, Development, and Technology
Audio Programming
Game Music Varieties
Musical Transitions
Game Audio History
MIDI and DLS
Environmental Geometry
Part 5.5

179. Game Music Varieties Opening Sequence (Main Title)
Designed to build excitement and set the mood for the game.
Closing Credits Sequence
Intended to reinforce the feeling of accomplishment at completing the game.
Cinematic Sequence
Film scoring to establish a pace, highlight plot shifts, and adjust tension and excitement.
Menu Screen
Maintain immersion in the virtual world without annoying the player.
Part 5.5
Audio Programming
Page 179

180. Musical Transitions
Moving smoothly from one musical cue to another in a nonlinear game environment is tricky, but necessary to maintain the thread of gameplay.
0:00
Combat #2
1:22
1:30
Sub-Ambient
2:14
2:16
Suspense
3:10
3:12
Ambient
4:08
4:12
Combat #1
5:33
5:42
Silence
Techniques for achieving transitions:
Cue-to-cue transitions on musical boundaries
When a new cue is called, the current cue plays until the next "boundary" (e.g., the next measure), and then the new cue begins. Advantage: simplicity; disadvantage: may sound sudden, or stilted.
Layering
Add and subtract musical instrument layers to build and diminish the score. Advantage: continuity; disadvantage: difficult to move quickly to a completely different cue.
Transition matrix
the game can select an appropriate transition between any pair of music cues. Advantage: custom transitions can be created for specific situations, resulting in dramatic, yet natural, sounding motion from cue to cue; disadvantage: composers must think about their music in a completely non-linear manner.
Part 5.5
Audio Programming
Page 180

181. Game Audio History Streaming Pros & Cons Sequencing Pros & Cons
Sound and music programming was once relegated to last-minute afterthought status in game development, but it has become increasingly viewed as a determining factor to a game’s success.
In general, music in games is either played back as a stream of sample data or as a sequence of instructions on an audio synthesizer.
Streaming Pros & Cons
Minimal hardware requirements
Simple integration with games
Quality limited only by sample rate and number of channels
Latency at start and stop of stream
Large memory requirements or decompression resources
Sequencing Pros & Cons
No latency problems
Many musical styles can be produced from a small data set
Music can be altered at runtime to create unique game interactions
Music and other audio voices may compete
Complex integration with games
Part 5.5
Audio Programming
Page 181

182. Musical Instrument Digital Interface (MIDI)
PIANO
ORGAN
BASS
ENSEMBLE
REED
SYNTH LEAD
SYNTH EFFECTS
PERCUSSIVE 1
Acoustic Grand 17
Drawbar Organ 33
Acoustic Bass 49
String Ensemble 1 65
Soprano Sax 81
Lead 1 (square) 97
FX 1
(rain)
113
Tinkle Bell 2
Bright Acoustic 18
Percussive Organ 34
Electric Bass (finger) 50
String Ensemble 2 66
Alto Sax 82
Lead 2 (sawtooth) 98
FX 2 (soundtrack)
114
Agogo 3
Electric Grand 19
Rock Organ 35
Electric Bass (pick) 51
Synth Strings 1 67
Tenor Sax 83
Lead 3 (calliope) 99
FX 3 (crystal)
115
Steel Drums 4
Honky-Tonk 20
Church Organ 36
Fretless Bass 52
Synth Strings 2 68
Baritone Sax 84
Lead 4
(chiff)
100
FX 4 (atmo-sphere)
116
Woodblock 5
Electric Piano 1 21
Reed Organ 37
Slap Bass 1 53
Choir Aahs 69
Oboe 85
Lead 5 (charang)
101
FX 5 (brightness)
117
Taiko Drum 6
Electric Piano 2 22
Accordian 38
Slap Bass 2 54
Voice Oohs 70
English Horn 86
Lead 6 (voice)
102
FX 6 (goblins)
118
Melodic Tom 7
Harpsichord 23
Harmonica 39
Synth Bass 1 55
Synth Voice 71
Bassoon 87
Lead 7 (fifths)
103
FX 7 (echoes)
119
Synth Drum 8
Clavinet 24
Tango Accordian 40
Synth Bass 2 56
Orchestra Hit 72
Clarinet 88
Lead 8 (bass+lead)
104
FX 8
(sci-fi)
120
Reverse Cymbal
CHROMATIC PERCUSSION
GUITAR
SOLO STRINGS
BRASS
PIPE
SYNTH PAD
ETHNIC
SOUND EFFECTS 9
Celesta 25
Nylon String Guitar 41
Violin 57
Trumpet 73
Piccolo 89
Pad 1
(new age)
105
Sitar
121
Guitar Fret Noise 10
Glockenspiel 26
Steel String Guitar 42
Viola 58
Trombone 74
Flute 90
Pad 2 (warmth)
106
Banjo
122
Breath Noise 11
Music Box 27
Electric Jazz Guitar 43
Cello 59
Tuba 75
Recorder 91
Pad 3 (polysynth)
107
Shamisen
123
Seashore 12
Vibraphone 28
Electric Clean Guitar 44
Contrabass 60
Muted Trumpet 76
Pan Flute 92
Pad 4
(choir)
108
Koto
124
Bird Tweet 13
Marimba 29
Electric Muted Guitar 45
Tremolo Strings 61
French Horn 77
Blown Bottle 93
Pad 5 (bowed)
109
Kalimba
125
Telephone Ring 14
Xylophone 30
Overdriven Guitar 46
Pizzicato Strings 62
Brass Section 78
Skakuhachi 94
Pad 6 (metallic)
110
Bagpipe
126
Helicopter 15
Tubular Bells 31
Distortion Guitar 47
Orchestral Strings 63
Synth Brass 1 79
Whistle 95
Pad 7
(halo)
111
Fiddle
127
Applause 16
Dulcimer 32
Guitar Harmonics 48
Timpani 64
Synth Brass 2 80
Ocarina 96
Pad 8 (sweep)
112
Shanai
128
Gunshot
Part 5.5
Audio Programming
Page 182

183. Example MIDI System
The composer would play the individual parts of a piece of music on the keyboard one at a time, and these individual parts would be captured by the sequencer, which would later be used to play the parts back together through the sound modules.
Each part would be played on a different MIDI Channel (MIDI is set up to allow up to 16 channels), and the sound modules would be set to receive different channels.
For example, Sound Module #1 might be set to play the part received on Channel 1 using a piano sound, while Module #2 plays the information received on Channel 5 using an acoustic bass sound, and the drum machine plays the percussion part received on MIDI Channel 10.
Part 5.5
Audio Programming
Page 183

184. Downloadable Sounds (DLS)
Yamaha’s DLS specifications expand the composer’s capabilities beyond the 128-instrument MIDI set.
Composers may create their own Pulse Code Modulation (PCM) instruments and sound effects and download them to DLS-compatible playback devices.
Most modern sound cards have hardware acceleration for DLS, and Microsoft has set up an API to accommodate DLS-compatible cards, as well as a DLS software synthesizer to produce the appropriate music and sound effects on PCs without sound cards.
Part 5.5
Audio Programming
Page 184

185. Environmental Reverberation
To take advantage of modern audio cards in games, programming accurate sound propagation is necessary.
Echo
Direct Path
Reverberation
Part 5.5
Audio Programming
Page 185

186. Environmental Geometry
To model sound accurately, the existence or absence of direct paths must be taken into account.
The lower volume echo and reverberation are heard, emphasizing the existence of the barrier.
Both the direct path and early reflections are blocked, so only the later reflections are heard.
Reverberation
No Echo Path
Echo
No Direct Path
No Direct Path
Reverberation
Part 5.5
Audio Programming
Page 186