Interactive Multimedia Sound Mikael Fernström

Data sources Microphones and transducers –Sample acoustic reality Synthesis –Simulate reality (and beyond ;-) –Many different forms of synthesis, e.g. additive, subtractive, FM, FOF, PM… Algorithms –e.g. auralisation/sonification

Primary processing Recording –Get clear recordings! –Signal-to-noise ratio –Acoustics Filtering –Supressing or emphasizing parts of the spectrum Mixing –Blending of several sounds sources, foreground, background, speech, music, f/x

Sampling “So called” CD quality –16 bit, 44.1 kHz –=> 90 dB dynamic range –Does not match normal human perception! Current professional standards –24 bit, 96 kHz –=> 138 dB dynamic range

Sampling… 8 bit sounds crackling, noisy Sample rates lower than 44.1 kHz, loss of treble Example: –Lowest standard encoding for human speech approx 6 bits 5 kHz (sounds like bad CB radio)

Quantisation

Simple sine wave

Spectrum of sine

Noise spectrum

Aliasing sound FFT

Aliasing Half of the FFT array gets reflected into the other half of the array

Hearing threshold

Masking

Formats Common formats: –WAV, AIFF, RAW… –Stores data “as is” Masking is used in perceptual compression, e.g. MP3, AAC, etc… MP3 is not MPEG! –It’s Layer 3 of MPEG-1 standard

Streaming Download complete file before playing Start playing file when length of file is sufficient to accommodate rest of download Start playing when length of buffer matches data rate

Streaming

MIDI Musical Instrument Digital Interface Only contains control info, no sound! –e.g. note-on, note-off, velocity/loudness

MIDI Sounds different depending on User synth, mapping, etc. General MIDI “standard”, GM Lack of expression, only 7 bit Sometimes problems with latency, 32.5 kb/s –Playing 10 notes simultaneously => 9.2 ms