BPC: Art and Computation – Summer 2007 Digital Media – Audio, part 2 Robert Putnam

BPC: Art and Computation – Summer Outline Perceptual qualities of sound (e.g., timbre, pitch) Sound recording and playback: analog v. digital Sound in VR

BPC: Art and Computation – Summer Curriculum connections Qualities of sound –Music theory Overtone/partial series, temperaments –Physics of musical instruments Modes of vibration

BPC: Art and Computation – Summer Curriculum connections Sound recording and playback –Physics of analog sound reproduction Loudspeaker as electromagnet (demo) –Electronics Amplification / Sound reproduction –Mathematics Digital signal processing: sampling, filtering

BPC: Art and Computation – Summer Curriculum connections Sound in VR –Film / Theatre Sound design

BPC: Art and Computation – Summer Qualities of sound Pitch –Perceptual quality –Closely related to frequency –Doubling of frequency: “octave” –Recorded examples –Demonstration (musical instrument)

BPC: Art and Computation – Summer Qualities of sound Timbre –Tone “color” –Made up of combinations of (related) harmonics or “partials” E.g., modes of vibration of string –Part of how we distinguish different instruments/voices –Recorded examples –Demonstrations (voice, musical instrument

BPC: Art and Computation – Summer Qualities of sound Noise –No clearly identifiable pitch –Combination of unrelated harmonics –Analogy with white light –Examples –Demonstrations (voice, musical instrument)

BPC: Art and Computation – Summer Qualities of sound Time variation –Most naturally occurring sounds are not static; i.e., they vary over time Amplitude Pitch Timbre –Examples –Demonstration

BPC: Art and Computation – Summer Sound recording technologies Analog recording – Changes in air pressure recorded as continuously variable physical quantity Phonautograph: squiggles on carbon- coated drum Phonograph: squiggles on vinyl Tape recorder: change in magnetic field orientation in granules of iron oxide Movie film: changing light/dark pattern on plastic film

BPC: Art and Computation – Summer Analog recording technologies Phonautograph

BPC: Art and Computation – Summer Analog recording technologies Mechanical: Gramophone, LP record, etc.

BPC: Art and Computation – Summer Analog recording technologies Magnetic: Wire, tape recorder.

BPC: Art and Computation – Summer Analog recording technologies Optical: movie soundtrack.

BPC: Art and Computation – Summer Analog sound reproduction Amplification Loudspeaker demo

BPC: Art and Computation – Summer Digital recording Digital: “device or system that represents changing values as discontinuous, or ‘discrete,’ values.” Example: clock with number readout.

BPC: Art and Computation – Summer Digital recording concepts Sampling rate: e.g., 44,100 samples/second (CD rate) Word size: e.g., 16 bits (= 2 bytes) Quantization: selection of sample from finite set of values. Nyquist frequency: 1/2 sampling rate (so = ~22Khz for CD)

BPC: Art and Computation – Summer Possible special topics CD recording technology MP3 compression MIDI: Musical Instrument Digital Interface

BPC: Art and Computation – Summer Sound in VR Sound server concepts –Sounds assigned location in 3D audio space (often corresponding to visible 3D model) –Sound rendering consists of altering sound volume in speaker array to indicate direction and distance –Sound mixing happens automatically –*Internet telephony (i.e., live audio) supported

BPC: Art and Computation – Summer Uses of sound in VR Communication (via telephony) Sound effects Music selections Ambient audio Live audio (via telephony) Previous projects have involved controlling synthesizers or musical instruments remotely.

BPC: Art and Computation – Summer Sources of sound in VR Studio and field recordings Free sounds: Commercial recordings* *discussion about “fair use”

BPC: Art and Computation – Summer Sound in VR Demonstration –Proximity triggering –(Variable) Distance attenuation –Sound localization

BPC: Art and Computation – Summer Discussion