Musical Acoustics & Psychoacoustics Donald Byrd 22 Sept Copyright © 2006, Donald Byrd

rev. 8 Sep Review: Representations of Music Three basic forms (representations) of music –Audio: most important for most people (general public) –MIDI files: often best/essential for some musicians, especially for pop, rock, film/TV –Notation: often best/ essential for musicians (even amateurs) & music scholars –Essential difference: how much explicit structure Music holdings of Library of Congress: over 10M items –Includes over 6M pieces of sheet music and 100K’s of scores of operas, symphonies, etc.: all notation! Differences are profound

1 Sep Review: Representations of Music & Audio Audio (e.g., CD, MP3): like speech Time-stamped Events (e.g., MIDI file): like unformatted text Music Notation: like text with complex formatting

1 Sep Review: Rudiments of Musical Acoustics Need some musical acoustics for almost anything in digital audio Acoustics: part of physics –Concepts like frequency & amplitude Psychoacoustics: part of psychology –Concepts like pitch & loudness

10 Sept Review: Materials for Studying Audio What are interesting sounds really like? –Sine waves, etc. are boring (cf. addsynenv) –Sounds of acoustic instruments are “rich” –Vary in every way: with pitch, loudness, time Musical instrument samples Audacity audio editor –For Windows, Mac OS 9 and X, Linux –Download from

10 Sept Review: Parameters of Musical Sound Four basic parameters of a definite-pitched musical note 1. pitch: how high or low the sound is: perceptual analog of frequency 2. duration: how long the note lasts 3. loudness: perceptual analog of amplitude 4. timbre or tone quality Above is decreasing order of importance for most Western music …and decreasing order of explicitness in CMN (Conventional Music Notation)!

rev. 11 Sept. 067 Musical Acoustics (1) Acoustics involves physics Musical (opposed to architectural, etc.) acoustics –Frequency (=> pitch) –Amplitude (=> loudness) –Spectrum, envelope, & “other” characteristics => timbre –Partials vs. harmonics Psychoacoustics involves psychology/perception –Perceptual coding (for “lossy” compression) Timbre –Old idea (thru ca. 1960’s?): timbre produced by static relationships of partials, plus envelope –…but attack often more distinctive than “steady state”! –Rich (interesting) sounds are complex; nothing is static –Time domain (waveform) vs. frequency domain (spectrum, spectrogram) views

11 Sept Creating Interesting Sounds (1) Periodic, nearly-, & non-periodic signals Approaches to creating sounds –Additive synthesis: like painting –Subtractive synthesis: like sculpture –Sampling: like collage –Others: modulation, physical modeling Most possible with analog or digital hardware, but analog is limited

rev. 13 Sept Creating Interesting Sounds (2) Cf. “Electronic Music Tutorial” (Ishkur) Additive Synthesis Fourier's Theorem –Any periodic signal => sum of harmonically-related sine waves Envelopes: continuous & piecewise linear –Important special case: “ADSR” –Attack, Decay, Sustain, Release Phase, interference, & beats –Phase by itself rarely important, but relationships are –Diagrams & demo in CECM Acoustics Primer, Sec. 8 –Interference between channels or very close partials

rev. 13 Sept Additive Synthesis & Envelopes addsynenv does additive synthesis of up to six partials –Each has arbitrary partial no., starting phase, "ADSR" envelope –Partial no. can be non-integer => not harmonic –ADSR = Attack/Decay/Sustain/Release (3 breakpoints) –…but addsynenv allows much more complex envelopes –Plays one note with waveform specified by partial nos. & their envelopes (maybe also phases) –Simultaneously displays “spectrogram” or “sonogram” –…but not waveform –Phase in real world normally has little effect, but can be critical in recording & digital worlds (e.g., cancellation) Additive synthesis can’t create aperiodic (non-definite pitch) sounds...or many realistic attacks

11 Sept Creating Interesting Sounds (3) Early CCRMA (Stanford) studies of acoustic instrument sounds –Envelope for each partial with a few segments –Similar to addsynenv Subtractive Synthesis Early (analog: Moog, etc.) synthesizer model Signal source: sine/square/sawtooth (for additive) …or noise & filters (for subtractive) LFO to add vibrato, tremolo, glissando, etc. ADSR envelope for entire sound

rev. 20 Sep Scholars (and others) Beware! Plausible (at the time) assumptions –Stomach ulcers can’t be caused by organisms (20th century) –Men have more teeth than women (ancient) What you expect & what you see –Sponges, dinosaurs, etc.: discuss later What you expect & what you hear –Don & the Kurzweil 250 flute sound –Don, a famous musician, & K250 handclaps –Huron on what he “knew” & learned R. Moog at Kurzweil & piano touch

22 Sep Uncompressed Audio Files are Big 1 byte = 8 bits (nearly always) How much data on a CD? –CD audio is 44,100 samples/channel/sec. * 2 bytes/sample * 2 channels = 176,400 bytes/sec., or 10.5 MByte/min. –CD can store up to 74 min. (or 80) of music –10.5 MByte/min. * 74 min. = 777 MBytes –Actually more: also index, error correction data, etc.

16 Feb Compressed Audio: Lossless & Lossy Don’t confuse data compression and dynamic- range compression (a.k.a. audio level compression, limiting) Codec = COmpressor/DECompressor Lossless compression –Standard methods (LZW:.zip, etc.) don’t do much for audio –Audio specific methods MLP used for DVD-Audio Apple & Microsoft Lossless Lossy compression –Depends on psychoacoustics (“perceptual coding”)

13 Feb Specs for Some Common Audio Formats

22 Sep Psychoacoustics & Perceptual Coding Pohlmann, Ken (2005). Principles of Digital Audio, 5 th ed., Chapter 10: Perceptual Coding Rationale: much better data compression Physiology of ear and critical bands –Not fixed frequency: any sound creates one or more critical bands Masking –Depends on relative loudness & frequency –Noise is much better than pitched sounds Threshhold of hearing –Depends greatly on frequency

22 Feb Compressed Audio: Lossy Compression General method 1. Divide signal into sub-bands by frequency 2. Take advantage of: Masking (“shadows”), via amplitude within critical bands Threshhold of audibility (varies w/ frequency) Redundancy among channels MPEG-1 layers I thru III (MP-1, 2, 3), AAC get better & better compression via more & more complex techniques –“There is probably no limit to the complexity of psychoacoustics.” --Pohlmann, 5th ed. –However, there probably is an “asymptotic” limit to compression! Implemented in hardware or software codecs

rev. 20 Sep Another “Analog vs. Digital” Battle Merton, Orren (2006, February). The Sum of All Tracks. Electronic Musician 22,2, pp Discusses summing (mixing) with analog vs. digital hardware Tested with excellent hardware, panel of experts Cf. methodology to –ISO’s validation of MP3, AAC, etc. –…or ff123 et al’s of Ogg Vorbis But is this fair? Does it really matter? –Electronic Musician isn’t a scholarly journal –True, but in this case...