Hearing Complex Sounds PSY 295 – Sensation & Perception Christopher DiMattina, PhD

PSY 295 - Grinnell College - Fall 2012 Complex Sounds PSY 295 - Grinnell College - Fall 2012

PSY 295 - Grinnell College - Fall 2012 Complex Sounds We have talked a lot about how the brain processes tones Real environmental sounds are a lot more complex! PSY 295 - Grinnell College - Fall 2012

Structure of natural sounds Many natural sounds have harmonic structure This means that there is spectral energy at a lowest frequency called the fundamental, and energy at integer multiples of the fundamental 200 Hz fundamental has harmonics at 400, 600, 800, etc… PSY 295 - Grinnell College - Fall 2012

PSY 295 - Grinnell College - Fall 2012 Harmonic structure PSY 295 - Grinnell College - Fall 2012

The missing fundamental When you remove the fundamental frequency, you still hear it the pitch of the fundamental! PSY 295 - Grinnell College - Fall 2012

The missing fundamental This happens because the harmonics all have periodicity at the fundamental frequency, so when added together the waveform has periodicity at the fundamental frequency PSY 295 - Grinnell College - Fall 2012

PSY 295 - Grinnell College - Fall 2012 Web activity http://sites.sinauer.com/wolfe3e/chap10/missingfundF.htm PSY 295 - Grinnell College - Fall 2012

Harmonic combination sensitivity Neurons in the auditory cortex often exhibit multi-peaked response areas with peaks at harmonic ratios PSY 295 - Grinnell College - Fall 2012

Harmonic combination sensitivity Response to two harmonically related tones is much greater than the response to one tone alone PSY 295 - Grinnell College - Fall 2012

PSY 295 - Grinnell College - Fall 2012 Marmoset calls PSY 295 - Grinnell College - Fall 2012

Harmonic combination sensitivity Also seen in songbirds where songs have a harmonic structure PSY 295 - Grinnell College - Fall 2012

PSY 295 - Grinnell College - Fall 2012 Timbre Can define a tone by its pitch and loudness Complex sounds have many spectral components Its qualitative character depends on it spectral shape PSY 295 - Grinnell College - Fall 2012

PSY 295 - Grinnell College - Fall 2012 Timbre Different musical instruments and vowel sounds with the same fundamental frequency PSY 295 - Grinnell College - Fall 2012

PSY 295 - Grinnell College - Fall 2012 Web activity http://sites.sinauer.com/wolfe3e/chap10/timbreF.htm PSY 295 - Grinnell College - Fall 2012

PSY 295 - Grinnell College - Fall 2012 Attack and Decay We are sensitive not only to spectral content but temporal properties Attack – part of sound where amplitude increases Decay – part of sound where amplitude decreases PSY 295 - Grinnell College - Fall 2012

Auditory cortex – ramped and damped Many neurons in auditory cortex distinguish ramped and damped tones PSY 295 - Grinnell College - Fall 2012

Ramped preferring neuron PSY 295 - Grinnell College - Fall 2012

Damped preferring neuron PSY 295 - Grinnell College - Fall 2012

Auditory Scene Analysis PSY 295 - Grinnell College - Fall 2012

Difference between hearing and vision Light waves from different objects block each other if objects are displaced in depth (occlusion) Sound waves from different sources add together PSY 295 - Grinnell College - Fall 2012

PSY 295 - Grinnell College - Fall 2012 A world of glass Imagine vision if everything was transparent This is what your auditory system has to deal with PSY 295 - Grinnell College - Fall 2012

Different sounds have lots of spectral overlap Position on cochlea is not sufficient to separate different sounds! PSY 295 - Grinnell College - Fall 2012

PSY 295 - Grinnell College - Fall 2012 Stream segregation The problem of auditory stream segregation is how we break a complex acoustical waveform into different auditory objects PSY 295 - Grinnell College - Fall 2012

PSY 295 - Grinnell College - Fall 2012 Example When tones are played rapidly at two alternating frequencies, one perceives a single warbling source However, when frequencies are sufficiently different, one hears two separate streams PSY 295 - Grinnell College - Fall 2012

PSY 295 - Grinnell College - Fall 2012 Web activity http://sites.sinauer.com/wolfe3e/chap10/audstreamsegF.htm PSY 295 - Grinnell College - Fall 2012

Bach was the master of auditory stream segregation http://www.youtube.com/watch?v=ipzR9bhei_o PSY 295 - Grinnell College - Fall 2012

PSY 295 - Grinnell College - Fall 2012 Grouping cues Tones “pop out” of stream if they don’t fit pattern Sounds with different timbre segregate PSY 295 - Grinnell College - Fall 2012

PSY 295 - Grinnell College - Fall 2012 Common onset Sounds group by onset time (different onsets  separate sources) PSY 295 - Grinnell College - Fall 2012

Continuity and restoration Gestalt cue of good continuation Sounds assumed to ‘continue’ behind noisy ‘occluder’ PSY 295 - Grinnell College - Fall 2012

Perceptual restoration of tones Sound deleted and replaced with noise, sound is perceived to continue through the noise PSY 295 - Grinnell College - Fall 2012

Perceptual restoration of speech Deleting parts of speech and replacing them with noise, cough, etc.. leads to completion Often people cannot say which segment was deleted! PSY 295 - Grinnell College - Fall 2012

PSY 295 - Grinnell College - Fall 2012 Restoration in birds Starlings trained to peck when they heard a difference between two starling song segments More likely to hear difference with silent gap More likely to fill in familiar rather than unfamiliar songs PSY 295 - Grinnell College - Fall 2012

PSY 295 - Grinnell College - Fall 2012 Context dependence “The *eel fell off the car” “The *eel fell off the table” PSY 295 - Grinnell College - Fall 2012

PSY 295 - Grinnell College - Fall 2012 Web activity http://sites.sinauer.com/wolfe3e/chap10/restorationF.htm PSY 295 - Grinnell College - Fall 2012