1 Psy280: Perception Prof. Anderson Department of Psychology Audition 1 & 2

2 Hearing: What’s it good for? Remote sensing Remote sensing Not restricted like visual field Not restricted like visual field Can sense object not visible Can sense object not visible

3 Hearing: The sound of silence A tree in the forest A tree in the forest Physical signal but no perception Physical signal but no perception One hand clapping One hand clapping No physical signal, no perception No physical signal, no perception Separate physical quantity from perceptual quality Separate physical quantity from perceptual quality Sound is the perceptual correlate of the physical changes in air pressure Sound is the perceptual correlate of the physical changes in air pressure Or water pressure when under water Or water pressure when under water John Cage’s 4:33 No. 2, 1962 John Cage’s 4:33 No. 2, 1962

4 What are the physical attributes associated with sound? Loudness Loudness Amplitude or height of pressure wave Amplitude or height of pressure wave Pitch Pitch Frequency of times per second (Hz) a pressure wave repeats itself Frequency of times per second (Hz) a pressure wave repeats itself

5 What is sound quality? Pure tones Pure tones Single frequency (f) Single frequency (f) Rarely exist in real world Rarely exist in real world Complex tones Complex tones More than one f More than one f Due to resonance Due to resonance Air pressure causes reverberations Air pressure causes reverberations E.g., tuning forks E.g., tuning forks E.g., Plucking the A string on a guitar E.g., Plucking the A string on a guitar Fundamental frequency 440 Hz (cycles/s) Fundamental frequency 440 Hz (cycles/s) Harmonics Harmonics Reverberations at multiples of the fundamental Reverberations at multiples of the fundamental E.g., 880, 1320 E.g., 880, 1320 Creates fullness of complex sounds Creates fullness of complex sounds Timbre is the relative amplification of harmonics Timbre is the relative amplification of harmonics

6 The human ear Outer ear Outer ear Focusing of sound Focusing of sound Resonance amplifies Hz range Resonance amplifies Hz range Converts from air to mechanical vibration Converts from air to mechanical vibration Middle ear Middle ear Amplification Amplification Fluid denser than air Fluid denser than air Focus vibrations onto stapes/oval window Focus vibrations onto stapes/oval window Increased leverage from ossicles Increased leverage from ossicles Inner ear Inner ear Sensory transduction Sensory transduction Physical to neural energy Physical to neural energy Fluid pressure changes Fluid pressure changes Bending of hair cells Bending of hair cells

7 Auditory sensory transduction: The inner ear Cochlea Cochlea Coiled and liquid filled Coiled and liquid filled 3 layers 3 layers Cochlear partition Cochlear partition Contains organ of corti Contains organ of corti Organ of corti Organ of corti Cilia (hair) cells Cilia (hair) cells Between basilar and tectorial membranes Between basilar and tectorial membranes Transduction Transduction Movement of cilia between membranes Movement of cilia between membranes

8 Auditory transduction Bending—>physical energy Bending—>physical energy Converted to neural signals Converted to neural signals Bend one direction —> depolarization Bend one direction —> depolarization More likely to fire AP More likely to fire AP Other direction —> hyperpolarization Other direction —> hyperpolarization Less likely to fire AP Less likely to fire AP

9 Auditory pathways

10 Audition: What and where What is it? What is it? *Pitch *Pitch Identification Identification Surprisingly, little is known beyond speech Surprisingly, little is known beyond speech Where is it? Where is it? *location *location

11 What: Pitch How does neural firing signal different pitches? How does neural firing signal different pitches? 1) Timing codes 1) Timing codes 2) Place codes 2) Place codes

12 Pitch: Temporal coding Idea: Diff f’s signaled by rate of neuronal firing Idea: Diff f’s signaled by rate of neuronal firing Hair cell response Hair cell response Bend one direction —> depolarization Bend one direction —> depolarization Other direction —> hyperpolarization Other direction —> hyperpolarization Result? Result? Bursting pattern of neural response related to frequency of oscillation Bursting pattern of neural response related to frequency of oscillation

13 Problems with temporal coding Problem: A single neuron can’t fire at the rate necessary to represent higher f tones Problem: A single neuron can’t fire at the rate necessary to represent higher f tones E.g., ,000 Hz (i.e., per second) E.g., ,000 Hz (i.e., per second) Max neuron firing rate: per second Max neuron firing rate: per second Solution: volley principle Solution: volley principle No single neuron represents f No single neuron represents f Coding across many neurons with staggered firing rates Coding across many neurons with staggered firing rates Evidence: Phase locking Evidence: Phase locking Diff neurons respond to Diff neurons respond to diff peaks Not every peak Not every peak Pool across multiple neurons to Pool across multiple neurons to represent high f’s

14 Pitch: Place coding Related to doctrine of specific nerve energies Related to doctrine of specific nerve energies What is pitch? What is pitch? Activation of different places in auditory system Activation of different places in auditory system Frequency specific Frequency specific Tonotopy Tonotopy Cochlear Cochlear Brainstem Brainstem Cortical Cortical Stimulate these regions Stimulate these regions Should result in pitch perception Should result in pitch perception Owl brainstem Human auditory cortex

15 Place coding starts in cochlea Von Bekesy studied basilar membrane in cadavers Von Bekesy studied basilar membrane in cadavers Base more narrow and stiffer Base more narrow and stiffer Apex wider and more flexible Apex wider and more flexible Observed traveling waves Observed traveling waves Diff frequencies (f) result in waves w/ diff envelopes Diff frequencies (f) result in waves w/ diff envelopes Higher f: Peak closer to base Higher f: Peak closer to base Lower f: Peak closer to apex Lower f: Peak closer to apex Thus, f related to “place” where peak fluctuation occurs Thus, f related to “place” where peak fluctuation occurs

16 Frequency tuning: Neural place coding Tonotopic arrangement of hair cell nerves Tonotopic arrangement of hair cell nerves Diff nerves innervate diff parts of basilar membrane Diff nerves innervate diff parts of basilar membrane Allows for “place” code for frequency Allows for “place” code for frequency Frequency tuning curves of single hair cells

17 Complex tones: Fourier decomposition Basilar membrane acts as f analyzer Basilar membrane acts as f analyzer Breaks down complex f inputs into constituent pure tone components Breaks down complex f inputs into constituent pure tone components

18 Auditory masking: Evidence for cochlear place coding Auditory masking Auditory masking Presence of certain tones decreases perception of nearby tones Presence of certain tones decreases perception of nearby tones Similar f result in greater masking Similar f result in greater masking Asymmetry in spread of masking Asymmetry in spread of masking Consistent with basilar vibrational overlap Consistent with basilar vibrational overlap E.g. 400 Hz mask overlaps more with 800 than 200 Hz E.g. 400 Hz mask overlaps more with 800 than 200 Hz 400 Hz mask Increases threshold for 800 more than 200 Hz

19 Mystery of the missing fundamental 400 Hz fundamental plus harmonics (800, 1200, 1600, 2000) 400 Hz fundamental plus harmonics (800, 1200, 1600, 2000) Sounds like 400 Hz pitch with complex timbre Sounds like 400 Hz pitch with complex timbre What if remove fundamental f (400Hz)? What if remove fundamental f (400Hz)? Perceived pitch doesn’t change! Perceived pitch doesn’t change! Hence: The missing fundamental Hence: The missing fundamental Problem for place coding Problem for place coding No direct stimulation of 400 Hz on basilar membrane No direct stimulation of 400 Hz on basilar membrane f Harmonic structure determines perceived pitch Harmonic structure determines perceived pitch Not what is present on basilar membrane Not what is present on basilar membrane What we hear is not what the basilar membrane tell us, but what our brain does What we hear is not what the basilar membrane tell us, but what our brain does

20 What does Barry White sound like on the telephone? Telephone carries Hz Telephone carries Hz Typical male voice Typical male voice Fundamental f = 120 Hz Fundamental f = 120 Hz Barry white Barry white 30 Hz? 30 Hz? Can’t speak to Barry on the telephone? Can’t speak to Barry on the telephone? Missing fundamental allows us to hear “virtual” pitch of voice Missing fundamental allows us to hear “virtual” pitch of voice

21 If its too loud your too old Db (SPL) scale Db (SPL) scale Loudness doubles about every 10 db at 1000 Hz Loudness doubles about every 10 db at 1000 Hz Audibility curves Audibility curves Loudness varies with f Loudness varies with f Low volume Low volume Attenuated low and high f relative to midrange Attenuated low and high f relative to midrange High volume High volume Less frequency attenuation Less frequency attenuation Low volume sounds muddy Low volume sounds muddy Mostly mid range Mostly mid range I like my music loud I like my music loud Pain and pleasure Each curve represents equal loudness

22 Otoacoustic emissions: Talking ears Ears don’t only receive sounds, they make them! Ears don’t only receive sounds, they make them! Discovered in 1978 Discovered in 1978 Tiny microphones Tiny microphones Occur spontaneously and also in response to sound Occur spontaneously and also in response to sound It like your ears are talking back! It like your ears are talking back! Created by movement of outer hair cells (ohc) Created by movement of outer hair cells (ohc) Part of auditory sensitivity is movement of ohc to change region specific flexibility of basilar membrane Part of auditory sensitivity is movement of ohc to change region specific flexibility of basilar membrane Allows tuning curves to be so narrow Allows tuning curves to be so narrow Hearing impairments often start with loss of ohc function Hearing impairments often start with loss of ohc function

23 Auditory localization Where is the sound coming from? Where is the sound coming from? Distance Distance Elevation (vertical) Elevation (vertical) Azimuth (horizontal) Azimuth (horizontal) Localization not nearly as precise as vision Localization not nearly as precise as vision Localization within degrees in front of head Localization within degrees in front of head 20 degrees behind head 20 degrees behind head Suggests important role of vision Suggests important role of vision Tunes auditory localization Tunes auditory localization

24 Why is is auditory localization not obvious? Vision Vision Stimulate different photoreceptors in eye Stimulate different photoreceptors in eye Audition Audition No such separation of sounds sources on sensory surface No such separation of sounds sources on sensory surface Sources combine to equally stimulate ear receptors Sources combine to equally stimulate ear receptors

25 Why have two ears? Two aural perspectives on the world Two aural perspectives on the world Like vision, can be used to get different sound pictures of environment Like vision, can be used to get different sound pictures of environment Binaural cues Binaural cues The disparities between ears is used for localization The disparities between ears is used for localization

26 Azimuth Interaural (between ears) Time Difference (ITD) Interaural (between ears) Time Difference (ITD) Air pressure changes are very slow relative to speed of light Air pressure changes are very slow relative to speed of light ITD at side = max 600 µS ITD at side = max 600 µS ITD at front = 0 ITD at front = 0 Can induce perception of location by varying ITD using headphones Can induce perception of location by varying ITD using headphones Interaural Level (intensity) Difference (ILD) Interaural Level (intensity) Difference (ILD) Amplitude decreases w/ distance Amplitude decreases w/ distance Head casts sound/acoustic shadow Head casts sound/acoustic shadow Reduced amplitude due to reflection Reduced amplitude due to reflection Measure w/ tiny microphones Measure w/ tiny microphones f dependent f dependent Greater shadow for higher f Greater shadow for higher f

27 Elevation ITD/ILD not very useful ITD/ILD not very useful Use spectral cues Use spectral cues Frequency information can result in different perceptual qualia Frequency information can result in different perceptual qualia Monaural: f serves as signal for pitch Monaural: f serves as signal for pitch Binaural: f serves as signal for location Binaural: f serves as signal for location Pinna differentially absorb f Pinna differentially absorb f Result: Notches in frequency spectra Result: Notches in frequency spectra Above Level Below

28 Distance At close distances (< 1 meter) At close distances (< 1 meter) ILD can discriminate near and far ILD can discriminate near and far At very close distances ILD is very large (e.g. 20 Db) At very close distances ILD is very large (e.g. 20 Db) But what’s that going to do for us? But what’s that going to do for us? At far distances At far distances We are very poor judges for unfamiliar sounds We are very poor judges for unfamiliar sounds Suggests that sound serves as signal for visual search Suggests that sound serves as signal for visual search Use sound level for familiar sources Use sound level for familiar sources Frequency: Auditory atmospheric haze Frequency: Auditory atmospheric haze Absorption of high f Absorption of high f Sound muffled Sound muffled Auditory parallax Auditory parallax Sounds move faster across ears at near relative to far distances Sounds move faster across ears at near relative to far distances

29 Brain basis for localization ITD detectors ITD detectors Brainstem: Superior olivary nucleus Brainstem: Superior olivary nucleus Primary auditory cortex Primary auditory cortex Coincidence detection Coincidence detection Neurons fire maximally when signals arrive at same time Neurons fire maximally when signals arrive at same time Thus: “coincidence” Thus: “coincidence” Axonal distance create input delays Axonal distance create input delays Sound to right Sound to left

30 Auditory scene analysis How do we segregate different sounds being produced by many sources simultaneously? How do we segregate different sounds being produced by many sources simultaneously? How do we tell what frequencies belong to what source? How do we tell what frequencies belong to what source? E.g., Cocktail party E.g., Cocktail party Don’t perceive an unorganized jumble of frequencies Don’t perceive an unorganized jumble of frequencies Not simply high vs low f Not simply high vs low f Most f ranges overlap Most f ranges overlap How do we segregate information as belonging to distinct auditory objects? How do we segregate information as belonging to distinct auditory objects?

31 Principles of auditory grouping Like gestalt visual principles Like gestalt visual principles Auditory stream segregation Auditory stream segregation Similarity Similarity Timbre Timbre Location Location Pitch Pitch Time Time 1 stream 2 streams

32 Auditory-visual interactions: Location and pitch Visual capture of sound Visual capture of sound Location: Ventriloquism effect Location: Ventriloquism effect Pitch: McGurk effect Pitch: McGurk effect “Ba” “Ba” “Va” “Va” “Tha” “Tha” “Da” “Da” Visual information is integrated with audition Visual information is integrated with audition Creates fused auditory visual perception Creates fused auditory visual perception

33 Auditory-visual interactions: Location and pitch Auditory experience is much more than pressure level changes Auditory experience is much more than pressure level changes