1. Chapter 11: Sound, The Auditory System, and Pitch Perception

2. Overview of Questions
If a tree falls in the forest and no one is there to hear it, is there a sound?
What is it that makes sounds high pitched or low pitched?
How do sound vibrations inside the ear lead to the perception of different pitches?
How are sounds represented in the auditory cortex?

3. Pressure Waves and Perceptual Experience
Two definitions of “sound”
Physical definition - sound is pressure changes in the air or other medium.
Perceptual definition - sound is the experience we have when we hear.

4. Sound Waves
Loud speakers produce sound by:
The diaphragm of the speaker moves out, pushing air molecules together called condensation.
The diaphragm also moves in, pulling the air molecules apart called rarefication.
The cycle of this process creates alternating high- and low-pressure regions that travel through the air.

5. Figure 11.1 (a) The effect of a vibrating speaker diaphragm on the surrounding air. Dark areas represent regions of high air pressure, and light areas represent areas of low air pressure. (b) When a pebble is dropped into still water, the resulting ripples appear to move outward. However, the water is actually moving up and down, as indicated by movement of the boat. A similar situation exists for the sound waves produced by the speaker in (a).

6. Sound Waves - continued
Pure tone - created by a sine wave
Amplitude - difference in pressure between high and low peaks of wave
Perception of amplitude is loudness
Decibel (dB) is used as the measure of loudness
Number of dB = 20 logarithm(p/po)
The decibel scale relates the amplitude of the stimulus with the psychological experience of loudness.
p is the sound pressure of the stimulus, and po is the standard sound pressure which is usually set at 20 micropascals. The standard pressure is close to the pressure of a 1,000 Hz tone at threshold in a free field. Note that adding sound pressure level (SPL) to a dB measure indicates that the standard pressure of 20 micropascals was used.

7. Figure (a) Plot of sine-wave pressure changes for a pure tone; (b) Pressure changes are indicated, as in Figure 11.1, by darkening (pressure increased relative to atmospheric pressure) and lightening (pressure decreased relative to atmospheric pressure.)

8. Figure 11. 3 Three different amplitudes of a pure tone
Figure 11.3 Three different amplitudes of a pure tone. Larger amplitudes are associated with the perception of greater loudness.

9. Table 11.1 Relative amplitudes and decibels for environmental sounds

10. The instructor can use this slide to explain the relationship between changes in amplitude of the physical stimulus and how this relates to changes in perceived loudness.
Figure 11.7 Loudness of a 1,000 Hz tone as a function of intensity, determined using magnitude estimation. (Adapted from Gulick, Gescheider, & Frisina, 1989.)

11. Sound Waves - continued
Frequency - number of cycles within a given time period
Measured in Hertz (Hz) - 1 Hz is one cycle per second
Perception of pitch is related to frequency.
Tone height is the increase in pitch that happens when frequency is increased.

12. Figure 11. 4 Three different frequencies of a pure tone
Figure 11.4 Three different frequencies of a pure tone. Higher frequencies are associated with the perception of higher pitches.

13. Complex Periodic Sounds
Both pure and some complex tones are periodic tones.
Fundamental frequency is the repetition rate and is called the first harmonic.
Periodic complex tones consist of a number of pure tones called harmonics.
Additional harmonics are multiples of the fundamental frequency.

14. Complex Periodic Sounds - continued
Additive synthesis - process of adding harmonics to create complex sounds
Frequency spectrum - display of harmonics of a complex sound
Attack of tones - buildup of sound at the beginning of a tone
Decay of tones - decrease in sound at end of tone
The instructor can tell the students that the sounds being discussed here are periodic sounds, which are the most widely studied type of sounds. Aperiodic sounds, which do have repeating sounds waves, are also widely heard in our environment, but have not been investigated as extensively as periodic sounds.

15. Figure 11.5 Left: Waveforms of (a) a complex periodic sound with a fundamental frequency of 200 Hz; (b) fundamental (first harmonic) = 200 Hz; (c) second harmonic = 400 Hz; (d) third harmonic = 600 Hz; (e) fourth harmonic = 800 Hz. Right: Frequency spectra for each of the tones on the left. (Adapted from Plack, 2005)

16. Complex Periodic Sounds - continued
Timbre - all other perceptual aspects of a sound besides loudness, pitch, and duration
It is closely related to the harmonics, attack and decay of a tone.
Effect of missing fundamental frequency
Removal of the first harmonic results in a sound with the same perceived pitch, but with a different timbre.
This is called periodicity pitch.

17. Figure Frequency spectra for a guitar, a bassoon, and an alto saxophone playing a tone with a fundamental frequency of 196 Hz. The position of the lines on the horizontal axis indicates the frequencies of the harmonics and their height indicates their intensities. (From Olson, 1967)

18. The instructor can talk about how removing the fundamental frequency changes the tone’s waveform but not the repetition rate, which is what indicates the frequency of the harmonic.
Figure 11.6 (a) The complex tone from Figure 11.5A, with its frequency spectrum; (b) the same tone with its first harmonic removed (Adapted from Plack, 2005)

19. Musical Scales and Frequency
Letters in the musical scale repeat.
Notes with the same letter name (separated by octaves) have fundamental frequencies that are multiples of each other.
These notes have the same tone chroma.
We perceive such notes as similar to one another.

20. Figure A piano keyboard, indicating the frequency associated with each key. Moving up the keyboard to the right increases the frequency and tone height. Notes with the same letter, like the A’s (arrows) have the same tone chroma.

21. Human hearing range - 20 to 20,000 Hz
Range of Hearing
Human hearing range - 20 to 20,000 Hz
Audibility curve - shows the threshold of hearing in relation to frequency
Changes on this curve show that humans are most sensitive to 2,000 to 4,000 Hz.
Auditory response area - falls between the audibility curve and and the threshold for feeling
It shows the range of response for human audition.
The instructor can note that the range from 2,000 to 4,000 Hz is the most important range for understanding human speech. In addition, the instructor can note how other animals can perceive sounds outside of the range of human hearing (elephants can hear below 20 Hz; dogs can hear frequencies above 40,000 Hz, cats above 50,000 Hz and dolphins as high as 150,000 Hz.

22. Range of Hearing - continued
Equal loudness curves - determined by using a standard 1,000 Hz tone
Two dB levels are used - 40 and 80
Participants match the perceived loudness of all other tones to the 1,000 Hz standard.
Resulting curves show that tones sound
Almost equal loudness at 80 dB.
Softer at 40 dB for high and low frequencies than the rest of the tones in the range.

23. Figure 11. 9 The audibility curve and the auditory response area
Figure The audibility curve and the auditory response area. Hearing occurs in the light green area between the audibility curve (the threshold for hearing) and the upper curve (the threshold for feeling). Tones with combinations of dB and frequency that place them in the light red area below the audibility curve cannot be heard. Tones above the threshold of feeling result in pain. The places where the dashed line at 10 dB crosses the audibility function indicate which frequencies can be heard at 10 dB SPL. (From Fletcher & Munson, 1933)

24. The Ear
Outer ear - pinna and auditory canal
Pinna helps with sound location.
Auditory canal - tube-like 3 cm long structure
It protects the tympanic membrane at the end of the canal.
The resonant frequency of the canal amplifies frequencies between 1,000 and 5,000 Hz.

25. Figure The ear, showing its three subdivisions -- outer, middle, and inner. (From Lindsay & Norman, 1977)

26. The Middle Ear
Two cubic centimeter cavity separating inner from outer ear
It contains the three ossicles
Malleus - moves due to the vibration of the tympanic membrane
Incus - transmits vibrations of malleus
Stapes - transmit vibrations of incus to the inner ear via the oval window of the cochlea

27. Figure The middle ear. The three bones of the middle ear transmit the vibrations of the tympanic membrane to the inner ear.

28. Outer and inner ear are filled with air.
Function of Ossicles
Outer and inner ear are filled with air.
Inner ear is filled with fluid that is much denser than air.
Pressure changes in air transmit poorly into the denser medium.
Ossicles act to amplify the vibration for better transmission to the fluid.
Middle ear muscles dampen the ossicles’ vibrations to protect the inner ear from potentially damaging stimuli.
The instructor can also note that the middle ear contains muscles, the smallest skeletal muscles in the body, which are responsible for dampening the ossicles’ vibration to protect the inner ear from damaging stimuli.

29. Figure 11. 13 Environments inside the outer, middle, and inner ears
Figure Environments inside the outer, middle, and inner ears. The fact that liquid fills the inner ear poses a problem for the transmission of sound vibrations from the air of the middle ear.

30. Figure (a) A diagrammatic representation of the tympanic membrane and the stapes, showing the difference in size between the two. (b) How lever action can amplify the effect of a small force, presented on the right, to lift the large weight on the left. The lever action of the ossicles amplifies the sound vibrations reaching the tympanic inner ear. (From Schubert, 1980).

31. The Inner Ear
Main structure is the cochlea
Fluid-filled snail-like structure (35 mm long) set into vibration by the stapes
Divided into the scala vestibuli and scala tympani by the cochlear partition
Cochlear partition extends from the base (stapes end) to the apex (far end)
Organ of Corti contained by the cochlear partition

32. Figure 11. 15 (a) A partially uncoiled cochlea
Figure (a) A partially uncoiled cochlea. (b) A fully uncoiled cochlea. The cochlear partition, indicated here by a line, actually contains the basilar membrane and the organ of Corti, which are shown in Figure and

33. Inner and outer hair cells are the receptors for hearing
The Organ of Corti
Key structures
Basilar membrane vibrates in response to sound and supports the organ of Corti
Inner and outer hair cells are the receptors for hearing
Tectorial membrane extends over the hair cells
There are approximately 3,500 inner hair cells and 12,000 outer hair cells.

34. The Organ of Corti - continued
Transduction takes place by:
Cilia bend in response to movement of organ of Corti and the tectorial membrane
Movement in one direction opens ion channels
Movement in the other direction closes the channels
This causes bursts of electrical signals.
The instructor can use the next slides to give details about how the movement of the structures works to cause transduction.

35. Figure 11. 16 (a) Cross-section of the cochlea
Figure (a) Cross-section of the cochlea. (b) Close-up of the organ of Corti, showing how it rests on the basilar membrane. Arrows indicate the motions of the basilar membrane and tectorial membrane that are caused by vibration of the cochlear partition. (Adapted from Denes and Pinson, 1993.)

36. Figure (a) Movement of hair cilia in one direction opens ion channels in the hair cell, which results in the release of neurotransmitter onto an auditory nerve fiber; (b) Movement in the opposite direction closes the ion channels so there is no ion flow and no transmitter release.

37. Neural Signals for Frequency
There are two ways nerve fibers signal frequency:
Which fibers are responding
Specific groups of hair cells on basilar membrane activate a specific set of nerve fibers;
How fibers are firing
Rate or pattern of firing of nerve impulses

38. Békésys’ Place Theory of Hearing
Frequency of sound is indicated by the place on the organ of Corti that has the highest firing rate.
Békésy determined this in two ways:
Direct observation of the basilar membrane in cadavers.
Building a model of the cochlea using the physical properties of the basilar membrane.

39. Figure Hair cells all along the cochlea send signals to nerve fibers that combine to form the auditory nerve. According to place theory, low frequencies cause maximum activity at the apex end of the cochlea, and high frequencies cause maximum activity at the base. Activation of the hair cells and auditory nerve fibers indicated in red would signal that the stimulus is in the middle of the frequency range for hearing.

40. Békésys’ Place Theory of Hearing - continued
Physical properties of the basilar membrane
Base of the membrane (by stapes) is:
Three to four times narrower than at the apex.
100 times stiffer than at the apex.
Both the model and direct observation showed that the vibrating motion of the membrane is a traveling wave .
The next slide provides the opportunity to present a description of a traveling wave.

41. Figure A perspective view showing the traveling wave motion of the basilar membrane. This picture shows what the membrane looks like when the vibration is “frozen,” with the wave about two thirds of the way down the membrane. (From Tonndorf, 1960).

42. Figure A perspective view of an uncoiled cochlea, showing how the basilar membrane gets wider at the apex end of the cochlea. The spiral lamina is a supporting structure that makes up for the basilar membrane’s difference in width at the stapes and the apex ends of the cochlea. From Schubert, 1980. 

43. Békésys’ Place Theory of Hearing - continued
Envelope of the traveling wave
Indicates the point of maximum displacement of the basilar membrane
Hair cells at this point are stimulated the most strongly leading to the nerve fibers firing the most strongly at this location.
Position of the peak is a function of frequency.

44. Figure Vibration of the basilar membrane, showing the position of the membrane at three instants in time, indicated by the blue, green, and red lines, and the envelope of the vibration, indicated by the yellow dashed line. P indicates the peak of the basilar membrane vibration. (From von Bekesy, 1960).

45. Figure The envelope of the basilar membrane’s vibration at frequencies ranging from 25 to 1,600 Hz, as measured by Békésy (1960). These envelopes were based on measurements of damaged cochleas. The envelopes are more sharply peaked in healthy cochleas.

46. Evidence for the Place Theory
Tonotopic map
Cochlea shows an orderly map of frequencies along its length
Apex responds best to low frequencies
Base responds best to high frequencies

47. Figure 11. 25 Tonotopic map of the guinea pig cochlea
Figure Tonotopic map of the guinea pig cochlea. Numbers indicate the location of the maximum electrical response for each frequency. (From Culler, Coakley, Lowy, & Gross, 1943).

48. Evidence for the Place Theory - continued
Neural frequency tuning curves
Pure tones are used to determine the threshold for specific frequencies measured at single neurons.
Plotting thresholds for frequencies results in tuning curves.
Frequency to which the neuron is most sensitive is the characteristic frequency.

49. Figure 11. 26 Frequency tuning curves of cat auditory nerve fibers
Figure Frequency tuning curves of cat auditory nerve fibers. The characteristic frequency of each fiber is indicated by the arrows along the frequency axis. The frequency scale is in kilohertz (kHz), where one kHz = 1,000 Hz. (From Palmer, 1987)

50. Evidence for the Place Theory - continued
Auditory masking experiments
First, thresholds for a number of frequencies are determined.
Then, a single intense masking frequency is presented at the same time that the thresholds for the original frequencies are re-determined.
The masking effect is seen at the masking tone’s frequency and spreads to higher frequencies more than lower ones.
The instructor can use the next three slides to explain the experiment and that the spread of the masking effect can be explained by the traveling wave seen in the basilar membrane and the tonotopic mapping of frequencies.

51. Figure 11. 27 The procedure for a masking experiment
Figure The procedure for a masking experiment. (a) Threshold is determined across a range of frequencies. Each blue arrow indicates a frequency where the threshold is measured. (b) The threshold is redetermined at each frequency (blue arrows) in the presence of a masking stimulus (red arrow). The larger blue arrows indicate that the intensities must be increased to hear these test tones when the masking tone is present.

52. Figure 11. 28 Results of Egan and Hake’s (1950) masking experiment
Figure Results of Egan and Hake’s (1950) masking experiment. The threshold increases the most near the frequencies of the masking noise and the masking effect spreads more to high frequencies than to low frequencies. (From Egan and Hake, 1950).

53. Figure Vibration patterns caused by 200- and 800-Hz test tones, and the 400-Hz mask (shaded), taken from basilar membrane vibration patterns in Figure Notice that the vibration caused by the masking tone overlaps the 800-Hz vibration more than the 200-Hz vibration.

54. Basilar Membrane Response to Complex Tones
Basilar membrane can be described as an acoustic prism.
There are peaks in the membrane’s vibration that correspond to each harmonic in a complex tone.
Each peak is associated with the frequency of a harmonic.

55. Figure (a) Waveform of a complex tone consisting of three harmonics; (b) Basilar membrane. The shaded areas indicate locations of peak vibration associated with each harmonic in the complex tone.

56. Updating Békésy’s Place Theory
Békésy used basilar membranes isolated from cadavers and his results showed no difference in response for close frequencies that people can distinguish.
New research with live membranes shows that the entire outer hair cells respond to sound by slight tilting and a change in length.
For this reason these cells are called the cochlear amplifier.

57. Figure The outer hair cells (a) elongate when cilia bend in one direction; (b) contract when the cilia bend in the other direction. This results in an amplifying effect on the motion of the basilar membrane.

58. The slide can be used to talk about the importance of the function of the outer hair cells and what happens if they are damaged.
Figure Effect of OHC damage on frequency tuning curve. The solid curve is the frequency tuning curve of a neuron with a characteristic frequency of about 8,000 Hz. The dashed curve is the tuning curve for the same neuron after the outer hair cells were destroyed by injection of a chemical (Adapted from Fettipalce & Hackney, 2006).

59. Temporal Coding and Frequency
Phase locking
Nerve fibers fire in bursts.
Firing bursts happen at or near the peak of the sine-wave stimulus.
Thus, they are “locked in phase” with the wave.
Groups of fibers fire with periods of silent intervals creating a pattern of firing.

60. Figure How hair cell activation and auditory nerve fiber firing area synchronized with pressure changes of the stimulus. The auditory nerve fiber fires when the cilia are bent to the right. This occurs at the peak of the sine-wave change in pressure.

61. Place and Temporal Coding for Pitch
Place coding is effective for the entire range of hearing.
Temporal coding with phase locking is effective up to 4,000 Hz.
Thus, both codes work for frequencies below 4,000 Hz.

62. Hearing Loss
Two types
Conductive hearing loss
Blockage of sound from the receptor cells
Sensorineural hearing loss
Damage to hair cells
Damage to the auditory nerve or brain
Most common type is prebycusis

63. Hearing Loss - continued
Presbycusis
Greatest loss is at high frequencies
Affects males more severely than females
Appears to be caused by exposure to damaging noises or drugs
Noise-induced hearing loss
Loud noise can severely damage the Organ of Corti
OSHA standards for noise levels at work are set to protect workers
Leisure noise can also cause hearing loss

64. Figure 11. 34 Hearing loss in presbycusis as a function of age
Figure Hearing loss in presbycusis as a function of age. All the curves are plotted relative to the 20-year-old curve, which is taken as the standard (Adapted from Dubno, in press).

65. Figure Sound level of game 3 of the 2006 Stanley Cup finals between the Edmunton Oilers (the home team) and the Carolina Hurricanes. Sound levels were recorded by a small microphone in a spectator’s ear. The red line indicates a “safe” level for a 3-hour game. (From Hodgetts & Liu, 2006)

66. Pathway from the Cochlea to the Cortex
Auditory nerve fibers synapse in a series of subcortical structures
Cochlear nucleus
Superior olivary nucleus (in the brain stem)
Inferior colliculus (in the midbrain)
Medial geniculate nucleus (in the thalamus)
Auditory receiving area (A1 in the temporal lobe)

67. Figure 11. 36 Diagram of the auditory pathways
Figure Diagram of the auditory pathways. This diagram is greatly simplified, as numerous connections between the structures are not shown. Note that auditory structures are bilateral -- they exist on both the left and right sides of the body -- and that messages can cross over between the two sides. (Adapted from Wever, 1949.)

68. Auditory Areas in the Cortex
Hierarchical processing occurs in the cortex
Neural signals travel through the core, then belt, followed by the parabelt area.
Simple sounds cause activation in the core area.
Belt and parabelt areas are activated in response to more complex stimuli made up of many frequencies.

69. Figure The three main auditory areas in the cortex are the core area, which contains the primary auditory receiving area (A1), the belt area, and the parabelt area. Signals, indicated by the arrows, travel from core, to belt, to parabelt. The dark lines show where the temporal lobe is pulled back to show areas that would not be visible from the surface. (From Kaas, Hackett, & Tramo, 1999).

70. What and Where Streams for Hearing
What, or ventral stream, starts in the anterior portion of the core and belt and extends to the prefrontal cortex.
It is responsible for identifying sounds.
Where, or dorsal stream, starts in the posterior core and belt and extends to the parietal and prefrontal cortices.
It is responsible for locating sounds.
Evidence from neural recordings, brain damage, and brain scanning support these findings.

71. Figure Areas in the monkey cortex that respond to auditory stimuli. The green areas respond to auditory stimuli, the purple areas to both auditory and visual stimuli. The arrows from the temporal lobe to the frontal lobe represent the what and where streams in the auditory system. (Adapted from Porembra, et al., 2003.)

72. Perceiving Pitch and Complex Sounds
Tonotopic maps are found in A1
Neurons that respond better to low frequencies are on the left and those that respond best to high frequencies are on the right.
Research on stroke patients and marmosets provided support for the link between perception and physiological response in A1.

73. Figure The outline of the core area of the monkey auditory cortex, showing the tonotopic map on the primary auditory receiving area, A1, which is located within the core. The numbers represent the characteristic frequencies (CF) of neurons in thousands of Hz. Low CF’s are on the left, and high CF’s are on the right. (Adapted from Kosaki et al., 1997).

74. The stroke patient showed normal responses to the duration of sounds and to the orientation of lines, but the patient could not detect differences in frequency change from low to high or high to low.
Figure Performance of patient A, with auditory cortex damage on four tasks. See text for details. (Based on data from Tramo et al., 2002).

75. Figure Records from a pitch neuron recorded from the marmoset auditory cortex. (a) Frequency spectra for tones with fundamental frequency of 182 Hz. Each tone contains three harmonic components of the 182 Hz fundamental frequency; (b) Response of the neuron to each stimulus. (Adapted from Bendor & Wang, 2005).

76. Recent Evidence of Pitch Perception in A1
Effect of training on tonotopic maps
Owl monkeys were trained to discriminate between two frequencies near 2,500 Hz.
Trained monkeys showed tonotopic maps with enlarged areas with neurons that responded to 2,500 Hz compared to untrained monkeys.
Cases of humans with brain damage to this area show perception difficulties with pitch.

77. Figure (a) Tonotopic map of the owl monkey’s primary auditory receiving area (A1), showing areas that contain neurons with the characteristic frequencies indicated. The blue area contains neurons with CF = 2,500 Hz. (b) Tonotopic map of an owl monkey that was trained to discriminate between frequencies near 2,500 Hz. The blue areas indicate that after training more of the cortex responds best to 2,500 Hz. (From Recanzone et al., 1993).

78. Recent Evidence of Pitch Perception in A1 - continued
Brain scans in humans
Tasks that require pitch recognition activate areas equivalent to the core area in monkeys.
Tasks that require recognition of complex stimuli activate areas equivalent to the parabelt area in monkeys.
Thus, stimuli that are more complex are processed farther “downstream” in the nervous system.

79. Effect of Experience on the Auditory Cortex
Musicians show enlarged auditory cortices that respond to piano tones and stronger neural responses than non-musicians.
Experiment by Fritz et al.
Marmosets were trained to lick a water spout in response to a pure tone embedded within a stream of complex tones.
Neurons became quickly tuned to the target frequency and maintained the effect for hours after the testing session.

80. Figure Response of a neuron in the ferret auditory cortex: (a) before training; (b) after training. See text for details. (Fritz et al., 2003_

81. Cochlear Implants
Electrodes are inserted into the cochlea to electrically stimulate auditory nerve fibers.
The device is made up of:
a microphone worn behind the ear,
a sound processor,
a transmitter mounted on the mastoid bone,
and a receiver surgically mounted on the mastoid bone.

82. Figure 11.46 Cochlear implant device. See text for details.

83. Cochlear Implants - continued
Implants stimulate the cochlea at different places on the tonotopic map according to specific frequencies in the stimulus.
These devices help deaf people to hear some sounds and to understand language.
They work best for people who receive them early in life or for those who have lost their hearing, although they have caused some controversy in the deaf community.