Chapter 10 Perception of Speech Perry C. Hanavan, Au.D.

Question How do we perceive speech? Individual sounds (phonemes)? Syllables? Words? Sentences? Listening to Mozart?

Speech Perception How do we perceive speech? Individual sounds (phonemes)? Syllables? Words? Sentences? How do we derive meaning from the ocean of sounds we hear? Speech is variable Speakers vary in speech Variant or invariant cues?

Question What is Pattern Playback? A music group from India Talking machine built by Dr. Franklin S. Cooper and colleagues at Haskins Laboratories A brain based device for speech perception

Pattern Playback

Question What is an invariant speech cue? Phonemes coarticulated A phoneme produced in isolation Transition from one phoneme to the next

Sensation: An internal representation of the stimulus. Excitation, Sensation, Cognition Excitation: The pattern of neural responses elicited by a given stimulus. Sensation: An internal representation of the stimulus. Cognition: The interpretation of a sensation on the basis of stored knowledge.

Measured by the Just-Noticeable-Difference threshold. Discrimination The ability to distinguish between two levels of a stimulus parameter (e.g., different wavelengths of light. Measured by the Just-Noticeable-Difference threshold. Uses sensory representation. Modality dependent

Uses cognitive representation: needs to refer to stored knowledge. Recognition The ability to distinguish categorize a stimulus as belonging to a particular class (e.g., colour or object type). Uses cognitive representation: needs to refer to stored knowledge. Representation dependent.

The relationship between discrimination and recognition Recognition relies on discrimination… but does recognition also influence discrimination? Discriminability seems to be affected by category structures – this is categorical perception.

Categorical perception: Discriminability across category boundaries is more sensitive than discriminability within categories.

Phonemes are the sounds that make up language: e.g., /b/ & /p/. First example of categorical perception: the phoneme boundary effect. Phonemes are the sounds that make up language: e.g., /b/ & /p/. The phonemes /b/ and /p/ differ in the time between the onset (stop) and voicing.

Liberman and colleagues (1957) showed a phoneme boundary effect: Alvin Liberman (1917 – 2000) Liberman and colleagues (1957) showed a phoneme boundary effect: A smaller change in delay was necessary to distinguish /b/ from /p/, than to distinguish two phonemes within these categories.

The phoneme boundary effect Motor theory of speech perception: The phoneme boundary effect is caused by activation of the motor program required to produce a phoneme.

Category boundary effects in the colour domain Question: Is the way we sense colour affected by the words for colours in our language? Benjamin Lee Whorf (1897-1941)

Color can be objectively measured in terms of its wavelength: The question about colour perception can be operationalized: Color can be objectively measured in terms of its wavelength: 400n m 550nm 700nm Wavelength

Not subsumed by another term. The question about colour perception can be operationalized: The number of basic color terms in a language can be measured. Basic color terms are: Single words. Not subsumed by another term. Not restricted to a particular class of objects.

Dani (New Guinea): Two basic colour terms - mili (light), mola (dark). Early research on color naming Different languages have a variation in the number of words for colour categories. Dani (New Guinea): Two basic colour terms - mili (light), mola (dark). English: eleven basic color terms – white, black, grey, red green, blue, yellow, orange, purple, pink, brown.

Compared English and Tamahumara speakers. Kay and Kempton (1984) Compared English and Tamahumara speakers. Tamahumara does not make a distinction between blue and green. Kay and Kempton theorized that the perceptual distance between blue and green would be exaggerated in English speakers.

3 green G G G 2 green, 1 blue G G B B 3 blue B B Kay and Kempton (1984) 3 green G G G 2 green, 1 blue G G B B 3 blue B B

Kay and Kempton (1984) Tamahumara speakers were equally likely to choose either extreme for all three types of triplet.

Kay and Kempton (1984) English speakers were the same when all chips came from the same category. When there was an odd one out, they were more likely to choose that one.

Perception of Vowels /a/ vowel has greatest intensity with unvoiced /θ/ as weakest consonant Front vowels perceived on basis of F1 frequency and average of F2 and F3, whereas back vowels are perceived on the basis of the average of F1 and F2, as well as F3 So is it the absolute frequency values of the formants? Or the ratio of F2 to F1? Perhaps it is the invariant cues (frequency changes that occur with coarticulation F1/F2 F3 F1 F2/F3

Invariant and Variant Cues Showing how onset formant transitions that define perceptually consonant [d] differ depending on the identity of the following vowel. (Formants highlighted by red dotted lines; transitions are the bending beginnings of the formant trajectories.) /di/ /da/ /du/

Perception of Diphthongs Perceived on basis of formant transitions Salient feature: rapidity of transition

Consonant Perceptions Perception different for consonants than vowels Greater variety of consonant types than vowels Greater complexity for consonants

Question Which is TRUE regarding the following statements about categorical perception? Experience of percept invariances in sensory phenomena that can be varied along a continuum. Can be inborn or can be induced by learning. Related to how neural networks in our brains detect the features that allow us to sort the things in the world into separate categories All the above are true All the above are false

Categorical Perception Experience of percept invariances in sensory phenomena that can be varied along a continuum. Can be inborn or can be induced by learning. Related to how neural networks in our brains detect the features that allow us to sort the things in the world into separate categories  area in the left prefrontal cortex has been localized as the place in the brain responsible for phonetic categorical perception

Categorical Perception

CI Speech Coding Strategies ACE™: Unique to Cochlear’s Nucleus® 24 CI system. ACE optimizes detailed pitch and timing information of sound. SPEAK: (spectral peak) Increases the richness of important pitch information by stimulating electrodes across the entire electrode array. MPEAK: multipeak CIS : (Continuous-Interleaved Sampling) This high rate strategy uses a fixed set of electrodes. Emphasizes the detailed timing information of speech.

ACE Strategy Sound enters the speech processor through the microphone and is divided into a maximum of 22 frequency bands. Up to 20 narrow-band filters divide sound into corresponding frequency (pitch) ranges. Each frequency band stimulates a specific electrode along the electrode array. The electrode stimulated depends on the pitch of the sound. For example, in the word "show," the high pitch sound (sh) causes stimulation of electrodes placed near the entrance cochlea, where hearing nerve fibers respond to high pitch sounds. The low pitch sound (ow) stimulates electrodes further into the cochlea, where hearing nerve fibers respond to low pitch sounds. ACE varies the rate of stimulation of the electrodes with a total maximum stimulation rate of 14,400 pulses per second.

SPEAK Sound enters the speech processor through the microphone and is divided into 20 frequency bands. SPEAK selects the six to ten frequency bands containing maximum speech information. Each frequency band stimulates a specific electrode along the electrode array. The electrode stimulated depends on the pitch of the sound. For example, in the word "show" the high pitch sound (sh) causes stimulation of electrodes placed near the entrance of the cochlea, where the hearing nerve fibers respond to high pitch sounds. The low pitch sound (ow) stimulates electrodes further into the cochlea, where the hearing nerve fibers respond to low pitch sounds. SPEAK's dynamic stimulation along 20 electrodes allows you to perceive the detailed pitch information of natural sound.

CIS Sound enters the speech processor through the microphone. The sound is divided into 4, 6, 8 or 12 bands depending upon the number of channels used. Each band stimulates one specific electrode along the electrode array, sequentially. The same sites along the electrode are stimulated for every sound at a fast rate to deliver the rapid timing cues of speech.