1. The Perception of Speech

2. Speech Speech is for rapid communication
Speech is composed of units of sound called phonemes
examples of phonemes: /ba/ in bat , /pa/ in pat

3. Acoustic Properties of Speech
Speech can be characterized by a spectrogram

4. Acoustic Properties of Speech
Spectrogram reveals differences between phonemes
The differences are in the formants and the formant transitions

5. Perceiving Speech
So perceiving (interpreting) speech sounds is simply a matter of matching the spectrotemporal properties (the shape of the spectrogram) of the incoming sound waves to the appropriate phoneme
right?…

6. Perceiving Speech
So perceiving (interpreting) speech sounds is simply a matter of matching the spectrotemporal properties (the shape of the spectrogram) of the incoming sound waves to the appropriate phoneme
Then specific phonemes must correspond to specific spectrograms - a property called acoustic-phonetic invariance

7. Perceiving Speech
Acoustic - Phonetic invariance says that phonemes should match one and only one pattern in the spectrogram
This is not the case! For example /d/ followed by different vowels:

8. Perceiving Speech
Acoustic - Phonetic invariance says that phonemes should match one and only one pattern in the spectrogram
This is not the case! For example /d/
Clearly perception and understanding of speech sounds is more elaborate than simply interpreting an internal spectrogram

9. Perceiving Speech
The phrase “Peter buttered the burnt toast” has five /t/ phonemes. There are not 5 identical sweeps in the spectrogram

10. Perceiving Speech
Segmentation is the perception of silence between words
Often illusory

11. Perceiving Speech
The phrase “I owe you a Yo-Yo” has no silence in it !

12. Perceiving Speech Some of the “strategies”: 1. reduce the data
So how do you perceive speech?
Some of the “strategies”:
1. reduce the data
2. use context clues
3. use vision

13. Categorical Perception
Categorical Perception is a phenomenon in which the brain assigns a stimulus into one or another category but never into an intermediate category

14. Categorical Perception
For example, /ba/ and /pa/ differ in their formant transitions
/ba/ is formed by stopping the flow of air from the lungs and releasing it after about 10 milliseconds (called voice onset time)
/pa/ is similar except that voice onset time is about 50 ms

15. Categorical Perception
Voice onset time can range from zero to >50 ms. For example, you could synthesize a sound with a voice onset time of 30 ms but...

16. Categorical Perception
Voice onset time can range from zero to >50 ms. For example, you could synthesize a sound with a voice onset time of 30 ms but...
Listeners will hear either /ba/ or /pa/ but never something in between

17. Categorical Perception is Part of Learning a Language
Babies can discriminate /ba/ from /pa/ and can discriminate these from phonemes with intermediate voice onset times!
By 10 to 12 months, babies (learning English) stop discriminating intermediate voice onset times

18. Categorical Perception is Part of Learning a Language
Once category boundaries are learned it is impossible to unlearn them
non-native speakers of any language often cannot hear certain phonemes the way native speakers do
as a consequence they will always have at least some slight accent

19. Categorical Perception
Another example:

20. Perception (of all types) Makes Use of Context
The stream of information contained in speech is usually ambiguous and incomplete
Your brain makes a “best guess” based on the circumstances

21. Perception (of all types) Makes Use of Context
Consider the following example:
shoe”.
“The __eel fell of the
cough
car”.

22. Perception (of all types) Makes Use of Context
Consider the following example:
Listeners report hearing the “appropriate” phoneme during the cough
shoe”.
“The __eel fell of the
cough
car”.

23. Much of Speech Perception isn’t Auditory !
Why rely on only one sensory system when there is information in two !?

24. Much of Speech Perception isn’t Auditory !
Why rely on only one sensory system when there is information in two !?
The brain seamlessly integrates any information it is given - this is called cross-modal integration

25. Cross-modal Integration
Speech perception involves the synthesis of vision and hearing
The McGurk effect demonstrates the critical role of vision on speech perception

26. Cross-modal Integration
The McGurk Effect

27. Cross-modal Integration
The McGurk Effect - demonstrates that visual and auditory information are combined to enhance speech perception

28. Next Time: Vision