Download presentation
1
Speech Perception in Infant and Adult Brains
Colin Phillips Cognitive Neuroscience of Language Laboratory Department of Linguistics University of Maryland
2
Overview of Talks 1. The Unification Problem
2. Building Syntactic Relations 3. Abstraction: Sounds to Symbols 4. Linguistics and Learning どの生徒に…
3
University of Delaware Evniki Edgar Bowen Hui
with help from ... University of Maryland Shani Abada Sachiko Aoshima Daniel Garcia-Pedrosa Ana Gouvea Nina Kazanina Moti Lieberman Leticia Pablos David Poeppel Beth Rabbin Silke Urban Carol Whitney University of Delaware Evniki Edgar Bowen Hui Baris Kabak Tom Pellathy Dave Schneider Kaia Wong Alec Marantz, MIT Elron Yellin, MIT National Science Foundation James S. McDonnell Foundation Human Frontiers Science Program Japan Science & Technology Program Kanazawa Institute of Technology
4
Sensory Maps Internal representations of the outside world. Cellular neuroscience has discovered a great deal in this area.
5
Encoding of Symbols: Abstraction
But most areas of linguistics (phonology, morphology, syntax, semantics) are concerned with symbolic, abstract representations, ...which do not involve internal representations of dimensions of the outside world. …hence, the notion of sensory maps does not get us very far into language
6
Outline Categories & Abstraction Speech Perception in Infancy
Electrophysiology: Mismatch Paradigm Speech Discrimination in Adult Brains Speech Categorization in Adult Brains Conclusion
7
A Category
8
Another Category 3 III
9
Categories for Computation
Membership in a category like “bird” is a graded property Membership in a category like “three” is an all-or-nothing property “Three” can be part of a symbolic computation “Bird” cannot
10
Abstraction Benefits of abstraction Costs of abstraction
representational economy representational freedom allow combinatorial operations Costs of abstraction distant from experience - impedes learning distant from experience - impedes recognition
11
Phonetic vs. Phonological Categories
Phonetic category membership is graded Phonological category membership is an all-or-nothing property: all members are equal Phonological categories are the basis of storage of lexical forms Phonological categories participate in a wide variety of combinatorial computations
12
Timing - Voicing
13
Voice Onset Time (VOT) 60 msec
14
Perceiving VOT ‘Categorical Perception’
15
Discrimination Same/Different
16
Discrimination Same/Different 0ms ms
17
Discrimination Same/Different 0ms ms Same/Different
18
Discrimination Same/Different 0ms ms Same/Different 0ms ms
19
Discrimination Same/Different 0ms 60ms Same/Different 0ms 10ms
20
Discrimination Same/Different 0ms 60ms Same/Different 0ms 10ms
21
Discrimination Same/Different 0ms 60ms Same/Different
Why is this pair difficult? 0ms ms Same/Different 40ms 40ms
22
Discrimination Same/Different 0ms 60ms Same/Different
Why is this pair difficult? 0ms ms (i) Acoustically similar? (ii) Same Category? Same/Different 40ms 40ms
23
Discrimination A More Systematic Test Same/Different 0ms 60ms
Why is this pair difficult? 0ms ms (i) Acoustically similar? (ii) Same Category? Same/Different 40ms 40ms
24
Discrimination A More Systematic Test Same/Different 0ms 60ms 0ms 20ms
25
Discrimination A More Systematic Test D D D T T T Same/Different
0ms ms 0ms 20ms D 20ms 40ms T Same/Different 0ms ms T T 40ms 60ms Same/Different Within-Category Discrimination is Hard 40ms 40ms
26
Cross-language Differences
27
Cross-language Differences
28
Cross-Language Differences
English vs. Japanese R-L
29
Cross-Language Differences
English vs. Hindi alveolar [d] retroflex [D] ?
30
Outline Categories & Abstraction Speech Perception in Infancy
Electrophysiology: Mismatch Paradigm Speech Discrimination in Adult Brains Speech Categorization in Adult Brains Conclusion
31
Development of Speech Perception
Unusually well described in past 30 years Learning theories exist, and can be tested… Jakobson’s suggestion: children add feature contrasts to their phonological inventory during development Roman Jakobson, Kindersprache, Aphasie und allgemeine Lautgesetze, 1941
32
Developmental Differentiation
Universal Phonetics Native Lg. Phonetics Native Lg. Phonology 0 months 6 months 12 months 18 months
33
#1 - Infant Categorical Perception
Eimas, Siqueland, Jusczyk & Vigorito, 1971
34
Discrimination A More Systematic Test D D D T T T Same/Different
0ms ms 0ms 20ms D 20ms 40ms T Same/Different 0ms ms T T 40ms 60ms Same/Different Within-Category Discrimination is Hard 40ms 40ms
35
English VOT Perception
To Test 2-month olds Not so easy! High Amplitude Sucking Eimas et al. 1971
36
General Infant Abilities
Infants’ show Categorical Perception of speech sounds - at 2 months and earlier Discriminate a wide range of speech contrasts (voicing, place, manner, etc.) Discriminate Non-Native speech contrasts e.g., Japanese babies discriminate r-l e.g., Canadian babies discriminate d-D
37
Universal Listeners Infants may be able to discriminate all speech contrasts from the languages of the world!
38
How can they do this? Innate speech-processing capacity?
General properties of auditory system?
39
What About Non-Humans? Chinchillas show categorical perception of voicing contrasts!
40
#2 - Becoming a Native Listener
Werker & Tees, 1984
41
When does Change Occur? About 10 months Janet Werker
U. of British Columbia Conditioned Headturn Procedure
42
When does Change Occur? Hindi and Salish contrasts tested on English kids Janet Werker U. of British Columbia Conditioned Headturn Procedure
43
What has Werker found? Is this the beginning of efficient memory representations (phonological categories)? Are the infants learning words? Or something else?
44
6-12 Months: What Changes?
45
Structure Changing Patricia Kuhl U. of Washington
46
#3 - What, no minimal pairs?
Stager & Werker, 1997
47
A Learning Theory… How do we find out the contrastive phonemes of a language? Minimal Pairs
48
Word Learning Stager & Werker ‘bih’ vs. ‘dih’ and ‘lif’ vs. ‘neem’
49
Word learning results Exp 2 vs 4
50
Why Yearlings Fail on Minimal Pairs
They fail specifically when the task requires word-learning They do know the sounds But they fail to use the detail needed for minimal pairs to store words in memory !!??
51
One-Year Olds Again One-year olds know the surface sound patterns of the language One-year olds do not yet know which sounds are used contrastively in the language… …and which sounds simply reflect allophonic variation One-year olds need to learn contrasts
52
Maybe not so bad after all...
Children learn the feature contrasts of their language Children may learn gradually, adding features over the course of development Phonetic knowledge does not entail phonological knowledge Roman Jakobson,
53
Summary of Development
Memory Surface 10 months Memory 18 months Constructed Lexical Surface Auditory Phonetic Articulatory Innate
54
Outline Categories & Abstraction Speech Perception in Infancy
Electrophysiology: Mismatch Paradigm Speech Discrimination in Adult Brains Speech Categorization in Adult Brains Conclusion
55
KIT-Maryland MEG System (160 SQUID detectors)
Brain Magnetic Fields (MEG) KIT-Maryland MEG System (160 SQUID detectors)
56
Brain Magnetic Fields (MEG)
SQUID detectors measure brain magnetic fields around 100 billion times weaker than earth’s steady magnetic field.
57
pickup coil & SQUID assembly
160 SQUID whole-head array
58
It’s safe…
59
Origin of the signal - noninvasive measurement - direct measurement.
skull CSF tissue MEG EEG B orientation of magnetic field recording surface scalp current flow - noninvasive measurement - direct measurement.
60
How small is the signal? Biomagnetism Intensity of magnetic signal(T)
Earth field EYE (retina) Steady activity Evoked activity LUNGS Magnetic contaminants LIVER Iron stores FETUS Cardiogram LIMBS Steady ionic current BRAIN (neurons) Spontaneous activity Evoked by sensory stimulation SPINAL COLUMN (neurons) HEART Cardiogram (muscle) Timing signals (His Purkinje system) GI TRACK Stimulus response Magnetic contaminations MUSCLE Under tension Intensity of magnetic signal(T) Urban noise Contamination at lung Heart QRS Biomagnetism Fetal heart Muscle Spontaneous signal (a-wave) Signal from retina Evoked signal Intrinsic noise of SQUID
61
Electroencephalography (EEG/ERP)
62
Event-Related Potentials (ERPs)
John is laughing. s1 s2 s3
63
Mismatch Response X X X X X Y X X X X Y X X X X X X Y X X X Y X X X...
64
Mismatch Response X X X X X Y X X X X Y X X X X X X Y X X X Y X X X...
65
Mismatch Response X X X X X Y X X X X Y X X X X X X Y X X X Y X X X...
Latency: msec. Localization: Supratemporal auditory cortex Many-to-one ratio between standards and deviants
66
Localization of Mismatch Response
(Phillips, Pellathy, Marantz et al., 2000)
67
Basic MMN elicitation © Risto Näätänen
68
Basic MMN elicitation MMN P300 Näätänen et al. 1978
69
MMN Amplitude Variation
Sams et al. 1985
70
How does MMN latency, amplitude vary with frequency difference
How does MMN latency, amplitude vary with frequency difference? 1000Hz tone std. Tiitinen et al. 1994
71
Different Dimensions of Sounds
Length Amplitude Pitch …you name it … Amplitude of mismatch response can be used as a measure of perceptual distance
72
Outline Categories & Abstraction Speech Perception in Infancy
Electrophysiology: Mismatch Paradigm Speech Discrimination in Adult Brains Speech Categorization in Adult Brains Conclusion
73
‘Vowel Space’
74
Näätänen et al. (1997) e e/ö ö õ o
75
Place of Articulation [bæ] Formant Transition Cues [dæ]
76
Place of Articulation Sharma et al. 1993
77
When does Change Occur? Hindi and Salish contrasts tested on English kids Janet Werker U. of British Columbia
78
6-12 Months: What Changes?
79
Structure Changing Patricia Kuhl U. of Washington
80
Place of Articulation Non-native continuum b -- d -- D
3 contrasts Native b -- d Non-native d -- D Non-phonetic b1 -- b5 Conflicting results!
81
Place of Articulation Non-native continuum b -- d -- D
3 contrasts Native b -- d Non-native d -- D Non-phonetic b1 -- b5 Conflicting results! Dehaene-Lambertz 1997
82
Place of Articulation Non-native continuum b -- d -- D
3 contrasts Native b -- d Non-native d -- D Non-phonetic b1 -- b5 Conflicting results! Rivera-Gaxiola et al. 2000
83
Place of Articulation Non-native continuum b -- d -- D
3 contrasts Native b -- d Non-native d -- D Non-phonetic b1 -- b5 Conflicting results! Tsui et al. 2000
84
Interim Conclusion MMN/MMF is a sensitive measure of discrimination
In some - but not all - cases, MMN amplitude tracks native language discrimination patterns When MMN fails to show native language category effects… could reflect that MMN accesses only low-level acoustic representations could reflect that MMN accesses multiple levels of representation, but response is dominated by acoustic representation These studies all implicate phonetic categories
85
Outline Categories & Abstraction Speech Perception in Infancy
Electrophysiology: Mismatch Paradigm Speech Discrimination in Adult Brains Speech Categorization in Adult Brains Conclusion
86
Objective Isolate phonological categories, not phonetic categories
87
A Category
88
Another Category 3 III
89
Auditory Cortex Accesses Phonological Categories: An MEG Mismatch Study
Colin Phillips, Tom Pellathy, Alec Marantz, Elron Yellin, et al. Journal of Cognitive Neuroscience, 2000
90
Voice Onset Time (VOT) 60 msec
91
Categorical Perception
92
Design Fixed Design - Discrimination 20ms 40ms 60ms 0ms 8ms 16ms 24ms
Grouped Design - Categorization
96
Phonological Features in Auditory Cortex
Colin Phillips Tom Pellathy Alec Marantz
97
Sound Groupings (Phillips, Pellathy & Marantz 2000)
98
Phonological Features
Phonological Natural Classes exist because... Phonemes are composed of features - the smallest building blocks of language Phonemes that share a feature form a natural class Effect of Feature-based organization observed in… Language development Language disorders Historical change Synchronic processes Roman Jakobson,
99
Voicing in English
100
Japanese - Rendaku s z ts z t d k g f b [-voice] [+voice]
take + sao takezao cito + tsuma citozuma hon + tana hondana yo: + karasi yo:garasi asa + furo asaburo Second member of compound word s z ts z t d k g f b [-voice] [+voice]
101
Sound Groupings in the Brain
pæ, tæ, tæ, kæ, dæ, pæ, kæ, tæ, pæ, kæ, bæ, tæ... (Phillips, Pellathy & Marantz 2000)
102
Sound Groupings in the Brain
pæ, tæ, tæ, kæ, dæ, pæ, kæ, tæ, pæ, kæ, bæ, tæ... (Phillips, Pellathy & Marantz 2000)
103
Feature Mismatch: Stimuli
(Phillips, Pellathy & Marantz 2000)
104
Feature Mismatch Design (Phillips, Pellathy & Marantz 2000)
105
Sound Groupings in the Brain
pæ tæ tæ kæ dæ pæ kæ tæ pæ kæ bæ tæ ... (Phillips, Pellathy & Marantz 2000)
106
Sound Groupings in the Brain
pæ tæ tæ kæ dæ pæ kæ tæ pæ kæ bæ tæ ... – – – – [+voi] (Phillips, Pellathy & Marantz 2000)
107
Sound Groupings in the Brain
pæ tæ tæ kæ dæ pæ kæ tæ pæ kæ bæ tæ ... – – – – [+voi] – – – – – [+voi] – … (Phillips, Pellathy & Marantz 2000)
108
Sound Groupings in the Brain
pæ tæ tæ kæ dæ pæ kæ tæ pæ kæ bæ tæ ... – – – – [+voi] – – – – – [+voi] – … Voiceless phonemes are in many-to-one ratio with [+voice] phonemes No other many-to-one ratio in this sequence Notice, there is no many-to-one ratio in this sequence unless we appeal to features, here [+voice] So, it means that if there is a MM between devi and standards, it’s due to the feature [+voice]. In other words, if your brain is surprised to head B’s, d’s and g’s, it’s because these sounds were rare and because all of them share this feature [+voice] that was absent in the frequent/standard sounds. (Phillips, Pellathy & Marantz 2000)
109
Feature Mismatch (Phillips, Pellathy & Marantz 2000)
110
Feature Mismatch Left Hemisphere Right Hemisphere
(Phillips, Pellathy & Marantz 2000)
111
Control Experiment - ‘Acoustic Condition’
Identical acoustical variability No phonological many-to-one ratio (Phillips, Pellathy & Marantz 2000)
112
Feature Mismatch (Phillips, Pellathy & Marantz 2000)
113
Hemispheric Contrast in MMF
Studies of acoustic and phonetic contrasts consistently report bilateral mismatch responses Paavilainen, Alho, Reinikainen et al. 1991; Näätänen & Alho, 1995; Levänen, Ahonen, Hari et al. 1996; Alho, Winkler, Escera et al. 1998; Ackermann, Lutzenberger & Hertrich, 1999; Opitz, Mecklinger, von Cramon et al. 1999, etc., etc. Striking difference in our finding of a left-hemisphere only mismatch response elicited by phonological feature contrast Our studies probe a more abstract level of phonological representation
114
Colin Phillips Shani Abada Daniel Garcia-Pedrosa Nina Kazanina
EEG Measures of Discrimination and Categorization of Speech Sound Contrasts Colin Phillips Shani Abada Daniel Garcia-Pedrosa Nina Kazanina
115
Design Fixed Design - Discrimination 20ms 40ms 60ms 0ms 8ms 16ms 24ms
Grouped Design - Categorization
116
Voice Onset Time (VOT) Mismatch Negativity (MMN): An ERP study
MMN: acoustic or perceptual phenomenon? Does an across-category distinction (20ms VOT /da/, 40ms VOT /ta/ evoke a greater MMN than a within-category distinction (40ms VOT /ta/, 60ms VOT /ta/)? Sharma & Dorman (1999): MMN only across categories; MMN represents perceptual, not physical, difference between stimuli; Double N100 for long VOT ‘Oddball’ paradigm - 7:1 ratio of standards to deviants Fixed condition standard VOT = 20ms, deviant VOT = 40 ms (across), or standard VOT = 40ms, deviant VOT = 60ms (within) Grouped condition no specific standard VOT, but 7/8 fall into either /da/ or /ta/ 20st,40dv st,60dv Dst,Tdv Tst,Ddv
117
Discrimination and Categorization of Vowels and Tones
Daniel Garcia-Pedrosa Colin Phillips
118
Two Concerns Are the category effects an artifact:
it is very hard to discriminate different members of the same category on a voicing scale subjects are forming ad hoc groupings of sounds during the experiment, and are not using their phonological representations?
119
Discrimination A More Systematic Test D D D T T T Same/Different
0ms ms 0ms 20ms D 20ms 40ms T Same/Different 0ms ms T T 40ms 60ms Same/Different Within-Category Discrimination is Hard 40ms 40ms
120
Vowels Vowels show categorical perception effects in identification tasks …but vowels show much better discriminability of within-category pairs
121
Method: Materials Tones: 290Hz, 300Hz, 310 Hz…470Hz Vowels
First formant (F1) varies along the same Hz continuum F0, F2, voicing onset, etc. all remain constant
122
Method: Procedure Subject’s category boundary determined by pretest
Grouped mismatch paradigm Standard stimulus (7/8) = 4 exemplars from one category Deviant stimulus (1/8) = 4 exemplars from other category MMN response therefore = deviance from a category, not from a single stimulus Tones and vowels presented in separate blocks
123
Results: Vowels
124
Results: Vowels
125
Results: Tones
126
Results: Tones
127
Preliminary conclusions
MMN appears about 150ms post-stimulus in vowel but not in tone condition Higher amplitude N100 for deviants in both conditions. Is this evidence for categorization of tones or just the result of habituation? Acoustic differences may be responsible for greater N100, while categorization elicits the MMN
128
Phonemic vs. Allophonic Contrasts
Nina Kazanina Colin Phillips in progress
129
Cross-Language Differences
Focus on meaning-relevant sound contrasts Russian d t Korean d t
130
Cross-Language Differences
Focus on meaning-relevant sound contrasts Russian d t Korean d t
131
Cross-Language Differences
Focus on meaning-relevant sound contrasts Russian d t Korean d t …ada ada ada ada ada ada ata ada ada ada ata…
132
EXECTIVE SUITE
133
Phonology - Syllables Japanese versus French
Pairs like “egma” and “eguma” Difference is possible in French, but not in Japanese
134
Behavioral Results Japanese have difficulty hearing the difference
Dupoux et al. 1999
135
ERP Results Sequences: egma, egma, egma, egma, eguma
French have 3 mismatch responses Early, middle, late Japanese only have late response Dehaene-Lambertz et al. 2000
136
ERP Results - 2 Early response Dehaene-Lambertz et al. 2000
137
ERP Results - 3 Middle response Dehaene-Lambertz et al. 2000
138
ERP Results - 4 Late response Dehaene-Lambertz et al. 2000
139
Cross-language Differences
Thai speakers: Thai *words*: [da] [ta] DIFFERENT English *words*: [daz] [taz] SAME Imsri (2001)
140
Varying Pronunciations
Voiceless stops /p, t, k/ Aspirated at start of syllable; unaspirated after [s] pit spit bit tack stack dack
141
Outline Categories & Abstraction Speech Perception in Infancy
Electrophysiology: Mismatch Paradigm Speech Discrimination in Adult Brains Speech Categorization in Adult Brains Conclusion
142
Conclusion Sound representations involve (multiple degrees of) abstraction Different levels of representation develop independently from 0-18 months of age Although much is known about course of development, many open questions about how change proceeds Possibility of a connection between adult electrophysiology and infant developmental findings
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.