1. Levels of Representation in Adult Speech Perception

3. The Big Questions
What levels of acoustic/phonetic/phonological representation can we distinguish in the brain?
How are these representations created or modified during development?
What is the flow of information (in space and time) in the mapping from acoustics to the lexicon in the brain?
How does knowledge of native language categories and phonotactics constrain perception?
How are phonological representations encoded?

4. /kæt/

5. A Category

6. Another Category 3
III

7. Types of Category Phonetic categories Phonological categories
Islands of acoustic consistency
Graded internal structure matters
Not good for computation
Phonological categories
Differences among category members are irrelevant
Good for computation
May correspond to complex acoustic distribution

8. /kæt/ Gradient Category Representations
Discrete Category Representations

9. Sensory Maps
Internal representations of the outside world. Cellular neuroscience has discovered a great deal in this area.

10. Vowel Space
Notions of sensory maps may be applicable to some aspects of human phonetic representations…
…but there’s been little success in that regard, and we shouldn’t expect this to yield much.

11. /kæt/ Gradient Category Representations
Discrete Category Representations

12. Phonological Categories are Different
Decisions about how to categorize a sound may be fuzzy, etc.
But phonological processes are blind to this
We don’t find gradient application of phonological transformations
Partial epenthesis
Gradient stress
etc.
Developmental dissociations of category types

13. Some Established Results
Search for phonetic ‘maps’ in the brain: consistently uninformative
Electrophysiology of speech perception has been dominated by studies of the Mismatch Negativity (MMN), a response elicited in auditory cortex ms after the onset of an oddball sound
MMN amplitude tracks perceptual distance between standard and deviant sound; i.e. measure of similarity along many dimensions
There are established effects and non-effects of linguistic category structure on the MMN
non-effects in comparison of within/across category contrasts
real effects in comparison of native/non-native contrasts

14. Electroencephalography (EEG)

15. Event-Related Potentials (ERPs)
John is laughing.
s1 s2 s3

16. Magnetoencephalography
pickup coil & SQUID assembly
160 SQUID whole-head array

17. Brain Magnetic Fields (MEG)
SQUID detectors measure brain magnetic fields around 100 billion times weaker than earth’s steady magnetic field.

18. Evoked Responses

19. M100 Elicited by any well-defined onset Varies with tone frequency
Varies with F1 of vowels
May vary non-linearly with VOT variation
Functional value of time-code unclear
No evidence of higher-level representations
(Poeppel & Roberts 1996)
(Poeppel, Phillips et al. 1997)
(Phillips et al. 1995; Sharma & Dorman 1999)

20. Mismatch Response
X X X X X Y X X X X Y X X X X X X Y X X X Y X X X...

21. Mismatch Response
X X X X X Y X X X X Y X X X X X X Y X X X Y X X X...

22. Mismatch Response X X X X X Y X X X X Y X X X X X X Y X X X Y X X X...
Latency: msec.
Localization: Supratemporal auditory cortex
Many-to-one ratio between standards and deviants

23. Localization of Mismatch Response
(Phillips, Pellathy, Marantz et al., 2000)

24. Basic MMN elicitation
© Risto Näätänen

25. MMN Amplitude Variation
Sams et al. 1985

26. How does MMN latency, amplitude vary with frequency difference
How does MMN latency, amplitude vary with frequency difference? 1000Hz tone std.
Tiitinen et al. 1994

27. Different Dimensions of Sounds
Length
Amplitude
Pitch
…you name it …
Amplitude of mismatch response can be used as a measure of perceptual distance

28. Impetus for Language Studies
If MMN amplitude is a measure of perceptual distance, then perhaps it can be informative in domains where acoustic and perceptual distance diverge…

29. Place of Articulation
Acoustic variation: F2 & F3 transitions

30. Place of Articulation [bæ] [dæ]
Acoustic variation: F2 & F3 transitions

31. Place of Articulation [bæ] [dæ]
within category
between category
[bæ]
[dæ]
Acoustic variation: F2 & F3 transitions

32. Place of Articulation [bæ] [dæ]
within category
between category
[bæ]
[dæ]
Acoustic variation: F2 & F3 transitions

33. Categories in Infancy High Amplitude Sucking - 2 month olds
Eimas et al. 1971
20 vs. 40 ms. VOT - yes 40 vs. 60 ms. VOT - no
Infants show contrast, but this doesn’t entail phonological knowledge

34. Place of Articulation
No effect of category boundary on MMN amplitude (Sharma et al. 1993)
Similar findings in Sams et al. (1991), Maiste et al. (1995)

35. but…

36. Näätänen et al. (1997)
e e/ö ö õ o

37. Phonetic Category Effects
Measures of uneven discrimination profiles
Findings are mixed (…and techniques vary)
Relies on assumption that effects of contrasts at multiple levels are additive, …plus the requirement that the additivity effect be strong enough to yield a statistical interaction
Logic of our studies:
Eliminate contribution of lower levels by isolating the many-to-one ratio at an abstract level of representations
Do this by introducing non-orthogonal variation among standards

38. Auditory Cortex Accesses Phonological Categories: An MEG Mismatch Study
Colin Phillips, Tom Pellathy, Alec Marantz, Elron Yellin, et al.
[Journal of Cognitive Neuroscience, 2000]

39. More Abstract Categories
At the level of phonological categories, within-category differences are irrelevant
Aims
use MMF to measure categorization rather than discrimination
focus on failure to make category-internal distinctions

40. Voice Onset Time (VOT)
60 msec

41. Design
Fixed Design - Discrimination
20ms
40ms
60ms

42. Design Fixed Design - Discrimination Grouped Design - Categorization
20ms
40ms
60ms
Grouped Design - Categorization
0ms
8ms
16ms
24ms
40ms
48ms
56ms
64ms
Non-orthogonal within-category variation: excludes grouping via acoustic streaming.

43. Design Fixed Design - Discrimination Grouped Design - Categorization
20ms
40ms
60ms
Grouped Design - Categorization
0ms
8ms
16ms
24ms
40ms
48ms
56ms
64ms
Grouped Design - Acoustic Control
20ms
28ms
36ms
44ms
60ms
68ms
76ms
84ms

45. /dæ/ standard vs. /dæ/ deviant

47. Discrimination vs. Categorization: Vowels
Daniel Garcia-Pedrosa Colin Phillips Henny Yeung

48. Some Concerns Are the category effects an artifact:
It is very hard to discriminate different members of the same category on a VOT scale
Perhaps subjects are forming ad hoc groupings of sounds during the experiment, not using their phonological representations?
Does the ~30ms VOT boundary simply reflect a fundamental neurophysiological timing constraint?

49. Vowels
Vowels show categorical perception effects in identification tasks
…but vowels show much better discriminability of within-category pairs

51. Vowels & Tones Synthetic /u/-/o/ continuum
F1 varied, all else constant
Amplitude envelope of F1 extracted for creation of tone controls
Pure tone continuum at F1 center frequency
Matched to amplitude envelope of vowel
Vowel, F1 = 310Hz
Pure Tone, 310Hz

52. Design
Tones
Vowels
First formant (F1) varies along the same Hz continuum
F0, F2, voicing onset, etc. all remain constant
300Hz
320Hz
340Hz
360Hz
400Hz
420Hz
440Hz
460Hz

53. Results: Vowels

54. Results: Vowels

55. Results: Tones

56. Results: Tones

57. Preliminary conclusions
Clear MMN in standard ms latency range in vowel but not in tone condition
Both vowels and tones yield larger N100 responses
Categorization effect for tones?
Response to rarity of individual deviant tones, without categorization?
Response to larger frequency changes when moving from standard to deviant category?

58. Phonological Features
Colin Phillips Tom Pellathy Henny Yeung Alec Marantz

59. Sound Groupings

60. Phonological Features
Phonological Natural Classes exist because...
Phonemes are composed of features - the smallest building blocks of language
Phonemes that share a feature form a natural class
Effect of Feature-based organization observed in…
Language development
Language disorders
Historical change
Synchronic processes
Roman Jakobson,

61. Sound Groupings in the Brain
pæ, tæ, tæ, kæ, dæ, pæ, kæ, tæ, pæ, kæ, bæ, tæ...

62. Sound Groupings in the Brain
pæ, tæ, tæ, kæ, dæ, pæ, kæ, tæ, pæ, kæ, bæ, tæ...

63. Feature Mismatch: Stimuli

64. Feature Mismatch
Design

65. Feature Mismatch

66. Control Experiment - ‘Acoustic Condition’
Identical acoustical variability
No phonological many-to-one ratio

67. Phoneme Variation: Features I
Alternative account of the findings
No feature-based grouping
Independent MMF elicited by 3 low-frequency phonemes
/bæ/
/dæ/
/gæ/
/pæ/
/tæ/
/kæ/
29%
29%
29% 4% 4% 4%
12.5%
87.5%

68. Phoneme Variation: Features II
Follow-up study distinguishes
Phoneme-level frequency
Feature-level status
/bæ/
/gæ/
/dæ/
/tæ/
37.5%
37.5%
12.5%
12.5%

69. Phoneme Variation: Features II
Follow-up study distinguishes
Phoneme-level frequency
Feature-level status
/bæ/
/gæ/
/dæ/
/tæ/
37.5%
37.5%
12.5%
12.5%
Phoneme-based classification

70. Phoneme Variation: Features II
Follow-up study distinguishes
Phoneme-level frequency
Feature-level status
/bæ/
/gæ/
/dæ/
/tæ/
37.5%
37.5%
12.5%
12.5%
Feature-based grouping

71. Phoneme Variation: Features II
Design
N = 10
Multiple exemplars, individually selected boundaries
2 versions recorded for all participants, reversing [±voice] value
Acoustic control, with all VOT values in [-voice] range
/bæ/
/gæ/
/dæ/
/tæ/
37.5%
37.5%
12.5%
12.5%
Feature-based grouping

72. Phoneme Variation: Features II
Left-anterior channels

73. Phoneme Variation: Features II
Left-anterior channels

74. Distinguishing Lexical and Surface Category Contrasts
Nina Kazanina Colin Phillips Bill Idsardi
Nina Kazanina, Univ. of Ottawa

75. Allophonic Variation
All studies shown so far fail to distinguish surface and lexical-level category representations (‘underlying’)
Phonological category ≠ Acoustic distribution

76. Allophonic Variation
All studies shown so far fail to distinguish surface and lexical-level category representations (‘underlying’)

77. Russian vs. Korean Three series of stops in Korean:
plain (lenis) pa ta ka
glottalized (tense, long) p’a t’a k’a
aspirated ph tha kha
Intervocalic Plain Stop Voicing:
/papo/ [pabo] ‘fool’
/ku papo/ [kbabo] ‘the fool’
Plain stops:
Bimodal distribution of +VOT and –VOT tokens
Word-initially: always a positive VOT
Word-medially intervocalically: a voicing lead (negative VOT)

79. Identification/
Rating
Discrimination

80. TA (voicing leads & lags)
MEG Stimuli
Russian (basic Russian [ta]-token: 00ms voicing lead, +13ms vowel lag):
DA (voicing leads)
TA (voicing leads & lags)
-40ms -34ms -28ms -24ms
-08ms -04ms +02ms +08ms (relative) -08ms -04ms +15ms +21ms (absolute)
Korean (basic Korean [ta]-token: 00ms voicing lead, +29ms vowel lag):
DA (voicing leads)
TA (voicing lags)
-40ms -36ms -30ms -24ms
00ms +07ms +11ms +15ms (relative)
+29ms +36ms +40ms +44ms (absolute)

81. Black: p < .05
White: n.s.

82. Russian vs. Korean
MEG responses indicate that Russian speakers immediately map sounds from [d-t] continuum onto categories
Korean speakers do not… … despite the fact that the sounds show bimodal distribution in their language
Perceptual space reflects the functional status of sounds in encoding word meanings

84. Basic understanding How strong is this
Adults are prisoners of their native language sound system
How strong is this
Structure-adding models predict residual sensitivity to non-native sounds
There is a great deal of motivation in L2 research to find ways to free perception from the constraints of L1

85. Phonology - Syllables Japanese versus French
Pairs like “egma” and “eguma”
Difference is possible in French, but not in Japanese

86. Behavioral Results Japanese have difficulty hearing the difference
Dupoux et al. 1999

87. EXECTIVE SUITE

88. ERP Results Sequences: egma, egma, egma, egma, eguma
French have 3 mismatch responses
Early, middle, late
Japanese only have late response
Dehaene-Lambertz et al. 2000

89. ERP Results - 2
Early response
Dehaene-Lambertz et al. 2000

90. ERP Results - 3
Middle response
Dehaene-Lambertz et al. 2000

91. ERP Results - 4
Late response
Dehaene-Lambertz et al. 2000

92. Implications
Cross-language contrast in MMN mirrors behavioral contrast
Relative timing of responses that are same and different across French & Japanese is surprising from a bottom-up view of analysis - suggests a dual route
Is this effect specific to comparison in an XXXXY task?
Is the result robust; does it generalize to other phonotactic generalizations?

93. What drives Perceptual Epenthesis?
Illegal syllables?
Illegal sequences of consonants?
(Kabak & Idsardi, 2004)

94. What drives Perceptual Epenthesis?
Korean syllables
Only [p, t, k, m, n, N, l] in coda
Other consonants neutralize in coda position
[c, c’, ch] --> [t] in coda
Voiced stops only in CVC environments (allophones of voiceless stops)
Korean contact restrictions
*C + N
Repair 1: nasalize C [path] + [ma] --> [panma]
Repair 2: denasalize N [tal] + [nala] --> [tallaPa]
Restrictions apply within IntPh
(Kabak & Idsardi, 2004)

95. What drives Perceptual Epenthesis?
(Kabak & Idsardi, 2004)

96. What drives Perceptual Epenthesis?
(Kabak & Idsardi, 2004)