Correlating Consonant Confusability and Neural Responses: An MEG Study Valentine Hacquard 1 Mary Ann Walter 1,2 1 Department of Linguistics and Philosophy, KIT-MIT MEG Laboratory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA, 2 National Science Foundation Fellowship

Nasal Confusability Numerous behavioral studies have investigated error rates in identification of phonemes masked with noise. These show that nasal consonants are more confusable with each other than oral consonants; e.g. m/n vs b/d (Miller & Nicely 1955, Wang & Bilger 1973).

Consequences Such perceptibility asymmetries motivate much current work in phonology and (it’s claimed) phonological processes (Hume 1998, Steriade 1999). Unsurprisingly, given this stance, Mohanan (1993) finds that nasals are particularly susceptible to place assimilation.

An Acoustic Model These confusability and assimilation facts, as well as studies on offline similarity judgments (Hura et al. 1992), support a model of similarity based on acoustic, perceptual factors. context-dependent, but inventory-independent Although nasality itself is highly salient, persevering nasality masks the F2 transition into the following vowel, which is the primary cue for place of articulation.

Another Proposal Frisch et al. (1997) and Frisch (1996) propose a metric in which similarity is computed according to natural classes: Similarity = shared natural classes shared + unshared natural classes context-independent, but inventory-dependent English: b/d (=.29) > m/n (=.28), where 1 is identity. So, nasals equally or less similar to each other than orals.

The Question Does a difference in similarity correlate with a difference in an auditory brain response? If so, brain data can be used to substantiate proposals for similarity metrics, as well as their internal organization.

MAGNETOENCEPHALOGRAPHY (MEG) MEG measures the magnetic fields (B) generated by electrical activity in the brain: specifically, by potentials in the apical dendrites of pyramidal cells in cortex To input Coil of SQUID cortex Scalp

Earth field Urban noise Contamination at lung Heart （ QRS ） Muscle Fetal heart Spontaneous signal (  -wave) Signal from retina Intrinsic noise of SQUID Intensity of magnetic signal(T) Evoked signal SCALE The magnetic field of the brain, recorded with MEG, is 100 billion times weaker than the earth’s magnetic field!

MMF: mismatch field: automatic, auditory brain response evoked by a deviant stimulus following a sequence of standards, peaking ~ ms post-stimulus onset. MMF M100 M100: automatic auditory evoked response that peaks ~100 ms post-stimulus onset

Origin of signal in auditory cortex MMF LOCALIZATION Note left-hemisphere concentration of the mismatch field with linguistic stimuli.

Properties of the MMF Sharma & Dorman (1999) and Phillips et al. (2000) show that the same VOT span crossing a phonemic category boundary evokes a far greater MMF than one that doesn’t. Näätänen et al. (1997) show that a small acoustic difference crossing a phonemic category boundary evokes a far greater MMF than a large one that doesn’t. Phonological difference outweighs acoustic difference. Do similarity distinctions matter when category is kept constant? 10 ms VOT span within category 10 ms VOT span across category MMF A M P L I T U D E 50 HZ F2 span within category 10 HZ F2 span across category

Procedure Oddball paradigm: Conditions (8x30=240) 1) ba (400 ms) ba (400 ms) ba (400 ms) ba (400 ms) dadeviant 2) da (400 ms) da (400 ms) da (400 ms) da (400 ms) dastandard 3) da (400 ms) da (400 ms) da (400 ms) da (400 ms) badeviant 4) ba (400 ms) ba (400 ms) ba (400 ms) ba (400 ms) bastandard 5) ma (400 ms) ma (400 ms) ma (400 ms) ma (400 ms) nadeviant 6) na (400 ms) na (400 ms) na (400 ms) na (400 ms) nastandard 7) na (400 ms) na (400 ms) na (400 ms) na (400 ms) madeviant 8) ma (400 ms) ma (400 ms) ma (400 ms) ma (400 ms) mastandard Subjects (n=16) made same-different button-press judgments. Synthesized stimuli (4 tokens) were presented in six blocks of 40 trials, randomly ordered, with self-regulated breaks in between.

Predictions According to an acoustic-based similarity framework, the MMF-baseline gap should be larger for oral consonant pairs than for nasals. According to a natural-class-based one such as Frisch’s, it should be the opposite, or equivalent. If abstract phonological features are the only relevant factor in perceptibility at this stage, the gaps should be equivalent.

Behavioral Results – Error Rate Deviants overall received significantly more errors than standards (p=.0009). No effect for manner was observed (p=.3538).  Cf. lack of masking noise or filtering, comparatively small number of trials.

Behavioral Results – Reaction Time RT to deviants overall was significantly faster than to standards (p<0001). (Some subjects reported a waiting strategy in standard trials.) RT to nasals was significantly faster than to orals (p=.0197).

MMF Results Oral Standard Oral Deviant single subject

Deviants have significantly greater amplitude in MMF window than standards (p<.0001). Orals have significantly greater amplitude in MMF window than nasals (p<.0001). But nasals have significantly greater M100 amplitude than orals (p<.0001). MMF Results selected sensors RMS waves

Nasal Deviant Oral Deviant MMF Comparison: Nasal/Oral single subject

MMF Comparison The MMF/baseline gap is significantly greater for oral consonant pairs than for nasals (p=.0399). single subject

Conclusions Oral mismatches elicit a stronger MMF than nasal mismatches. So oral consonants are perceived as more different from each other than nasal ones at this stage. Phonological categories are not the only relevant factors in perception at this stage: acoustic similarity also plays a role. Finally, it is an acoustic-based similarity metric that appears to be operating at this time period, rather than a natural-class one, or feature-counting.

For Future Research Our results suggest that the MMF can be used to test proposals about degree of perceptual distance in acoustic-based similarity frameworks. Our next study will isolate Frisch-style similarity as a variable, testing the same phonological contrast with speakers of languages that have the contrast, but whose inventories differ in other ways.

THANK YOU ! Alec MarantzDiana Sonnenreich Donca SteriadeKaren Froud Linnaea StockallPranav Anand Ben Bruening, Elissa Flagg, Vivian Lin you, the audience