Effects of Mispronunciation on Spoken Word Recognition in Cochlear Implant Users & Typically Hearing Listeners T. Ellis 1, K. Apfelbaum 2, H. L. Rigler 1, M. Seedorff 3, B. McMurray 1,4 1 Department of Psychological and Brain Sciences, University of Iowa 2 Department of Psychology, Ohio State University 3 Department of Biostatistics, University of Iowa 4 Delta Center Background Methods Effect of Mispronunciation Participants: 53 post-lingually deafened CI users (mean age 52.2y) 36 typically hearing age-matched controls (mean age 52.1y) Subjects excluded if accuracy fell below 90% on correctly pronounced trials (N=3) Auditory stimuli: Presented over loudspeakers in sound-attenuated booth 40 CVC English words 4 mispronounced tokens of each word 2 degrees of mispronunciation (single feature or multi-feature) 2 places of mispronunciation (onset or offset) 320 randomized trials Procedure: 4 AFC, visual world paradigm On screen: Target image + 3 phonetically unrelated images Task: Hear a token, click on visual referent Measure fixations to the target for trials on which correct referent is selected Statistical Approach: Looks to target as a function of time for correct trials only. Bootstrapped Difference of Time Series (bdots): Estimates time window over which two time series are different (with appropriate correction for multiple comparisons). (Olseon et al., in press). How much does mispronunciation impede word recognition? Mispronunciation effect: difference in target fixations for correct and incorrect pronunciations. Speech information arrives sequentially. At early points in time, signal is temporarily ambiguous. Listeners immediately begin activating many lexical candidates. Visual World Paradigm (VWP) Speech is highly variable within and between talkers (e.g., rate, prosody, coarticulation, production errors). Mispronunciations influence the lexical competition dynamic, and are a useful experimental tool for measuring how people respond to variable input. bakery Hear: “ba… basic barrier barricade bait baby Onset mispronunciation ----- CI ----- CI Single feature Multi-feature TH Mispronunciation effect 200 ms 1 2 3 4 5 Trials Time % fixations Target Cohort Rhyme Unrelated p = 0.0018 p = 0.0021 Time (msec) Time (msec) Onset MP Offset MP Target Single feature Multi-feature beach deach heach beaj beag fish sish nish fiss fid jet chet thet jek jech Offset mispronunciation Mispronunciation effect p = 0.0008 ----- CI Mispron. Type Time windows detected (msec) Single feature onset 432-896; > 1136 Multi-feature onset 452-976; > 1496 Single feature offset > 1804 Multi-feature offset Not analyzed due to poor fit of data Single feature TH Swingley (2009) studied adults’ and children’s sensitivity to consonant mispronunciations in VWP. Target fixations time-locked to location of mispronunciation Time (msec) Early in processing Onset: CI users show less interference from mispronunciation. Offset: CI users show similar interference effects. Late in processing Onset: CI users show more interference. Offset: CI users show less interference Discussion & Future Directions People who use cochlear implants: Can perceive speech accurately despite degraded input. Develop compensatory listening strategies. Altered weighting of acoustic cues (Moberly et al., 2014). Modified time course of speech processing (Farris-Trimble et al,. 2014) Adult CI users are slower to fixate the target; fewer overall looks. CI users sustain looks to competitors longer than typically hearing listeners, suggesting a “wait and hedge bets” compensatory strategy. Results CI users are slower than TH listeners to activate correct target. Both groups show similar time course of processing for onset and offset MP  CI users show similar incremental processing to TH listeners. CI users’ processing delay allows more information to accumulate before making strong commitments: more flexible strategy for dealing with uncertain input.  They show less effect of mispronunciation and can recover better in some circumstances. Correctly Pronounced Condition CI ---- TH (msec) Proportion fixations Target Cohort Effect of CI use: Target fixations are delayed and reduced in CI users relative to TH listeners across conditions Significant group differences between 252 and 3000 msec (yellow interval). adult CI adult control Future analyses: within-group differences in time course of processing. Preliminary results suggest unexpected advantage for CI users without acoustic hearing, especially later during processing. Time p = 0.001 Question References Target fixations by location of mispronunciation Do CI users’ compensatory listening strategies simultaneously help overcome surface variation (mispronunciations) in the speech signal? Farris-Trimble, A., McMurray, B., Cigrand, N., & Tomblin, J. B. (2014). The process of spoken word recognition in the face of signal degradation. Journal of Experimental Psychology. Human Perception and Performance, 40(1), 308–27. doi:10.1037/a0034353 Moberly, A. C., Lowenstein, J. H., Tarr, E., Caldwell-Tarr, A., Welling, D. B., Shahin, A. J., & Nittrouer, S. (2014). Do adults with cochlear implants rely on different acoustic cues for phoneme perception than adults with normal hearing? Journal of Speech, Language, and Hearing Research, 57(3), 566–582. doi:10.1044/2014 Oleson, J. J., Cavanaugh, J. E., McMurray, B., & Brown, G. (in press). Detecting time-specific differences between temporal nonlinear curves: Analyzing data from the visual world paradigm. Statistical Methods in Medical Research Swingley, D. (2009). Onsets and codas in 1.5-year-olds’ word recognition. Journal of Memory and Language, 60, 252–269. doi:10.1016/j.jml.2008.11.003 Proportion of Fixations E-mail: tyler-p-ellis@uiowa.edu 2015 ASHA Convention