Effectiveness of Visual Biofeedback in Speech Training of Children with Hearing Impairment Elizabeth Reid, BSLT and Emily Lin, PhD Introduction It is well known that the limitations of a hearing-impaired child’s perceptual system can prevent them from perceiving differences in sounds, resulting in speech production that is delayed or disordered (Ruffin-Simon, 1983). To compensate for this lack of access to auditory cues, there has been a substantial increase in the development of real-time visual feedback displays such as spectrograms. Sspectrograms provide a visual representation of the frequency, intensity, and time domains of an acoustic signal (Ertmer & Maki, 2000). Unlike many other visual feedback devices that provide feedback on a single dimension of speech, spectrographic displays can provide many segmental and suprasegmental speech features simultaneously. Spectrographic displays (SDs) provide immediate and objective feedback, allowing a child to compare his/her own speech production with a correct visual template from the clinician (Dagenais, Critz-Crosby, Fletcher & McCutheon, 1994). Despite the growing interest in visual feedback tools, there have been few studies that have objectively examined the effectiveness of such devices. More research on their effectiveness is needed before they are accepted by clinicians as an effective treatment tool. Therefore the main objective of this study was to evaluate, using objective measures, the effectiveness of spectrograms compared to traditional speech training approaches for hearing- impaired children. The second objective of the study was to describe the temporal behaviour and formant characteristics of speech produced by hearing-impaired children and examine how the acoustic properties are related to the perceived accuracy of their speech production. The majority of studies describing the speech production of hearing-impaired children has been confined to perceptual analysis of phonetic and phonologic errors and acoustic analyses of temporal aspects of the speech signal. A recent study by Uchanski & Geers (2003) used spectral moment analysis to examine the acoustic energy characteristics of fricatives spoken by hearing-impaired children. Their study provided an interesting basis for further exploration of hearing impaired children's’ consonant production. Department of Communication Disorders, University of Canterbury, Christchurch, New Zealand Method Subjects: 3 subjects (S1=12y; S2=9y; S3=7y) with bilateral moderate-severe sensorineural hearing losses. Instrumentation: Hheadset microphone (AKG C420, Austria), mixer (Eurorack MX602A, Behringer), 12- bit A/D converter (National Instrument DAQCard-AI-16E-4, USA), SCB pin shielded connector box, with a low-passed filter (cutoff frequency = 20 KHz), laptop installed with TF32 (Paul Milenkovic, 2000) & PRATT (Boersma & Weenink, 2005). Procedure: Recordings were done in a quiet room with the microphone 5 cm from the mouth. Initial recordings of the Goldman Fristoe Test of Articulation were obtained. Commonly occurring error processes were identified for each subject. Training targets were chosen (S1=Deletion of Final Consonant (DFC) and Consonant Cluster Reduction (CCR); S2=DFC; S3=DFC). Probe lists were developed for each target and were recorded throughout the training period. 30mins treatment sessions were carried out over 12 weeks for subject 1, 4 weeks for subject 2, and 2 weeks for subject 3. Subject 1 received traditional therapy followed by visual therapy; subjects 2 and 3 received visual therapy only. Traditional therapy involved verbal instruction with visual & tactile cues. Visual therapy used spectrogram displays of a correct production which the subjects were required to match using real-time pitch and intensity displays, and then judge their accuracy. (picture ***) Subjective analysis: phonemic transcriptions of each recording. Acoustic analysis: vowel and consonant lengths, F1 and F2, and spectral moments 1(mean, indicating …), 2 (standard deviation, indicating …), 3 (skewness, indicating …) and 4 (kurtosis indicating…). Statistical Analysis: Abstract The effectiveness of spectrograms in speech training of hearing-impaired children was examined and compared to traditional therapy approaches. Subjective and objective analyses suggested that spectrograms were effective in improving particular speech targets. The temporal and spectral properties of speech produced by the subjects were also examined and acoustic cues were identified which were related to the perceived accuracy of their speech productions. These results have the potential to provide clues to the type of compensatory feedback needed in therapy. Discussion Individually, all three subjects showed positive but different effects of training with spectrograms. The acoustic measures were more sensitive than subjective measures in identifying changes and highlighting differences in training approaches. VOT for all three subjects reduced over the training period. VOT length provides an important cue for the phonemic contrast between voiced plosives and their voiceless counterparts. The distinction requires fast movements of the articulators and good coordination of motor control between the larynx and upper articulators. Therefore the reduction in VOT indicates that visual training has improved all subjects’ coordination of phonation and articulation, which is likely to result in improved intelligibility. Temporal measures showed an increase in consonant cluster length for the trained target /fl/ for subject 1, but no improvement for the untrained target /pr/. This suggests that subject 1’s awareness and production of the two components of the consonant cluster has improved, however further treatment is necessary to facilitate generalisation to other consonant clusters. Subject 2, showed an increase in final consonant length over the training period suggesting an improved awareness and production of final consonants. Conversely, These results suggest that visual training is effective in improving subjects’ awareness of the targets and their production accuracy. Subject 3 showed a negative decrease in final consonant length. This may be due to the small number of measures taken or the fact that he only received one session of training. Although vowels were not targeted, subjects 1 & 2 showed an increase in vowel space following visual training. This appeared to be largely due to the improved production range of Formant 2 for S1 and Formant 1 for S2. A reduced vowel space area represents a restriction of tongue elevation and front-back tongue movement (Liu Tso & Kuhl, 2005). Therefore the improved vowel space following training suggests that subjects 1 & 2 were producing a greater range of formant frequencies, resulting in greater distinction between vowels. Subject 3 showed a decrease in vowel space following the training period, which may be due to the shorter training period he experienced compared to the other two subjects. A number of acoustic properties were found which differentiated the correct and incorrect speech productions. Perception of vowel accuracy was found to be related to an increased vowel space as well as shorter vowel durations. Researchers (Monsen, 1974; Gulian et al., 1983) have identified vowel prolongation as one of the speech characteristics of the hearing-impaired. In this study, vowel durations for incorrect productions were prolonged compared to those for correct productions. A smaller vowel space was seen for incorrect productions, indicating a more restricted articulation range than for correct productions. This result is similar to Angelocci et al.’s (1964) comparison between hearing-impaired and normal-hearing speakers in that the vowel space derived from normal data was larger than that from the abnormal comparison groups. This result suggests that training aimed at the expansion of vowel space could be potentially beneficial to improve the speech intelligibility of hearing-impaired children. Perception of consonant accuracy was most closely related to VOT for plosives, and moment 1 (mean) and moment 3 (skewness) for fricatives, affricates and plosives. Correct plosive consonant productions contained a normal range of VOT measures, however incorrect productions were more variable and many were prolonged outside these ranges. As discussed previously, VOT is an important cue for the voiced-voiceless distinction. These results show that a reduced VOT improves perceptual intelligibility of speech production. M1 values for incorrect consonant productions tended to be much lower than those for correct productions, suggesting that tongue placement was more posterior in incorrect productions. Since the M1 measure appeared to be sensitive in differentiating correct and incorrect consonant productions, it could be used in clinical application to provide feedback in speech training and monitor progress. *** other moments?? Acknowledgements: This research is part of a Masters thesis which is currently being completed by the first author and directed by the second author at the University of Canterbury. Support for this research was provided by the Oticon Foundation New Zealand. Moment 1 (mean) and Moment 2 (standard deviation) for consonant productions perceived as correct vs incorrect. Vowel space for correct and incorrect vowel productions. Results A total of 180 values (3 pitch levels X 2 vowels X 3 groups X 10 subjects) for each measure were submitted to a one-way Analysis of Variances (ANOVA) to determine whether the three subject groups differed on each measure. An increase in vowel space was seen for subjects 1 & 2 following the training period, while subject 3 showed a slight decrease in vowel space. The increase for subject 1 was attributed most to an increase in the range of F2 productions, while the increase for subject 2 was due to the increase in the range of F1 productions. Calculation of the vowel working space area encompassing /i/, /a/, and /u/ showed a smaller working space area for incorrect productions than for correct productions. There was a reduction in vowel space for subject 3, which may have been due to the small number of recordings taken. Minimal change was seen with the Goldman Fristoe recordings, which was likely due to the small number of tokens for each target in the recording. For the probe list, Subject 1 showed no improvement in target processes with traditional training, however a clear trend of improvement was seen with visual training. The Goldman Fristoe recordings for Subjects 2 and 3 showed minimal change in target accuracy scores over the training period, which was likely due to the small number of tokens as well as their high accuracy scores pre-training. Percentage of deletion of final consonant targets correct for subjects 2 & 3 Percentage of targets correct for subject 1 Measures of VOT displayed a downward trend for all three subjects, indicating reduced VOT over the training period. For subject 1, a reduction in VOT was seen immediately with traditional training, however the trend was variable making comparisons between training approaches difficult. VOT for subjects 2 & 3 Voice Onset Time for subject 1 Subject 1 showed an increase in consonant cluster length for the trained /fl/ target with traditional training. During the visual training period, the length was maintained at a similar level with a slight drop in length over the period. Measures for the untreated control were variable over the training period suggesting no treatment effect. Final consonant length for subjects 2 showed a positive upward trend over the training period, however improvement were not maintained in the follow-up recording, indicating lack of maintenance. For subject 3, only three measures were taken of final consonant length, which showed a reduction in length, however the small number of recordings is likely to affect reliability. Consonant cluster length for subject 1 Final consonant length for subjects 2 & 3 Vowel space pre & post-treatment for each subject Those vowel productions perceived to be correct (ABS = ) had larger vowel spaces compared to those perceived as incorrect (ABS = ). Most incorrect consonant productions consistently exhibited lower M1 values than correct consonant productions, which covered a greater frequency range. All fricatives had M1 values lower than those reported for normal hearing speakers Fry (2001). Iincorrectly produced fricatives exhibited lower M2 values and incorrectly produced plosives higher M2 values than those of their correct counterparts, indicating that incorrectly produced fricatives and plosives tended to deviate from a normal pattern, Conclusion Investigation of the effectiveness of spectrographic displays suggested that spectrograms can enhance the awareness and improve the production of particular speech targets that children with hearing impairment would otherwise miss with traditional training. Results of the acoustic-perceptual investigation highlighted the usefulness of acoustic analysis in establishing a link between the hearing-impaired children’s production and perceptual deficits and thus providing clues to the type of compensatory feedback needed for aural rehabilitation. Results also emphasize the importance of using acoustic measures in research, as they are able to provide more detailed information and more sensitive to changes compared to subjective measures. Vowel Space Pre- Training Post- Training Demaris Blake Jack Final consonant length for subjects 2 & 3

Final consonant length Consonant cluster length % targets correct for subject 1 % DFC targets correct for subjects 2 &3 VOT for subjects 2 & 3 VOT Subject 1