Danielle Werle Undergraduate Thesis Intelligibility and the Carrier Phrase Effect in Sinewave Speech

Sinewave Speech 3 sinusoids of the same center freq.’s of the lowest 3 formants in a speech sample. Not harmonically related – aperiodic Narrow formant peaks Weird.

Remez, Rubin, Pisoni, and Carrell (1981) CONDITION A Presented 18 listeners with the SW sentence, “Where were you a year ago?” 2 transcribed the utterance, about half chose speech-like answers

Remez et al. (1981) cont’d CONDITION B Presented the same sentence to 18 new listeners with the instruction that they were about to hear speech 9 listeners transcribed the sentence completely Several others transcribed part or most of the sentence

Remez et al. (1981) Authors concluded that even in the absence of traditional speech cues, listeners could phonetically perceive SWS. The instruction that they were about to hear speech allowed listeners to direct their attention to the phonetic properties of the sinusoidal signals.

Problem? Remez et al. (1981) did not measure SWS perception at the phonetic level. With complete sentences listeners may utilize their existing higher level knowledge of language to reconstruct unfamiliar utterances.

Hillenbrand, Clark, and Baer (2011) Comprehensive study of SW vowels. Listeners must rely exclusively on perceptual pattern- matching mechanisms operating at the phonetic level. Focused on 3 training procedures

Hillenbrand et al. (2011) cont’d 71 phonetically trained listeners Initial vowel intelligibility test 300 sinewave /hVd/ syllables Database from Hillenbrand, Getty, Clark, and Wheeler (1995) Spoken by 48 men, 45 women, and yr. old children Same geographical region as listeners

Hillenbrand et al. (2011) cont’d Feedback 180 sinewave /hVd/ syllables from Hillenbrand et al. (1995) Sentence Transcription 50 sinewave sentences from HINT Triad SW, NS, SW Control: Male/Female Identification Task

Hillenbrand et al. (2011) cont’d Results of vowel intelligibility post- test * Transcription accuracy for sentences was 89.6%

Hillenbrand et al. (2011) cont’d Second Experiment 12 phonetically trained listeners Sinewave /hVd/ with feedback If correct, listener moved on to the next trial If incorrect: Sinewave /hVd/ Natural /hVd/ Sinewave /hVd/

Hillenbrand et al. (2011) cont’d Intelligibility rates in experiment 2 increased 19.8 percentage points. While phonetic information may be conveyed through sinewave speech, results are still low compared to vowel intelligibility rates – 95.4% (Hillenbrand and Nearey, 1999)

Hillenbrand et al. (2011) cont’d Results of vowel intelligibility post- test * Transcription accuracy for sentences was 89.6%

Hillenbrand, Clark, Houde, K. Hillenbrand, and M. Hillenbrand (2012) Sentence v. syllable intelligibility discrepancy Higher linguistic knowledge Length allows for accommodation to weird acoustic characteristics

Hillenbrand et al. (2012) cont’d 103 phonetically trained listeners Test signals recorded by 10 men and 10 women of same geographic region “The next word on the list is /hVd/” 16 vowels

Hillenbrand et al. (2012) cont’d ConditionSignals Presented SWISSinewave /hVd/ syllables in isolation. ISNSNaturally spoken /hVd/ syllables in isolation. SWCP-SWIS-WT Sinewave /hVd/ syllables preceded by a sinewave carrier phrase replicated from the same speaker. NSCP – WT Naturally spoken /hVd/ syllables preceded by a naturally spoken carrier phrase from the same speaker. SWCP-SWIS-XT Sinewave /hVd/ syllables preceded by a sinewave carrier phrase of a randomly paired speaker. NSCP- XT Naturally spoken /hVd/ syllables preceded by a naturally spoken carrier phrase of a randomly paired speaker. NSCP-SWIS-WTSinewave /hVd/ syllables preceded by a naturally spoken carrier phrase of the same speaker.

Hillenbrand et al. (2012) cont’d RESULTS ISNS – 95.1% NSCP – WT – 96.5% NSCP – XT – 95.2%

Hillenbrand et al. (2012) cont’d

Experiment 2 New set of 23 phonetically trained listeners 10 blocks of 32 trials Presented with alternating utterances from SWIS and SWCP

Hillenbrand et al. (2012) cont’d

Carrier Phrase Effect Does the carrier phrase need to be intelligible in order to increase sinewave vowel intelligibility? OR Does the listener need to accommodate to the unusual acoustical characteristics of the sinewave signal?

The Study 43 phonetically trained participants Pre-trial – naturally spoken /hVd/ vowels ‘heed’, ‘hawd’, ‘hayed’, etc. 16 vowel choices Six conditions

Conditions ISNS (control) – Natural speech recordings of /hVd/ vowels ISSS – Sinewave synthesized /hVd/ vowels presented in isolation

Conditions, cont’d CP-E - Sinewave synthesized /hVd/ vowels preceded by a consistent sinewave synthesized carrier phrase “the next word on the list is…” CP-J- Sinewave synthesized (English) vowels preceded by the same consistent carrier phrase, except in Japanese.

Conditions, cont’d HINT-E - vowels preceded by one of 240 sinewave synthesized sentences in the Hearing in Noise Test database HINT-J - vowels preceded by one of 240 sinewave synthesized sentences in the Hearing in Noise Test database, except in Japanese

99% 77% 61% 52% 50%48% 99% 52% 77% 50% 48.6% 61%

Japanese v. English Simply having time to accommodate to a weird sound with unique acoustical characteristics was not enough to increase intelligibility of the following vowels

HINT v. Fixed CP HINT-E = more intelligible than Japanese Fixed CP = most intelligible 1 CP v. 240 CP More intelligible = Greater /hVd/ scores

Why is Intelligibility Important? Results show – an intelligible CP enhances vowel intelligibility scores aides in unconscious signaling to speech processors 99%-77% - What else is helping? Not acoustical accommodation

References Assmann, P., Nearey, T., and Hogan, J. (1982). Vowel identification: Orthographic, perceptual, and acoustic aspects. Journal of the Acoustical Society of America, 71, Hillenbrand, J., Getty, L. A., Clark, M. J., & Wheeler, K. (1995). Acoustic characteristics of American English vowels. Journal of the Acoustical Society of America, 97(5 I), Hillenbrand, J. M., Clark, M. J., & Baer, C. A. (2011). Perception of sinewave vowels. Journal of the Acoustical Society of America, 129(6), Hillenbrand, J. M., Clark, M. J., & Houde, R. A. (2000). Some effects of duration on vowel recognition. Journal of the Acoustical Society of America, 108(6), Hillenbrand, J. M., Clark, M. J., Hillenbrand, M. W., Hillenbrand, K. S., & Hourde, R. A. (2012) Perceptual accommodation to sinewave speech. In preparation. Hillenbrand, J. M., Houde, R. A., & Gayvert, R. T. (2006). Speech perception based on spectral peaks versus spectral shape. Journal of the Acoustical Society of America, 119(6), Hillenbrand, J. M., & Nearey, T. M. (1999). Identification of resynthesized /hVd/ utterances: Effects of formant contour. Journal of the Acoustical Society of America, 105(6), Macleod, A., & Summerfield, Q. (1987). Quantifying the contribution of vision to speech perception in noise. British Journal of Audiology, 21(2), Nilsson, M., Soli, S. & Sullivan, J. A. (1994). Development of the hearing in noise test for the measurement of speech reception thresholds in quiet and noise. Journal of the Acoustical Society of America, 95(2),

References cont’d Remez, R. E., Rubin, P. E., Nygaard, L. C., & Howell, W. A. (1987). Perceptual normalization of vowels produced by sinusoidal voices. Journal of Experimental Psychology: Human Perception and Performance, 13(1), Remez, R. E., Rubin, P. E., Pisoni, D. B., & Carrell, T. D. (1981). Speech perception without traditional speech cues. Science, 212(4497),