PSY 369: Psycholinguistics Language Comprehension: Speech perception

Problem: Invariance Wave form

Problem: Invariance Invariance One phoneme should have a one waveform But, the /i/ (‘ee’) in ‘money’ and ‘me’ are different There aren’t invariant cues for phonetic segments Although the search continues

Motor theory of speech perception A. Liberman (and others, initially proposed in late 50’s) Direct perception of acoustic speech as articulatorally defined categories Holds that speech perception and motor control involved linked (or the same) neural processes Theory held that categorical perception was a direct reflection of articulatory organization Categories with discrete gestures (e.g., consonants) will be perceived categorically Categories with continuous gestures (e.g., vowels) will be perceived continuously

Motor theory of speech perception Implications of motor theory Held that “speech is special”: there is a speech perception module that operates independently of general auditory perception Under this view, the percepts of speech are not auditory, but articulatory Some experimental support for this claim But recent evidence suggests that that evidence may not have been properly controlled (Fowler and Rosenblum, 1990)

Categorical Perception Liberman et al (1957) Used the speech synthesizer to create a series of syllables panning categories /b/, /d/, /g/ (followed by /a/ Was done by manipulating the F2 formant Stimuli formed a physical continuum Result, people didn’t “hear” a continuum, instead classified them into three categories Research began at Haskins Labs not long after invention of the research-oriented speech synthesizer

Categorical Perception 1. Set up a continuum of sounds between two categories /ba/ - /da/ 1 ... 3 … 5 … 7

Categorical Perception 2. Run an identification experiment 100 Sharp phoneme boundary % /ba/ 1 ... 3 … 5 … 7

Phoneme restoration effect Listen to a sentence which contained a word from which a phoneme was deleted and replaced with another noise (e.g., a cough) The state governors met with their respective legi*latures convening in the capital city. * /s/ deleted and replaced with a cough Warren (1970) examined whether listeners noticed when a phoneme was missing from a word. In these experiments, subjects listened to a series of spoken sentences. Some of the sentences contained a word from which a phoneme had been deleted and replaced with a piece of extraneous noise. For example, when participants heard the word [legislature] the /s/ phoneme was removed and replaced with a cough. The state governers met with their respective legi*latures convening in the capital city. Example of sentence used by Warren (1970). The /s/ phoneme in legislatures was deleted  and replaced with a cough.  http://www.acsu.buffalo.edu/~bmb/Courses/PSY341/Exercises/Phon-rest/phon-rest.html

Phoneme restoration effect Typical results: Participants heard the word normally, despite the missing phoneme Usually failed to identify which phoneme was missing Interpretation We can use top-down knowledge to “fill in” the missing information Warren found that participants heard the words normally despite the missing phoneme. Moreover, only one participant reported a missing phoneme (but failed to identify which phoneme was missing). Warren referred to this finding as the phonemic restoration effect, as listeners appeared to have filled in the missing phoneme as they listened to the words.   Warren found that the effect persisted when a larger unit of sound was replaced. For instance, if the sounds corresponding to [gis] in the word [legislatures] was deleted and replaced with a cough, participants were still able to recognise the word and reported being unaware of the missing phonemes. These experiments demonstrate that knowledge about the likely spoken word can 'fill in' missing phoneme information.

Phoneme restoration effect Further experiments (Warren and Warren, 1970): What if the missing phoneme was ambiguous The *eel was on the axle. The *eel was on the shoe. The *eel was on the orange. In a further experiment,  Warren and Warren (1970) examined what happened when the word with the missing phoneme was ambiguous. For instance, if the initial /p/ phonemeof the spoken word [peel] is deleted to produce [*eel] it is ambiguous between several possible words, including peel, meal, seal, heel and steal. These sentences were presented in sentence contexts that were biased in favour of one of the possible forms of the ambiguoous word (see examples  to 11 below)   (7)    The *eel was on the axle. (8)    The *eel was on the shoe. (9)    The *eel was on the orange. (10)  The *eel was on the table. In each case, listeners reported hearing the word that was appropriate to the sentence contexts. For (7), listeners reported hearing the word wheel, for (8) they reported hearing heel, for (9) they reported hearing peel, and for (10) they reported hearing meal. The results of this experiment demonstrates that knowledge about the context in which an  ambiguous spoken word occurs also can 'fill in' missing phoneme information. These demonstrations of the phonemic restoration effect show that speech perception is an active, constructive process in which listeners do not simply attend to spoken information, but use lexical (word) and contextual knowledge to generate hypotheses about the likely form of the spoken information.  The *eel was on the table. Results: Participants heard the contextually appropriate word normally, despite the missing phoneme

Phoneme restoration effect Possible loci of phoneme restoration effects Perceptual loci of effect: Lexical or sentential context influences the way in which the word is initially perceived. Post-perceptual loci of effect: Lexical or sentential context  influences decisions about the nature of the missing phoneme information.

McGurk effect McGurk and MacDonald (1976) Show people a video where the audio and the video don’t match Think “dubbed movie” McGurk effect McGurk effect2 In the 1970s, McGurk and MacDonald were studying how young children perceive speech. For example, in one study they placed a videotape of a mother talking in one location while the sound of her voice played in another in order to determine if the children attended to the direction of the sound or to the video. In one version of the experiment, they replaced a [ga] sound with a [ba] sound. For example, if the mother said the word [garage] they substituted [ba] into the first part of this word to produce the sound [barage]. Importantly, this meant that the mother made lip movements that were consistent with the sound [ga], but the tape produced the sound [ba].  However, when McGurk and MacDonald watched the tape they didn't hear the word [garage] or [barage] but instead heard [darage]. What appeared to have happened was that hearing the sound [ba] and viewing lip movements consistent with the sound [ga] resulted in them perceiving the sound [da]. In a subsequent series of experiments McGurk and MacDonald demonstrated that this effect occurred in adults and children. These studies demonstrated that listeners will make use of both sound information and visual information when identifying phonemes. When the sound that was produced was inconsistent with the visual information, participants integrated both sources of information when perceiving the sound, without being aware that they were doing this.  McGurk and MacDonald found that audio information favouring a [ba] sound and visualinformation favouring a [ga] sound was integrated to produce a [da] sound.  This illusion has become known as the McGurk effect or the McGurk illusion, and demonstrates that phoneme perception is an active process that may be influenced by both audio and visual information. It  demonstates that phoneme perception may be influenced by knowledge about the visual context.  Implications phoneme perception is an active process influenced by both audio and visual information

Beyond the segment NUDIST Shillcock (1990): hear a sentence, make a lexical decision to a word that pops up on computer screen (cross-modal priming) Hear: The scientist made a new discovery last year. As subjects read either the phrase new discovery or novel discovery the word NUDIST appeared on the computer screen. However, Shillcock and Tabossi found that subjects responded more quickly when NUDIST appeared on the screen as they heard the sentence in (7) as compared to when it appeared on the screen as they listened to the sentence in (8). Shillcock and Tabossi explain that this must have occurred as a results of a short-lived speech segmentation error in which subjects took the sound of the word new, i.e. [nu], and added this to the sound [disc] at the beginning of discovery to hear the word [nudist]. We know this was a very short-lived effect that did not linger long enough for the subject to become consciously aware of the error, as when asked subjects do not report having heard the word nudist. Shillcock (1990) argues that the misheard word is only active in the lexicon (where words are stored in the brain) for a very short period of time and the misperception is quickly corrected. However, the word is active for a long enough period of time for it to prime recognition of the visual version of the word presented on the computer screen NUDIST

Cross-modal priming NUDIST Shillcock (1990): hear a sentence, make a lexical decision to a word that pops up on computer screen (cross-modal priming) Hear: The scientist made a novel discovery last year. As subjects read either the phrase new discovery or novel discovery the word NUDIST appeared on the computer screen. As this is a real word, subjects should press the YES key on their response box. However, Shillcock and Tabossi found that subjects responded more quickly when NUDIST appeared on the screen as they heard the sentence in (7) as compared to when it appeared on the screen as they listened to the sentence in (8). Shillcock and Tabossi explain that this must have occurred as a results of a short-lived speech segmentation error in which subjects took the sound of the word new, i.e. [nu], and added this to the sound [disc] at the beginning of discovery to hear the word [nudist]. We know this was a very short-lived effect that did not linger long enough for the subject to become consciously aware of the error, as when asked subjects do not report having heard the word nudist. Shillcock (1990) argues that the misheard word is only active in the lexicon (where words are stored in the brain) for a very short period of time and the misperception is quickly corrected. However, the word is active for a long enough period of time for it to prime recognition of the visual version of the word presented on the computer screen NUDIST

Cross-modal priming Shillcock (1990): hear a sentence, make a lexical decision to a word that pops up on computer screen (cross-modal priming) Hear: The scientist made a novel discovery last year. The scientist made a new discovery last year. faster Shillcock and Tabossi explain that this must have occurred as a results of a short-lived speech segmentation error in which subjects took the sound of the word new, i.e. [nu], and added this to the sound [disc] at the beginning of discovery to hear the word [nudist]. We know this was a very short-lived effect that did not linger long enough for the subject to become consciously aware of the error, as when asked subjects do not report having heard the word nudist. Shillcock (1990) argues that the misheard word is only active in the lexicon (where words are stored in the brain) for a very short period of time and the misperception is quickly corrected. However, the word is active for a long enough period of time for it to prime recognition of the visual version of the word presented on the computer screen

Cross-modal priming NUDIST gets primmed by segmentation error Shillcock (1990): hear a sentence, make a lexical decision to a word that pops up on computer screen (cross-modal priming) Hear: The scientist made a novel discovery last year. The scientist made a new discovery last year. faster Shillcock and Tabossi explain that this must have occurred as a results of a short-lived speech segmentation error in which subjects took the sound of the word new, i.e. [nu], and added this to the sound [disc] at the beginning of discovery to hear the word [nudist]. We know this was a very short-lived effect that did not linger long enough for the subject to become consciously aware of the error, as when asked subjects do not report having heard the word nudist. Shillcock (1990) argues that the misheard word is only active in the lexicon (where words are stored in the brain) for a very short period of time and the misperception is quickly corrected. However, the word is active for a long enough period of time for it to prime recognition of the visual version of the word presented on the computer screen NUDIST gets primmed by segmentation error Although no conscious report of hearing “nudist”