Phonetic Context Effects

Major Theories of Speech Perception Motor Theory: Specialized module (later version) represents speech sounds in terms of intended gestures through a model of or knowledge of vocal tracts Direct Realism: Perceptual system recovers (phonetically- relevant) gestures by picking up the specifying information in the speech signal. General Approaches: Speech is processed in the same way as other sounds. Representation is a function of the auditory system and experience with language. Explanatory level = gestureExplanatory level = sound

Fluent Speech Production adjacent speech becomes more similar The vocal tract is subject to physical constraints... Mass Inertia Coarticulation = Assimilation Radical Context Dependency Also a result of the motor plan

An Example Place of Articulation in stops Say /da/Say /ga/ AnteriorPosterior

An Example Place of Articulation in stops Say /al/Say /ar/ AnteriorPosterior

An Example Place of Articulation in stops Say /al da/ Say /ar da/ Say /al ga/ Say /ar ga/ Place of articulation changes = Coarticulation

An Example Place of Articulation in stops Say /ar da/ Say /al ga/ Coarticulation has acoustical consequences

/al da/ /ar da/ /al ga/ /ar ga/ * * f t How does the listener deal with this?

Speech Perception /ar//al/ /ga/ /da/

Identifying in Context Percent “g” Responses [ga] [da] /al/ /ar/ Percent “g” Responses

Direction of Effect Production/al/ More /da/-like /ar/ More /ga/-like Perception/al/ /ar/ More /da/-like

Perceptual Compensation ForCoarticulation

What happens when there is no coarticulation? AT&T Natural Voices Text-To-Speech Engine “ALL DA”“ARE GA”

Further Findings 4 ½ month old infants (Fowler et al. 1990) Native Japanese listeners who do not discriminate /al/ from /ar/ (Mann, 1986)

“There may exist a universally shared level where representation of speech sounds more closely corresponds to articulatory gestures that give rise to the speech signal.” (Mann, 1986) “Presumably human listeners possess implicit knowledge of coarticulation.” (Repp, 1982) Theoretical Interpretations Motor Theory

Major Theories of Speech Perception Motor Theory: “Knowledge” of coarticulation allows perceptual system to compensate for its predicted effects on the speech signal. Direct Realism: Coarticulation is information for the gestures involved. Signal is parsed along the gestural lines. Coart. is assigned to gesture. General Approaches: Those other guys are wrong.

Theoretical Interpretations Common Thread: Detailed correspondence between speech production and perception Special Module for Speech Perception Talker-Specific Speech-Specific Two Predictions:

Testing Hypothesis #1 Testing Hypothesis #1 Talker-specific Should only compensate for the speech coming from a single speaker

Testing Hypothesis #1 Testing Hypothesis #1 Talker-specific Male /al/ Male /ar/ Male /da/ - /ga/ Female /al/ Female /ar/

Testing Hypothesis #2 Testing Hypothesis #2 Speech-specific Compensation should only occur for speech sounds /al/ SPEECH TONES /ar/ SPEECH TONES

Testing Hypothesis #2

Does this rule out motor theory? It may be that the special speech module is broadly tuned. If it acts like speech it went through speech module. If not, not. /al/ /ar/ SPEECH PRECURSORS

Training the Quail /da/ /ga/ 1267 /al//ar/ Withheld from training Withheld from training CV series varying in F3 onset frequency

Context-Dependent Speech Perception by an avian species Normalized Response (Pecks or “GA” Responses)

Conclusions General auditory processes play a substantive role in maintaining perceptual compensation for coarticulation 3 Links to speech production are not necessary 1 Neither speech-specific nor species-specific Learning is not necessary 2 Quail had no experience with covariation

Major Theories of Speech Perception Motor Theory: “Knowledge” of coarticulation allows perceptual system to compensate for its predicted effects on the speech signal. Direct Realism: Coarticulation is information for the gestures involved. Signal is parsed along the gestural lines. Coart. is assigned to gesture. General Approaches: General Auditory Processes GAP

Effects of Context a familiar example How well does this analogy hold up for context effects in speech?

Effects of Context a familiar example

/al da/ /ar da/ /al ga/ /ar ga/ * * f t

Hypothesis: Spectral Contrast the case of [ar] Time Production Production F3 assimilated toward lower frequency Frequency /ar da/ Perception Perception F3 is perceived as a higher frequency F3 Step Percent /ga/ Responses

Evidence for General Approach

The Empire Strikes Back

Fowler, et al. (2000) video audio Visual cue: face “AL” or “AR” Ambiguous precursorTest syllable: /ga/-/da/ series Precursor conditions differed only in visual information

Results of Fowler, et al. (2000) More /ga/ responses when video cued /al/

Experiment 1: Results No context effect on test syllable –F(1,8) = 3.2, p =.111 %ga responses by condition for 9 participants

A closer look… 2 videos: /alda/ /arda/ Video information during test syllable presentation Should be the same in both conditions

…more consistent with /ga/? …more consistent with /da/? /alda/ video/arda/ video

Results

Comparisons

Conclusions Spectral contrast is best current account 3 No evidence of visually mediated phonetic context effect 1 No evidence that gestural information is required 2 But what about backwards effects???

The Stimulus Paradigm Time Target Speech Stimulus /da-ga/ Noise Burst (/t/ or /k/) Sine-wave Tone Context (High or Low Freq) Got Dot LowHigh Gawk Dock     Time (ms) Frequency (Hz)

Speaker Normalization Ladefoged & Broadbent (1957) CARRIER SENTENCE “Please say what this word is…” Original, F1 , F1  TARGET “bit”, “bet”, “bat”, “but” + TARGET acoustics were constant TARGET perception shifted with changes in “speaker” Spectral characteristics of the preceding sentence predicted perception ‘Talker/Speaker Normalization’ Sensitivity to Accent, Etc.

Experiment Model /ga/ /da/ 19 Time Natural speech F2 & F3 onset edited to create 9-step series Varying perceptually from /ga/ to /da/ Speech Token 589 ms

No Effect of Adjacent Context with intermediate spectral characteristics Time Silent Interval 50 ms Standard Tone 70 ms Speech Token 589 ms 2300 Hz PILOT TEST: No context effect on speech perception (t (9) =1.35, p=.21)

Acoustic Histories ACOUSTIC HISTORY: The critical context stimulus for these experiments is not a single sound, but a distribution of sounds  ms tones, sampled from a distribution  30-ms silent interval between tones Time Acoustic History 2100 ms Silent Interval 50 ms Standard Tone 70 ms Speech Token 589 ms

Acoustic History Distributions Tone Frequency (Hz) Frequency of Presentation High Mean = 2800 Hz Low Mean = 1800 Hz

Example Stimuli Time (ms) Frequency (Hz) AB 2800 Hz Mean 1800 Hz Mean

Characteristics of the Context Context is not local  Standard tone immediately precedes each stimulus, independent of condition. On its own, this has no effect of context on /ga/-/da/ stimuli. Context is defined by distribution characteristics  Sampling of the distribution varies on each trial Precise acoustic characteristics vary with trial  Context unfolds over a broad time course Time Acoustic History 2100 ms Silent Interval 50 ms Standard Tone 70 ms Speech Token 589 ms

Results p<.0001 Contrastive

Notched Noise Histories Time (ms) Frequency (Hz) A B 100 Hz BW for each notch

Results TonesNotched Noise p<0.04 p<0.01 N=10

Joint Effects? Time Acoustic History 2100 ms Silent Interval 50 ms Standard Tone 70 ms Speech Token 589 ms Time Acoustic History 2100 ms Silent Interval 50 ms /al/ or /ar/ 300 ms Speech Token 589 ms Conflicting  e.g., High Mean + /ar/ Cooperating  e.g., High Mean + /al/

Interaction of Speech/N.S. p=.007 p<.0001 p=.009 Significantly greater than speech alone Same magnitude as Speech Only, opposite direction Follows NS spectra