Prosody-driven Sentence Processing: An Event-related Brain Potential Study Ann Pannekamp, Ulrike Toepel, Kai Alter, Anja Hahne and Angela D. Friederici Presented by Laura Matzen, 9/1/2005

Pannekamp et al. (2005) Goal of study: –Determine what causes the closure positive shift (CPS) –Is this effect driven by prosody alone or by other factors?

Pannekamp et al. (2005) Basic Design: –Systematically reduce linguistic content of sentences –Record ERPs in each condition to see if CPS is present in all cases or to see how it changes

Background on Event-related Potentials (ERPs)

Background Closure Positive Shift (CPS) –seen in response to normal spoken sentences –somehow related to intonation contour Intonation contour –“sentence melody” –provides information about syntactic structure, sentence mode –fundamental frequency – F0

“Actually, we were the ones who said we didn’t want a regular cake, so you can blame it on us.” Frequency

“Actually, we were the ones who said we didn’t want a regular cake, so you can blame it on us.” Frequency

Intonational Phrases (IPh) One or more in each sentence Group and organize words into phrases Structure is usually determined by syntax

Intonational Phrases (IPh) “...defined as containing at least one nuclear accent and a boundary tone at their right edges” Pitch contour drops at the end, resets at start of the next IPh Last syllable usually lengthened Often followed by a pause

Examples She went to the store, then she picked up the dry cleaning on the way home. She went to the store, then she dropped the kids off at school, then she went home.

Intonational Phrases (IPh) Seem to be secondary to syntax in some ways –supporting role BUT, listeners can identify prosodic boundaries in the absence of semantic and syntactic information (de Rooij, 1975)

Intonational Phrases (IPh) Beckman (1996) “the prosodic structure of the utterance has to be seen as a full grammatical property also requiring its own parsing” However, it might be harder to process prosody in absence of other linguistic information –Off-line behavioral studies can’t address this issue

Steinhauer et al. (1999) Studied prosody with ERPs –Two sentence types: 1 or 2 IPh boundaries –Saw positive-going waveform in response to all boundaries – CPS –Possible Confound: Close relationship between prosody and syntax CPS could still be related to processing syntactic boundaries, not just prosodic boundaries

Steinhauer & Friederici (2001) ERP study with delexicalized speech –Filtered to removed phonemic, semantic and syntactic information –Only prosodic info left (pitch, amplitude, rhythm)  CPS in this case could only be caused by prosody

Steinhauer & Friederici (2001) Results –Strange-looking CPS at first boundary –No CPS at second boundary –Contingent negative variation (CNV) across whole sentence  Is this because the sentences are so unnatural? Is this task completely different from language processing?

Meyer et al. (2002) fMRI study –Very different responses to natural speech and delexicalized speech –Response to prosody stronger in right hemisphere “strong evidence that pitch processing in the absence of additional linguistic information such as syntax and/or semantics takes place in right hemisphere regions”

Current Experiments Motivations: –Investigate partial replication of CPS findings by Steinhauer & Friederici (2001) Why didn’t they get the expected results? Try using more natural stimuli –See if CPS shifts to the right hemisphere as segmental information is removed

Experiments Experiment 1: Normal sentences –Normal semantic, syntactic and phonemic info Experiment 2: Jabberwocky sentences –Remove semantic information Experiment 3: Pseudo sentences –Remove semantic and syntactic information Experiment 4: Hummed sentences –Remove semantic, syntactic, and phonemic info (only prosody left)

Expt 1: Normal Sentences A1: [Kevin promises mom to sleep] IPh1 [and to be a good boy for a while.] B1: [Kevin promises] IPh1 [mom to kiss] IPh2 [and to be a good boy for a while.]

Expt 1: Normal Sentences A1: one IPh boundary at 1950 msec B1: two IPh boundaries at 950 and 2700 msec “First part” (subject + verb) of sentence longer than A1 High boundary tone at end of first part Longer pause after first part Both conditions have IPh boundary after second verb (marked by high boundary tone)

Expt 2: Jabberwocky Sentences All content words replaced with pseudo words A2: [The bater rabels Onna to lubol] IPh1 [and the rado to nupe.] B2: [The bater rabels ] IPh1 [Onna to lubol] IPh2 [and the rado to nupe.]

Expt 2: Jabberwocky Sentences A2: one IPh boundary at 2100 msec B2: two IPh boundaries at 1100 and 2600 msec “First part” (subject + verb) of sentence longer than A2 High boundary tone at end of first part Longer pause after first part Both conditions have IPh boundary after second verb (marked by high boundary tone)

Expt 3: Pseudo Sentences All function and content words replaced with pseudo words A3: [Bater saklimm Onna ko labei keg ] IPh1 [nug som Rado lie nupes.] B3: [Bater saklimm] IPh1 [Onna ko labei keg] IPh2 [nug som Rado lie nupes.]

Expt 3: Pseudo Sentences A3: one IPh boundary at 2000 msec B3: two IPh boundaries at 920 and 2400 msec “First part” (subject + verb) of sentence longer than A3 High boundary tone at end of first part Longer pause after first part Both conditions have IPh boundary after second verb (marked by high boundary tone)

Expt 4: Hummed Sentences A4: [mm mmm mmmm mm mmmm] IPh1 [mmmm mmm mmm mmm mmmmm.] B4: [mm mmm] IPh1 [mmmm mm mmm] IPh2 [mmmm mmm mmm mmm mmmmm.]

Expt 4: Hummed Sentences A4: one IPh boundary at 1850 msec B4: two IPh boundaries at 850 and 2150 msec Longer pause after first part High boundary tone at end of first part (First part itself is NOT longer in this case) Both conditions have IPh boundary after second verb (marked by high boundary tone)

Procedure Subjects only saw one type of sentence Auditory presentation for all sentences Trial followed by a probe word –Subjects had to say whether word was in sentence or not (Words were placed randomly into hummed filler sentences)

Data Collection 23 Ag/AgCl electrodes 200 msec prestimulus baseline Averages computed over whole sentences – 4500 msec segments

Results

A1: IPh at 1950 ms Response at 2000 ms B1: IPh1 at 950 ms Response at 1500 ms IPh2 at 2700 ms Response at 2700 ms

A2: IPh at 2100 ms Response at 2200 ms B2: IPh1 at 1100 ms Response at 1500 ms IPh2 at 2600 ms Response at 2800 ms

A3: IPh at 2000 ms Response at ms B3: IPh1 at 920 ms Response at 1500 ms IPh2 at 2400 ms Response at 2500 ms

A4: IPh at 1850 ms Response at 2200 ms B4: IPh1 at 850 ms Response at 1000 ms IPh2 at 2150 ms Response at ms Negative peak from ms

Waveforms measured from offset of 1 st sentence fragment

Their interpretation... Positivity resembles CPS Observed in all experimental conditions –Related to processing of prosodic boundaries Scalp distribution changes across conditions

Their interpretation... CPS associated with first IPh seems related to the amount of segmental content in sentences –Moved forward and rightward as info decreased CPS associated with second IPh seems UNrelated to amount of segmental content

Scalp Distributions – IPh1 Experiment 1: CPS over whole head Experiment 2: CPS at midline and lateral sites, also moves to anterior sites Experiment 3: Moves to right anterior sites Experiment 4: CPS broadly distributed

Scalp Distributions First IPh: –CPS moved anterior sites as linguistic information decreased –CPS for psuedo sentences moved rightward, but not for hummed sentences (possible explanation – left hemisphere processing timing of hums?)

Scalp Distributions Second IPh: –CPS distributed broadly over midline sites also over lateral sites for normal and pseudo sentence (why not for other types of sentences?) –CPS moved rightward for hummed sentences ONLY (CPS not seen here at all in previous study- conclude that naturalness is important)

Negativity Early negativity in hummed sentences –Different processing mechanism for pure prosody? –Search for early accent (that isn’t there)? –Different task demands?

Their Conclusions CPS is independent of expt manipulations –Seems to be related to perception of prosodic boundaries –Dependent exclusively on prosody “The observed differences in the scalp distribution of the CPS as a function of the segmental content of the acoustic speech stream suggest that prosodic processing interacts with other information types involving different systems.”

Questions Why did scalp distribution change? (It doesn’t seem to change in a systematic way) Why are there different results for the 1 st and 2 nd IPh boundaries? Are the differences in timing meaningful? Why not compare sentences with no IPh boundaries? (one phrase) Is it reasonable to say that these are “natural” manipulations?