Some comments on the topic of this session Peter Hagoort SLE Conference Poznan, September 12, 2014

The topic

The topic My comments relate to the processing consequences of this insight. “… there is the important fact (unrecognized in the parsing and comprehension literature) that, in cases of a question-answer-driven topic-comment structure, all that is needed for comprehension and interpretation is an identification of the comment – the topic being already available. In the vast majority of cases, therefore, the main task of the interpretation machinery consists in the identification of the comment …” Seuren,2009, p. 238

Information structure (IS) IS refers to the way of linking novel/important information with given information. (Halliday, 1967; Jackendoff, 2002) e.g. What vegetables did mum buy to cook today? Today mum bought eggplant to cook. Behavioral results suggest that focused information receives more attentional resources and is processed more deeply. (Cutler & Fodor, 1979; Birch & Rayner, 1997; Ward & Sturt, 2007) non-focus focus

skull dura mater pia mater axon

The recording of event related brain potentials

The recording of event related brain potentials

MEG

MEG: Sentence reading (n=102); sensor-level “The author decided to pay the lawyer ….”

N400 effect

Unifying semantic and world knowledge “Dutch trains are yellow and very crowded.” (correct) “Dutch trains are white and very crowded.” (world knowledge violation) “Dutch trains are sour and very crowded.” (semantic violation)

Dutch trains are white ….

ERP effects of semantic and world knowledge violations Hagoort, Hald, Bastiaansen, & Petersson, Science, 2004

The P600: sensitive to syntax

Research question How does IS modulate the depth of syntactic processing? (Wang, Bastiaansen, Yang & Hagoort, PLoS ONE, 2012)

How did we test it? Design: Materials (Originally in Dutch): 2 Syntactic (Correct, Incorrect) 2 Context (Focus, Non-focus) Materials (Originally in Dutch): Focus Wie bestelt er een taxi na het feest? (Dutch) Who orders a taxi after the party? (English) Non-focus Wanneer bestelt men een taxi? (Dutch) When orders someone a taxi? (English) Correct Na afloop van het feest bestellen de nogal boze gasten een taxi. After the ending of the party order the rather angry guests a taxi. Incorrect * Na afloop van het feest bestelt de nogal boze gasten een taxi. * After the ending of the party orders the rather angry guests a taxi.

Results: P600 effects Focus Non-focus clear P600 effect no P600 effect Correct Incorrect uV 0.4 0.8 1.2 -0.15 4 6 -2 -4 0.2 0.6 1.0 s 2 Cz Focus Correct Incorrect Cz Non-focus 0.6-1.2 s 0.6-1.2 s clear P600 effect no P600 effect

(Wang, Bastiaansen, Yang & Hagoort, PLoS ONE, 2012) Conclusions Focused information: full syntactic analysis. Non-focused information: shallow syntactic processing. (Wang, Bastiaansen, Yang & Hagoort, PLoS ONE, 2012)

Research question How does IS modulate the depth of semantic processing?

(Wang, Hagoort, & Yang, Brain Res., 2009) Method Materials: (Originally in Chinese) Focus Non-focus What vegetables did mum buy to cook today? Today mum bought eggplant to cook. Who bought vegetables to cook today? Today mum bought beef to cook. Congruent Incongruent (Wang, Hagoort, & Yang, Brain Res., 2009)

Non-focus, incongruent Results uV Non-focus, incongruent Non-focus, congruent s -0.2 0.05 0.3 0.55 0.8 1.05 -2.5 -5.0 2.5 5.0 7.5 CZ uV s -0.2 0.05 0.3 0.55 0.8 1.05 -2.5 -5.0 2.5 5.0 7.5 CZ Focus, incongruent Focus, congruent Focus Non-focus What vegetables did mum buy to cook today? Today mum bought eggplant to cook. Today mum bought beef to cook. Who bought vegetables to cook today? Today mum bought eggplant to cook. Today mum bought beef to cook. Congruent Incongruent N400 effect reduced N400 effect

(Wang, Hagoort, & Yang, Brain Res., 2009) Conclusions Focused information: deep semantic processing Non-focused information: shallow semantic processing (Wang, Hagoort, & Yang, Brain Res., 2009)

Interim conclusion We have found that markers of information structure modulate the amplitude of semantic (N400) and syntactic-like (P600) ERP effects. Information that is in focus (or new) gets processed more deeply. Relevant in the context of “good-enough processing” (Ferreira & Patson, 2007), or the “windowing of attention in language” (Talmy, 1996).

Hypothesis: Syntactic, prosodic and other markers of Information Structure are built-in linguistic devices that trigger the contribution of the attentional networks in the brain, in the surface of in-depth processing of the focus constituent.

The brain in action

A localizer of the auditory attention network

C+P+ (Congruent, With pitch accent) “According to the book of Genesis, NOAH brought two animals of each kind on the ark” C-P+ (Incongruent, With pitch accent) “According to the book of Genesis, MOSES brought two animals of each kind C+P- (Congruent, Without pitch accent) “According to the book of Genesis, Noah brought two animals of each kind C-P- (Incongruent, Without pitch accent) “According to the book of Genesis, Moses brought two animals of each kind

Attentional localizer Under conditions of increased processing complexity, general attentional networks are recruited by prosodic marking of information structure.

Conclusion The topic-comment structure of ongoing discourse triggers extra processing of the focus constituent (the new information). This is in line with the ideas of Seuren (e.g., Seuren, 2009)

N400 effect

Thanks Pieter

Intonation and information structure (IS) Another way to mark IS: pitch accent (context) What vegetable did mum buy to cook today？ Today mum bought eggplant to cook. (pitch accent) Today mum bought EGGPLANT to cook. top-down non-focus focus ACCENTED unaccented bottom-up non-focus focus

(Wang, Bastiaansen, Yang, & Hagoort, Neuropsychologia, 2011) Research question How do accentuation and context together influence the depth of semantic processing ? (Wang, Bastiaansen, Yang, & Hagoort, Neuropsychologia, 2011)

(Wang, Bastiaansen, Yang, & Hagoort, Neuropsychologia, 2011) Methods Design: Context (Focus: F+, Non-focus: F-) Semantic congruency (Congruent: C+, Incongruent: C-) Accentuation (Accented: A+, Unaccented: A-) Dependent variable: N400 effects in response to the semantic congruity Influence on the N400 effect by four combinations of conditions: F+A+, F+A-, F-A+, F-A- (Wang, Bastiaansen, Yang, & Hagoort, Neuropsychologia, 2011)

Results: N400 effect F+ A+: focus, accented F+ A-: focus, unaccented F- A+: non-focus, accented F- A-: non-focus, unaccented 0.2 0.4 0.6 0.8 1.0 1.2 2 -2 -4 -0.2 uV s Cz

Conclusions F+ A+: The most elaborate semantic processing. F- A- = F+ A- = F- A+: The lack of focus marking, or the mismatch between context and accentuation led to shallow processing. Context and accentuation are highly interactive.