Brain response to Intonational Phrase structure is purely prosody-driven Barbara Leone-Fernandez1, Rita Jerónimo2, Shuang Lu3, Susana Correia1, Marina Vigário1, Kai Alter4 & Sónia Frota1 1Universidade de Lisboa, 2ISCTE-IUL/CIS, Lisboa, 3Renmin University of China, 4Newcastle University/University of Oxford leonefernadezb@gmail.com, sonia.frota@mail.telepac.pt

Introduction Phrase-level prosody plays a crucial role in language processing Lexical access (e.g., Christophe et al. 2004) Syntactic ambiguity resolution (e.g., Millotte et al., 2007; Speer et al., 2011) Artificial grammar learning (e.g., Langus et al., 2012) It also plays a fundamental role in language development Prosodic bootstrapping approach (e.g., Jusczyk 1997; Christophe et al., 2008; Hohle 2009) Prosodic boundaries used to constrain lexical segmentation and guide syntactic parsing, especially Intonational Phrase boundaries (IPB)

Introduction Intonational phrase boundaries (IPB) cue speech chunks, which typically coincide with major syntactic boundaries (e.g. sentence, clause, topic, parenthetical) Cues to IPB, and their weighting, may vary across languages (Seidl 2007, Wellmann et al. 2012) Pitch change (rise, fall, reset) Pre-boundary lengthening Pause Although there is no one-to-one correspondence From Frota (2014: 13) Fig.1. [the students]IP [as far as we know]IP [have got good marks]IP

Introduction Syntax-prosody mapping Major syntactic boundaries From Frota (2014: 18) Syntax-prosody mapping Major syntactic boundaries Phonology (length, weight, balance…) IPB Set of acoustic cues Fig.2. [the Brazilian girl’s megalomaniac boyfriend]IP [V OBJ]IP IPBs tend to coincide with From Frota (2014: 13) Fig.1. [the students]IP [as far as we know]IP [have got good marks]IP

Introduction Event-related brain potentials (ERPs) are a useful tool for the on-line examination of auditory sentence processing And thus to investigate the processing of Prosodic Boundaries Fig.3. From http://neuronetrix.com/technology-i-36.html

Introduction Event-related brain potentials (ERPs) are a useful tool for the on-line examination of auditory sentence processing And thus to investigate the processing of Prosodic Boundaries The Closure Positive Shift (CPS) is the ERP component that indexes the perception of IPB (Steinhauer et al. 1999, Pannekamp et al. 2005; Boegels et al., 2011, for a review) Fig.4

Introduction The nature of the CPS has been under debate (in adult and developmental studies) It is triggered by prosodic information only, given that it is found in utterances with reduced semantic, syntactic and phonemic content (jabberwocky, hummed – Pannekamp et al. 2005) > Reflects the processing of the prosodic form It is driven by IPBs that reflect syntactic boundaries, since an adult-like effect (no pause) depends on children’s syntactic abilities > Indicates the processing of prosodic and syntactic information (Mannel & Friederici, 2011; Mannel et al., 2013) Cross-linguistic differences in IPB cues. The CPS is not (just) a response to the pause cue, at least for adults (Steinhauer et al., 1999; Mannel & Friederici, 2009) > Argues against the interpretation that it simply reflects sentence continuation after speech silence

Introduction The nature of the CPS has been under debate (in adult and developmental studies) Very few studies have addressed the arguably purely prosodic nature of the CPS In particular in the absence of the pause cue Considering the contribution of the nuclear contour (the melody on the most prominent word, typically the last prosodic word before the boundary) and of the pre-boundary lengthening cues Despite the fact that it remains unclear which aspects are necessary to elicit a CPS

Present study Investigates the CPS as a marker of IPB perception in European Portuguese (EP) Relevant prosodic features of EP > IPB especially prominent: sparse pitch accent distribution (most IP-internal prosodic words unaccented) pitch movements mostly confined to IP edges – last stressed syllable of the IP bears the nuclear pitch accent – boundary syllable bears the boundary tone pre-boundary lengthening only at the IP edge (from the last stressed syllable) IPB especially prominent in EP prosodic phrasing Frota (2000, 2014)

Goals Directly address the debate on the nature of the CPS Pause Syntax Goals Prosody Directly address the debate on the nature of the CPS Using delexicalized stimuli with no syntactic (and semantic) information Using stimuli without the pause cue Varying the position of the nuclear syllable with respect to the boundary If the CPS is prosody-driven in EP The elicitation of a CPS in response to an IPB in delexicalized utterances The CPS response modulated by the distribution of prosodic boundary cues: A more pronounced (and later) CPS response when the nuclear syllable and the boundary syllable overlap, than when the nuclear syllable precedes the boundary syllable

Method Participants Stimuli and Experimental design Twenty-four healthy, right-handed, native EP speakers (17 women, age range: 18-39 years, mean age: 27 years) Stimuli and Experimental design Nine pairs of SVO sentences naturally produced by a native EP speaker: [SVO] – no IPB / [S]IP [VO] – with IPB All CV syllables in each sentence replaced by /ba/, /na/, /mi/, /ti/, /pu/, /lu/ using MBROLA manipulations (occasional codas replaced by /S/) 54 (9x6) sentences with IPB (WB); 54 with no IPB (NB) The manipulation removed all the semantic/syntactic information, but preserved all the prosodic information of the original sentences. IPB location not triggered by syntax (Elordieta et al. 2005)

Method Nuclear syllable penultimate or final in the IP (penult or final stress) NO BOUNDARY (NB) /na/ WITH BOUNDARY (WB) Fig.5 Fig.6 [A nananana nana na nana] (NB, penult) A rapariga brinca na sala ‘The girl plays in the room’ [Asna nanana nana na nana] (NB, final) Este cangurú brinca na sala ‘This kangaroo plays in the room’ [A nananana]IP nana na nana (WB, penult) A rapariga brinca na sala ‘The girl plays in the room’ [Asna nanana]IP nana na nana (WB, final) Este cangurú brinca na sala ‘This kangaroo plays in the room’

Main acoustic cues for the IPB Method Table 1. Acoustic analysis: Pitch range (Hz) and duration (ms) for the target syllable in the NB and WB conditions Pitch range Duration Nuclear contour Pre-boundary (Boundary tone) lengthening Main acoustic cues for the IPB

Method Procedure Experiment lasted ∼20min: 108 sound stimuli delivered aurally in pseudo-randomized order with an inter-stimulus-interval of 2000ms. E-Prime was used. Stimuli were presented in 3 blocks. Block sequence was randomized. Participants were instructed to avoid eye blinking and other body movements during stimulus presentation To minimize eye movements, an eye image was displayed in the center of a monitor before each sentence onset

Method EEG recordings and analysis 30 cap-mounted active Ag/AgCl electrodes (Neuroscan) 500Hz, LF0.05Hz, HF40Hz, Notch50Hz ANOVAs: IPB (NB, WB), ML (Fz, FCz, Cz, CPz, Pz) 1250-1500 time-window ms ML (5) IPB (NB,WB) AP (2) x Hem (2) x IPB (NB,WB) ANOVAs: IPB and Nuclear syllable 800-950 time-window ms 1350-1550 time-window ms Penultimate syllable: ML (5) IPB (NB,WB) Final syllable: ML (5) x IPB (NB,WB) Penultimate syllable: AP (2) x Hem (2) x IPB (NB,WB) Final syllable: AP (2) x Hem (2) x IPB (NB,WB) ANOVAs for the midline electrodes were performed considering the two conditions (NB and WB) and electrodes (Fz, FCz, Cz, CPz, Pz) as factors. Further analyses included the lateral sites with Hemisphere (left, right) and location (anterior, posterior) as factors.

Results With R, t-contrasts from onset to the end (.05) (around 450ms from the boundary=end of boundary syllable) Figure 7. Visual inspection of the average ERP responses revealed a positive shift that peaks between 1250 and 1500ms from sentence onset

Results Table 2. ANOVAS for mean amplitudes in the TW 1250-1500 from sentence onset A significant main effect of condition over all scalp, but more centrally distributed. Both distribution and latency relative to IPB in line with previously described CPS responses.

Results IPB and Nuclear syllable Penultimate syllable, i.e. no overlap with boundary syllable (‘rapariga’) Final syllable, i.e. overlap with boundary syllable (‘canguru’) NB NB WB WB CPS (800-850ms from the onset) CPS (1350-1550ms from the onset) IPB x AP: F(4,92)=5.51, p<0.01, ɳ2=0.19 IPB: F(1,23)=10.34, p<0.01, ɳ2=0.31 CPS response was stronger and later when nuclear syllable and boundary syllable overlapped, than when the nuclear syllable preceded the boundary syllable

Discussion We addressed the debate of whether the CPS is, or not, purely prosody- driven Using delexicalized stimuli with no syntactic (and semantic) information, obtained from natural EP sentences where IPB location is not triggered by syntax IBP cues: No pause. Pitch (nuclear contour) and duration (pre-boundary lengthening) Varying cue distribution: spread (nuclear boundary syllables); overlap (nuclear=boundary)

Discussion We addressed the debate of whether the CPS is, or not, purely prosody- driven A CPS was identified in response to the IPB The CPS response was modulated by the distribution of prosodic boundary cues Our findings demonstrate that brain responses to IPB are prosody-driven Implications for prosodic bootstrapping accounts in language acquisition: prosody-first view, with prosodic bootstrapping of syntactic boundaries (e.g., Morgan & Demuth, 1996; Seidl & Cristia, 2008; Wellmann et al., 2012) Not pause, not syntax

Obrigada! Thank you! Thanks to all the participants, and to Simão Cortês, Ana Chainho and Ana Alexandra Malho for their help with data collection Funding: FCT – Portugal, EXCL/MHC-LIN/0688/2012 leonefernadezb@gmail.com, sonia.frota@mail.telepac.pt

References Christophe, A., Peperkamp, S., Pallier, C., Block, E., & Mehler, J. (2004). Phonological Phrase boundaries constrain lexical access I. Adult data. Journal of Memory and Language, 51, 523–547 Frota (2014). The intonational phonology of European Portuguese. In Jun (ed.) Prosodic Typology II. Oxford: OUP. 6-42 Höhle (2009). Bootstrapping mechanisms in first language acquisition. Linguistics, 47(2): 359-382 Langus A., Marchetto, E., Bion, H. R. A., & Nespor, M., (2012). Can prosody be used to discover hierarchical structure in continuous speech? Journal of Memory and Language, 66: 285–306 Männel, C., Schipke S. C., & Friederici, A. D., (2013). The role of pause as a prosodic boundary marker: Language ERP studies in German 3- and 6-year-olds. Developmental Cognitive Neuroscience, 5: 86-94 Pannekamp, A., Toepel, U., Alter, K., Hahne, A., & Friederici, A. D., (2005). Prosody-driven sentence processing: an event-related brain potential study. Journal of Cognitive Neuroscience, 17(3): 1-15 Männel & Friederici, (2011). Intonational Phrase structure processing at different stages of syntactic acquisition. Developmental Science 14(4): 786-798 Steinhauer, K., Alter, K., & Friederici, A. D., (1999). Brain potentials indicate immediate use of prosodic cues in natural speech processing. Nature Neuroscience 2:191-196 Wellmann, C., Holzgrefe, J., Truckenbrodt., H., Wartenburger, I., & Höhle, B., (2012). How each prosodic boundary cue matters: evidence from German infants. Frontiers in psychology, 3 Bӧgels, S., Schriefers, H., Vonk, W., & Chwilla, D., J. (2011). Prosodic Breaks in Sentence Processing Investigated by Event-Related Potentials. Language and Linguistics Compass 5/7, : 424–440, 10.1111/j.1749-818x.2011.00291.x

Experimental Design Puniltimate Stress Syllable Original sentence: A rapariga brinca na sala Replaced sounds : (ba): A babababa baba ba baba (lu): U lulululu lulu lu lulu (mi): I mimimimi mimi mi mimi (na): A nananana nana na nana (pu): U pupupupu pupu pu pupu (ti): I titititi titi ti titi Ultimate Stress Syllable Original Sentences Este canguru brinca na sala Replaced sounds : (ba): Ashba bababa baba ba baba (lu): Ushlu lululu lulu lu lulu (mi): Ishmi mimimi mimi mi mimi (na): Ashna nanana nana na nana (pu): Ushpu pupupu pupu pu pupu (ti): Ishti tititi titi ti titi

Method Nuclear syllable penultimate or final in the IP (penult or final stress) NO BOUNDARY (NB) /na/ WITH BOUNDARY (WB)

Results Table 3. ANOVAS for mean amplitudes by nuclear syllable position, from sentence onset Penultimate syllable 800-950 ms time window ANOVA HEMxDMLxAPxIPB Amp (μV, mean) F and p values Main effect of IPB F(1,23)=2.25, p=.14 Interaction IPBxAP Frontal: NB, WB   NB=0.48, WB=0.94 F(4,92)=5.51, p<.05 t(23)=-2.89, p hocb .<.05 Final syllable 1350-1550 ms time window ANOVA HEMxDMLxAPxIPB Amp (μV, mean) F and p values Main effect of IPB Mean values NB, WB NB=0.04, WB=1.02 F(1,23)=10.3467, p<.001 Interaction IPBxAP   F(4,92)=2.91, p=.07 Posthoc – hochberg correction

IPB CPS_R NS BS stress

IPB CPS_R NS=BS stress