2. 1 Speech, Reading and the Linguistic Process: A conference in honor of Ignatius G. Mattingly

3. 2 Prosody and Reading Janet Dean Fodor Janet Dean Fodor Graduate Center, CUNY Graduate Center, CUNY

4. 3 Collaborative research with Dianne Bradley, Eva Fernández, Yuki Hirose, Yoshihisa Kitagawa, Nenad Lovri ć, Deirdre Quinn, Amit Shaked.

5. 4 Ignatius’ synthesis by rule research 1964 Holmes, J. N., Mattingly I. G., and Shearme, J. N. Speech synthesis by rule. Language and Speech 7, 127. 1966 Mattingly, I. G. Synthesis by rule of prosodic features. Language and Speech 9, 1-13. 1968 Mattingly, I. G. Synthesis by rule of General American English. Haskins Laboratories Status Report on Speech Research. Supplement.

6. 5 Ignatius’ synthesis by rule research 1964 Holmes, J. N., Mattingly I. G., and Shearme, J. N. Speech synthesis by rule. Language and Speech 7, 127. 1966 Mattingly, I. G. Synthesis by rule of prosodic features. Language and Speech 9, 1-13. 1968 Mattingly, I. G. Synthesis by rule of General American English. Haskins Laboratories Status Report on Speech Research. Supplement.

7. 6 Some input strings for the JSRU speech synthesis system A ‘BERD in DHA ‘HAAND IZ WERTH ‘TUU IN DHA ‘BOOSH. A ‘BERD in DHA ‘HAAND IZ WERTH ‘TUU IN DHA ‘BOOSH. DID YUU ‘KUM BIE ‘MOATAKAR? DID YUU ‘KUM BIE ‘MOATAKAR? Prosodic marks:. falling tone, final pause Prosodic marks:. falling tone, final pause ? rising tone, final pause ? rising tone, final pause + fall-rise tone, non-final pause + fall-rise tone, non-final pause

8. 7 Realization of these tones Adjust the F0 (pitch) of the syllable immediately following the stressed syllable of the last prominent word in the sentence 8 steps down, for a falling tone 8 steps down, for a falling tone 4 steps up, for a rising tone 4 steps up, for a rising tone and extend the fall / rise to end of sentence.

9. 8 The motor car Did you come by motor car? Did you come by motor car? (original, synthesized) (original, synthesized) Thanks to Matthew Mattingly, Gary Chant

10. 9 The original motorcar Did you co me by mo tor car? Did you co me by mo tor car?

11. 10 The available technology – a computer An excerpt from an interview with Ignatius by Matthew Mattingly in 1999: “The procedure was to make paper tapes at JSRU…you couldn’t see what you were typing at the time...you could run the tape through a printer and find your mistakes… When you’d made these tapes, that consisted of your program and your data, you shipped the whole thing to the installation several miles away, where the computer was. They would run the program, and make more paper tapes as output, and ship it all back to you. In the case of the speech synthesis, we’d send a paper tape with the synthesis by rule program, and more paper tapes with the input sentences that we were synthesizing, and they would send back paper tapes that contained groups of values for each parameter in the synthesizer. The value for the pitch, and each of the three formants, and the amplitudes and so on, one such set for every ten milliseconds. We would take this paper tape and mount it on a series of pulleys, and an electric eye read the tape and sent the signals to the actual speech synthesizer. As the tape went round on the pulleys, you heard the output that you’d called for.”

12. 11 The motor car Did you come by motor car? (original) Did you come by motor car? (original) Did you come by motor car? (JDF high) Did you come by motor car? (JDF high) Did you come by motor car? (JDF low) Did you come by motor car? (JDF low) You came by motor car. (JDF) You came by motor car. (JDF) ?? ??

13. 12 JDF Did you come by motorcar? H* vs L* question

14. 13 JDF Did you come by motorcar? Question vs statement

15. 14 Group data by Tanya Viger 15 Ss, 5 items e.g., Su sa nna has an en e my. e.g., Su sa nna has an en e my. (Does) Su sa nna have an en e my? (Does) Su sa nna have an en e my?

16. 15 The motor car Did you come by motor car? (original) Did you come by motor car? (original) Did you come by motor car? (JDF high) Did you come by motor car? (JDF high) Did you come by motor car? (JDF low) Did you come by motor car? (JDF low) You came by motor car. (JDF) You came by motor car. (JDF) ?? ??

17. 16 Relevance for sentence processing In speech, the prosodic contour can disambiguate some syntactic ambiguities. Many studies since Lehiste 1973. They fed her dog biscuits. They fed her dog biscuits. They fed her dog-biscuits. They fed her dog-biscuits. Flying planes can be dangerous. Flying planes can be dangerous.

18. 17 At CUNY we study silent prosody For readers, few prosodic cues in the input. For readers, few prosodic cues in the input. In reading aloud, the reader computes a prosodic contour and imposes it on the word string. In reading aloud, the reader computes a prosodic contour and imposes it on the word string. We claim that this happens in silent reading too. We claim that this happens in silent reading too. How can we know this? Why does it matter? How can we know this? Why does it matter?

19. 18 Syntactic parsing may be biased by silent prosody For an ambiguous sentence, a reader who assigns a certain prosodic contour may treat that prosody as if it had been part of the input stimulus, and use it to resolve the ambiguity. If so, ambiguity resolution preferences will not be a reliable source of information about syntactic parsing strategies – even in experiments on silent reading.

20. 19 In fact, a crisis in syntactic parsing theory: universal parsing principles in jeopardy 1978 Frazier & Fodor: The Sausage Machine. Hypothesis: The human sentence parsing mechanism is innate (universal); just plug in a grammar. The parser’s task: Take in words; build a syntactic tree. Chop the word string into approx 6-word chunks Chop the word string into approx 6-word chunks Minimal Attachment Minimal Attachment Late Closure (  local / low attachment) Late Closure (  local / low attachment) Minimal Chain Principle Minimal Chain Principle 1988 Cuetos & Mitchell: Late Closure is not universal. Spanish doesn’t obey it. Spanish doesn’t obey it.

21. 20 The relative clause attachment ambiguity Someone shot the servant of the actress of the actress who was… who was… Who was on the balcony? The servant = HIGH ATTACHMT favored in Spanish * LC The actress = LOW ATTACHMT favored in English LC

22. 21 the servant of the actress who… NP NP 2 3 the N’ the N’ 2 3 N1 PP N’ RC servant 2 2 of NP N1 PP 2 servant 2 the N’ of NP 2 2 the N’ of NP 2 2 N2 RC the N2 actress actress N2 RC the N2 actress actress

23. 22 Proposed explanation Prosodic breaks are optimally aligned with syntactic phrase edges (Selkirk 2000)  For perceivers, a prosodic discontinuity favors a syntactic discontinuity.  For perceivers, a prosodic discontinuity favors a syntactic discontinuity. Prosody N1 of N2 / RC  HIGH attachment SPANISH Prosody N1 / of N2 RC  LOW attachment Prosody N1 of N2 RC  LOW attachment ENGLISH

24. 23 Prosody could explain the curious grouping of languages Hypothesis: Weaker vs stronger tendency to break before RC. LOW RC-ATTACHMENT HIGH RC-ATTACHMENT TENDENCY TENDENCY LOW RC-ATTACHMENT HIGH RC-ATTACHMENT TENDENCY TENDENCY American English Afrikaans British English Croatian Egyptian Arabic Dutch American English Afrikaans British English Croatian Egyptian Arabic Dutch Norwegian French Norwegian French Romanian German Romanian German Swedish Russian Swedish Russian Spanish Spanish

25. 24 Prosody can explain a universal effect of RC-length Short RC  Less likely break before RC  More low attachment Short RC  Less likely break before RC  More low attachment …the servant …the servant of the actress of the actress who smokes. who smokes. who smokes twenty cigarettes a day. who smokes twenty cigarettes a day. This is reminiscent of the ‘packaging’ effects of the Sausage Machine, which were also length-dependent. This is reminiscent of the ‘packaging’ effects of the Sausage Machine, which were also length-dependent. HIGH attachment is easier for phrases packaged as a separate package. Package = prosodic phrase? HIGH attachment is easier for phrases packaged as a separate package. Package = prosodic phrase?

26. 25 Crisis over. Why do readers of different languages sometimes make different ambiguity resolution choices? Many interesting and plausible explanations, but the evidence now suggests: The parsing routines obey fully universal principles. The parsing routines obey fully universal principles. Any differences are due to differences in the grammar that is applied to input strings of words. Any differences are due to differences in the grammar that is applied to input strings of words. The language-specific grammar includes prosodic principles – which are applied even in silent reading. The language-specific grammar includes prosodic principles – which are applied even in silent reading.

27. 26 Prosody in silent reading can explain a variety of other parsing facts Effect of preposition in Croatian, Hebrew, Effect of preposition in Croatian, Hebrew, German, Greek. German, Greek. Clause boundary placement in Japanese. Clause boundary placement in Japanese. Effect of focus particles in German. Effect of focus particles in German. Wh-scope interpretation in Japanese. Wh-scope interpretation in Japanese. Not-because scope preference in English. Not-because scope preference in English. PP-attachment in English questions. PP-attachment in English questions.

28. 27 Prosody can explain the effect of prepositions Croatian (Lovric 2003) Croatian (Lovric 2003) no preposition between nouns preposition between the nouns  no prosodic break there  prosodic break there no preposition between nouns preposition between the nouns  no prosodic break there  prosodic break there  prosodic break before RC  no prosodic break before RC  prosodic break before RC  no prosodic break before RC  high RC-attachment  low RC-attachment  high RC-attachment  low RC-attachment Hebrew (Shaked 2004) Hebrew (Shaked 2004) Similar results but shifted along the scale. Hebrew “construct state” with no preposition is one phonological word. Permits absolutely no prosodic break between the nouns. Similar results but shifted along the scale. Hebrew “construct state” with no preposition is one phonological word. Permits absolutely no prosodic break between the nouns.

29. 28 Prosody can explain preferences in garden-path reanalysis Hirose 1998: Japanese clause boundaries are highly ambiguous. Readers tends to locate them where there is a prosodic break for reasons of phrase length. Hirose 1998: Japanese clause boundaries are highly ambiguous. Readers tends to locate them where there is a prosodic break for reasons of phrase length. Bader 1998: German focus particles force a particular stress pattern. Garden-path recovery is harder if the prosody needs correcting as well as the syntax. Bader 1998: German focus particles force a particular stress pattern. Garden-path recovery is harder if the prosody needs correcting as well as the syntax.

30. 29 Effects of focus prosody as well as prosodic phrasing Kitagawa & Fodor 2004: Japanese wh-questions are ambiguous if there are 2 clauses, each with a potential scope-marker for the wh-phrase. Kitagawa & Fodor 2004: Japanese wh-questions are ambiguous if there are 2 clauses, each with a potential scope-marker for the wh-phrase. [[……wh-NP……… ka]………ka] [[……wh-NP……… ka]………ka] Scope can be disambiguated by post-focus de- prosodification, which extends from the wh-phrase (focus) to its (later) scope-marker. Scope can be disambiguated by post-focus de- prosodification, which extends from the wh-phrase (focus) to its (later) scope-marker. Prosodic pressures (e.g., retain rhythmicity; align with syntax) predict which scope interpretation readers prefer in different cases. Prosodic pressures (e.g., retain rhythmicity; align with syntax) predict which scope interpretation readers prefer in different cases.

31. 30 Not-because scope ambiguity (pilot data only, so far) Koizumi 2004: Based on Frazier & Clifton 1996. Koizumi 2004: Based on Frazier & Clifton 1996. Preferred BECAUSE > NOT : Sue doesn’t cry because she realizes life is hard. Preferred BECAUSE > NOT : Sue doesn’t cry because she realizes life is hard. Inside an if-clause, preferred NOT > BECAUSE : Inside an if-clause, preferred NOT > BECAUSE : a. Sue didn’t cry because she was in public. Was she tearful later? a. Sue didn’t cry because she was in public. Was she tearful later? b. If Sue didn’t cry because she was in public, was she tearful later? b. If Sue didn’t cry because she was in public, was she tearful later? c. Sue didn’t cry because she felt lonely. What else was the matter? c. Sue didn’t cry because she felt lonely. What else was the matter? d. If Sue didn’t cry because she felt lonely, what else was the matter? d. If Sue didn’t cry because she felt lonely, what else was the matter?  The if-clause context▪ reduces the prosodic break before because ▪ induces F0 rise at end of the because-clause ▪ induces F0 rise at end of the because-clause This prosody is typical of the NOT>BECAUSE reading ( Hirschberg & Avesani 1997 ) This prosody is typical of the NOT>BECAUSE reading ( Hirschberg & Avesani 1997 ) (But also pragmatics of if? How to dissociate prosody & pragmatics?) (But also pragmatics of if? How to dissociate prosody & pragmatics?)

32. 31 More on NOT-BECAUSE Troseth, Fodor, Koizumi & Fernandez 2004: Force the NOT > BECAUSE reading by using negative polarity item: John didn’t leave the meeting early because he was mad at anyone. John didn’t leave the meeting early because he was mad at anyone. Grammaticality judgment task. Readers accepted only 14% Listeners accepted 49% Listeners accepted 49% - but they regard the sentence as incomplete.

33. 32 Now – the prosody of questions A current investigation, far from complete Now – the prosody of questions A current investigation, far from complete PP-attachment in English questions, e.g., PP-attachment in English questions, e.g., a. The nanny seated the cranky little child on the swing in Oakwood Park. a. The nanny seated the cranky little child on the swing in Oakwood Park. b. The nanny seated the cranky little child on the swing in his stroller. DIFFICULT! b. The nanny seated the cranky little child on the swing in his stroller. DIFFICULT! c. Did the nanny seat the cranky little child on the swing in his stroller? NOT SO DIFFICULT? c. Did the nanny seat the cranky little child on the swing in his stroller? NOT SO DIFFICULT? Hypothesis: final rise in questions is a discontinuity Hypothesis: final rise in questions is a discontinuity  favors a discontinuity in the syntactic tree structure  facilitates high attachment of the final PP.  favors a discontinuity in the syntactic tree structure  facilitates high attachment of the final PP. Fodor, Bradley & Shaham 2004 Fodor, Bradley & Shaham 2004

34. 33 PP1 high  reanalyze PP1 to low VP VP 9 V NP PP1 PP2 V NP PP1 PP2 seated child on swing in stroller! VP VP 9 V NP PP2 V NP PP2 seated 1 in stroller seated 1 in stroller NPP1 NPP1 child on swing child on swing

35. 34 Recall the sharp prosodic discontinuity in questions e.g., Su sa nna has an en e my. e.g., Su sa nna has an en e my. (Does) Su sa nna have an en e my? (Does) Su sa nna have an en e my?

36. 35 In sum Ignatius’ research encompassed both prosody and reading It seems now that these are even more closely bound together than we knew I am happy to be following (a little bit) in his footsteps.