1. PERCEPTION, CONCEPTUALIZATION, AND SYNTAX IN SENTENCE PRODUCTION Andriy Myachykov University of Edinburgh

2. What factors influence the choice of word order? Bottom-up (event perception and conceptualization) Perceptual factors (Tomlin 1995, Forrest 1996, Nappa, et al. in press, but see Griffin & Bock 2000) Conceptual factors (Bock & Warren 1985, Bock, et. al. 1992, Prat-Sala & Branigan 2000, Altmann & Kemper 2006, Chang, et al. 2003) Prominent referents assume prominent syntactic positions Top-down (organization of lexicon and grammar) Syntactic priming (Bock 1986, Branigan, et al. 2000, Smith & Wheeldon 2001) Lexical frequency (Levelt & Maassen 1981, Bock 1987, Meyer, et al. 1998) Lexical priming (Tannenbaum & Williams 1968, Griffin & Weinstein-Tull 2003) Syntactic X Semantic priming (Pickering & Branigan 1998, Clelland & Pickering 2003) Prominent lexical and grammatical material assumes higher priority

3. Is it sorted out then? No!

4. Some questions remain How do perceptual, conceptual, and linguistic factors bias speakers’ choices? Are these factors independent of each other? Are global (long-term) factors stronger than local (short- term) factors? Are the language-perception interfaces language- specific?

5. Global Workspace model (Dehaene, Kerzberg, & Changeux, 1998)

6. General problem: what factors may bias assignment of word order? sentence construction site (workspace) structural preference (e.g. canonical WO) global structural priming local structural information pragmatic factors (e.g. intentionality, discourse status) discourse pragmatics perceptual priming (e.g visual cueing) local perceptual input perceptual/conceptual preference (i.e. causality, animacy, humanness) global perceptual/conceptual priming (e.g., from referent preview) local lexical preference global lexical priming local lexical information

7. General problem: what factors may bias assignment of word order? sentence construction site (workspace) structural preference (e.g. canonical WO) global structural priming local structural information perceptual priming (e.g visual cueing) local perceptual input perceptual/conceptual preference (i.e. causality, animacy, humanness) global perceptual/conceptual priming (e.g., from referent preview) local lexical preference global lexical priming local lexical information

8. Model of sentence production (Bock & Levelt, 1994)

9. Production model message use event information lemma retrieve lemmas assembly map word order perceptual constraints (0-1) Agent =1, Patient = 0 lexical constraints (0-1) Preferred noun = 1 Otherwise = 0 structural constraints (0-1) Canonical structure = 1 Otherwise = 0 VERBVERB perceptual/ conceptual priming (0-1) Referent 1, Referent 2 lexical priming (0-1) Noun 1, Noun 2 structural priming (0-1) Structure 1...Structure n VERBVERB Global constraintsPrimesProduction Local constraintsInteractions workspace

10. Experimental logic –Free description –Perceptual priming –Perceptual and conceptual priming –Perceptual, conceptual, and syntactic priming

11. Measurement set logic t i m e PERCEPTUAL/CONC EPTUAL (from perceptual analysis to rapid apprehension) LEXICAL/SEMANTIC (from lemma selection to lexical access ) OVERT GENERATION (from lexical retrieval to phonological encoding) FIRST FIXATIONS EYE-VOICE SPANS NAME ONSET LATENCIES STRUCTURAL CHOICE

12. Study 1. Free description (English vs Russian) Event description self-paced time-out 7700 msec fixation “Boxer is punching a cowboy” 

13. Study 1. Word order In both languages canonical active voice SVO was predominantly used

14. Study 1. First fixations in both languages “event – agent – patient” apprehension no effect of language at - event - agent reliable effect of language at - patient

15. Study 1. Name onsets reliable effect of language on - subject - verb - object

16. Study 1. Eye-voice spans reliable effect of language on - subject - verb no effect of language on - object

17. Study 1. Discussion In the absence of other cues, canonical causality was used as the matrix for sentence formulation Two scanning processes in both languages : 1. Rapid apprehension “event - agent – patient” (cf. Knoeferle, et al.2005) 2. Incremental, syntax-driven gazes “agent – event - patient” (cf. Griffin and Bock 2000) The sequential pattern of initial fixations suggests that referential analysis is biased toward the canonical causality early on during sentence production initial interrogation of the scene is driven by the nature of the linguistic task Eye-voice span data: freedom of grammatical choice doesn’t facilitate production (cf. Ferreira 1996); freedom of processing does Formulation of Russian sentences took longer at each stage Case marking may complicate freedom of form choice: once you have committed, you better maintain it cause repair is costly

18. Study 2. Fish Film (cf. Tomlin 1995) ENGLISH (Tomlin 1995, 1997): agent cued – “The (red) fish ate the (blue) fish” patient cued – “The (blue) fish was eaten by the (red) fish”

19. Study 2. Russian word order Free word order. Weak grammatical function. Communicative function: topicalization of referents Active case marking Canonical SVO –…with neutral intonation, topics precede discourse neutral constituents… (King 1995) or: in neutral register Russian word order maps directly onto distinction between theme and rheme with old or given information always coming first

20. Study 2. Hypotheses 160Patient-cued (PC) 016Agent-cued (AC) Patient-first (OVS, OSV) (PF) Agent-first (SVO, SOV) (AF) word order type of trial 016Patient-cued (PC) 016Agent-cued (AC) Patient-first (OVS, OSV) (PF) Agent-first (SVO, SOV) (AF) word order type of trial H0H1 Larger range of word order options should promote active alternation between agent- and patient-initial orders (Ferreira 1996) Case marking may complicate incremental production because early commitment to Nominative form makes switch between case-marked forms difficult Passive voice is another candidate for the patient-cued condition. However, it is very infrequent, alien to free word order languages, and very difficult to elicit (Gennari, et al. 2005)

21. Study 2. Word order total

22. Study 2. Word order (Agent-cued) Most frequent forms: S(N) – V – O(N)48% S(Pr) – V – O(N)31% S(Pr) – V – O(Pr)13% S(Pr) – O(Pr) – V5% 97%

23. Study 2. Word order (Patient-cued) Most frequent forms: S(N) – V – O(N)49% O(Pr) – V – S(N)17% S(N) – O(Pr) – V13% O(N) – V – S(N)12% PV8% 99%

24. Study 2. Word order by agent- and patient-initial structures Major effect of cue on the choice of word order but definitely not as strong as in Tomlin’s study

25. Study 2. Reaction times agent-initial/agent-cued faster than agent-initial/patient-cued (182 msec) patient-initial/patient-cued faster than agent-initial/patient-cued (101 msec) agent-initial/agent-cued faster than patient-initial/patient-cued (82 msec)

26. Study 2. Reaction times. Continued S(Pr)-initial/agent-cued faster than O(Pr)-initial/patient-cued (113 msec)

27. Study 2. Discussion Attention to the patient improved its chances of assuming frontal syntactic position in target sentence focus of attention influences the choice of word order RTs for non-canonical constructions longer than canonical constructions canonical sentences are easier to generate Strong tendency to use canonical constructions with wider WO inventory. Longer RTs for patient-initial/patient-cued than agent-initial/agent-cued constructions freedom of choice doesn’t always make life easier RTs for agent-initial/patient-cued sentences longer than agent-initial/agent-cued sentences attentional switch to arrive at SVO when the patient is cued case marking can make life harder

28. Fish Film problems 1. No independent manipulation of attention 2. No control for lexical or syntactic priming 3. No filler materials 4. Same event used 32 times. Routinization

29. Study 3. Finnish (cf. Gleitman, et al, 2007) HEITÄÄ scene 7000 fixation cue 70 fixation verb 2000 fixation 1000    cueing - exogenous perceptual, agent, patient, or theme events - transitive or ditransitive

30. Study 3. Word order No effect of cue

31. Study 3. Name onsets (transitive) Attentional switch is reflected in speech onset data

32. Study 3. Name onsets (ditransitive) Attentional switch is NOT reflected in speech onset data

33. Study 3. Discussion Implicitly driven attentional focus did not predict the ordering of the sentence constituents in Finnish The reliance upon automated canonical grammar seems to be very persistent, more so in case-marking flexible WO languages Perceptual effects during sentence production are quite weak. The more complex the structure, the less likely they are to survive through later stages of sentence preparation

34. Study 4a (perceptual priming. English) cueing - exogenous perceptual, agent or patient SOA - implicit (70 msec) or explicit (700 msec) constraint - participants’ gaze toward the non-cued entity is (1) free or (2) banned cue 70/700 PUNCH scene 7000 fixation verb fixation gaze-contingent   

35. Study 4b (perceptual and conceptual priming. English) cueing - exogenous mixed, agent or patient SOA - implicit (70 msec) or explicit (700 msec) constraint - participants’ gaze toward the non-cued entity is (1) free or (2) banned cue 70/700 PUNCH scene 7000 fixation verb fixation gaze-contingent   

36. Study 4a. Word order Reliable effects of: - instruction - cue - cue x SOA - cue x instruction

37. Study 4b. Word order Reliable effects of: - instruction - cue - SOA - cue x SOA - cue x instruction

38. Study 4. Discussion Cue to the referent improved its chances of assuming prominent syntactic positions Grammatical knowledge biases activation of canonical word order more than perceptual or conceptual prominence Activation of canonical word order occurs even when visual focus is highly constrained Only highly constraining perceptual instructions results in substantial boost of non-canonical structures (cf. Tomlin 1995) Availability of conceptual information about the referent boosts the cueing effect

39. Study 5a. (triple priming. English) cueing - exogenous perceptual, agent or patient syntactic priming - active voice or passive voice verb - matching or non-matching the event cue 500 “The house is being painted by an artist” scene 7000 fixation prime (read out loud) fixation gaze-contingent   

40. Study 5b. (triple priming. English) cueing - exogenous mixed, agent or patient syntactic priming - active voice or passive voice verb - matching or non-matching the event cue 500 “The house is being painted by an artist” scene 7000 fixation prime (read out loud) fixation gaze-contingent   

41. Study 5a. Word order Reliable effects of: - cue - prime - verb match - prime x verb - cue x verb

42. Study 5b. Word order Reliable effects of: - cue - prime - verb match - prime x verb - cue x verb

43. Study 5. Discussion Independent effects of Cue, Syntactic Prime, and Verb Match on the choice of word order Activation of canonical word order was pervasive even with combination of perceptual, conceptual, and syntactic priming. Further evidence that perceptual effects are relatively weak Interaction between Syntactic priming and Verb Match replicates effect from Pickering and Branigan (1998) in transitive sentence production Cue interacted with Verb match but not with Prime. Perceptual effects are effective at message level, are less likely to directly influence syntactic planning Conceptual component (referent preview) boosted priming effect

44. General discussion Do perceptual factors bias the choice of word order? They do but… Not alone. Perceptual effects per se are quite weak. Additional biasing factors (conceptual priming, contextual priming) and/or more powerful cueing manipulations are necessary Not directly. They affect thematic perspective within the message level, but their influence decays quickly under the effect of the language grammar Not equally for all structures. Perceptual effects are more likely to prevail when the structure is relatively simple

45. Production model message use event information lemma retrieve lemmas assembly map word order Perceptual/conceptual constraints (0-1) Agent =1, Patient = 0 lexical constraints (0-1) Preferred noun = 1 Otherwise = 0 structural constraints (0-1) Canonical structure = 1 Otherwise = 0 VERBVERB perceptual/ conceptual priming (0-1) Referent 1, Referent 2 lexical priming (0-1) Noun 1, Noun 2 structural priming (0-1) Structure 1...Structure n VERBVERB ConstraintsPrimesProduction Biases (accessibility)Interactions workspace

46. Further directions Use other tasks –Attentional Network Test (separate effects of alerting, orienting, and conflict resolution networks) Use other structures –Clefts, dislocations, PO/DO attachment, linguistic focus Use other languages –V-initial (Welsh, Malagasy), V-final (German) Use other age-groups –Syntactic representations in young children seem to be less abstract. Expect more reliance on perception – word order interface. Same may be true for bilinguals (e.g., Leow 1997).

47. Thank you very much!