1. grateful acknowledgments to
On using eyetracking data to evaluate theories of on-line sentence processing: The case of reduced relative clauses
Charles Clifton, Jr.
grateful acknowledgments to
Mohamed Mohamed
Matt Traxler
Rihana Williams
Keith Rayner
Robin Morris
Sungryong Koh
Lyn Frazier

2. Eye movements and sentence comprehension
Existing measures not a transparent window into cognitive processes
Lexical processing: some good ideas about mapping
Comprehension: another story
Consider one case of garden-path sentences…
Since F&R 82, eyetracking has been the technique of choice to examine processing difficulty in sentence comprehension. Know gp sentences cause difficulty: some combination of long fixations, multiple fixations, long fixations after leaving a word, regressions to earlier material, re-reading of critical material. But don’t have really good link between procesing complexity or breakdown and eye movements.
Better off in case of lexical processing - EZ reader model plus others- largely duck questions of severe processing breakdown and don’t always deal seriously with processing difficulty above level of word identification
But don’t have a good model of how and where comprehension difficulty wil show up. Do the different responses to difficulty go together or complement each other; do they all tend to show up together? Or does one happening preclude the others? Does difficulty always have to show up somewhere in the ET record?

3. Garden Paths in Reduced Relative Clause Sentences (the curse of Tom Bever)
The horse raced past the barn fell.
But sometimes reduced relative clauses are easy to comprehend.
Sometimes RRCs are no trouble at all. Use eyetracking to assess and analyze processing difficulty, try to identify when and why they are hard vs. easy

4. McKoon & Ratcliff, 2003 The window broken by John couldn’t be repaired
The window examined by John needed repairing
Both cases: (easy) reduced relative construction denotes an entity that participates in an externally caused event
Corpus search; turn up commonly used RRC constructions, show they are easy to comprehend, easier than full RC

5. Trueswell, Tanenhaus, & Garnsey, 1994
The defendant examined by the lawyer proved to be unreliable
Produces clear reading disruption in disambiguating region compared to full relative clause (The defendant who was…)
The evidence examined by the lawyer proved to be unreliable
Initial NP inanimate, and a good theme but a poor agent
Claimed to eliminate difficulty of reduced relative clause
TTG 1994 actually a followup to Ferreira & Clifton, 1986
Much-cited paper; get g-p with animate subject (good agent of simple active sentence) – reduced relative clause read more slowly in disambiguating region than full rc (first pass times)
Reported result: eyetracking indicated that garden-path disappeared when made first noun inanimate, bad agent, good patient of first verb
Theoretical implication: meaning, semantic factors, plausibility can guide parsing, eliminate strong frequency/structural bias toward active sentence
Supports interactive model. Means Frazier & Rayner modular g-p model wrong. Plausibility can determine initial structural analyses, contrary to that model.
Step back a bit

6. Animate: The defendant examined by the lawyer…
Equal disruption in RRC compared to FRC for both kinds of initial nouns (measure: first pass time on disambiguating region).
Animate: The defendant examined by the lawyer…
Inanimate: The evidence examined by the lawyer…

7. Criticism: Trueswell, Tanenhaus, & Garnsey, 1994
Some of the Ferreira & Clifton inanimate sentences weren’t all that implausible as main clause sentences
The trash smelled…. (pretty bad)
The car towed…(the trailer)
TTG, better materials, first noun normed for plausibility as agent and as patient of first verb
TTG right about perhaps ¼ of our sentences. Did make up better materials, not just intiutively chosen, did 2 ET studies. Experiment 1 came up with very clear results (E2 statistically the same but not quite as neat)

8. Difficulty of RedRel completely disappears when initial NP was inanimate – good patient, bad agent of initial verb
These were F&C’s original expectations, but not at all what we found

9. Why the differences? Maybe TTG were right – the F&C materials were bad
But maybe something else: parafoveal preview of “by the…”
Evidence from Burgess, Spivey, McRae, etc, in self-paced reading
Possible display limitations in F&C (42 character display, line break typically before PP would have prevented effective preview)
Notion: maybe initial noun inanimacy can guide parsing, but only in the presence of clearly guiding grammatical information I.e.the “by the” phrase. Keep a limited priority for grammatical information in parsing.

10. Clifton, Traxler, Mohamed, Williams, Morris, Rayner, in press
Redo Ferreira & Clifton using Trueswell et al. materials (thanks to John et al.)
Add boundary change manipulation to permit or deny parafoveal preview of “by the noun”
Perform additional analyses
Boundary change: random letters replace “by the” on screen until the eye crosses an imaginary boundary 1 character space before them. Prevent preview of “by the.”

11. Clifton et al. details
Used materials from Trueswell, Tanenhaus, & Garnsey
8 conditions
Animate (“difficult”) vs inanimate (“easy”) initial noun
Reduced vs full (control) relative clause
Parafoveal preview vs. no preview of “by the” (reader saw random 6-letter string instead of “by the”)
Measured eye movements while reading for comprehension (24 Ss)

12. First pass times arguably a lot like TTG
First pass times arguably a lot like TTG. We did get some sig disruption for inanimate nouns, unlike TTG, but disruption was sig reduced. Lack of preview did slow reading, but did not interact with size of ambiguity effect
But – FP time not everything. Many people have realized that when a reader encounters processing difficulty, his/her eyes can linger, regress and linger on earlier material, regress and return, or go ahead to try to find more info. FP measures only the first response.
How to get at other responses to disruption?
Regression path duration (first used by Duffy & Rayner 1986, called “go past” at UMass in the late 80s, and used seriously in the 1990s by people like Liversedge, Pickering, Konieczny, Brysbart, etc. – sum duration of all fixations in region from first entering region until going past it (including time spent re-reading earlier material). Sum of lingering + going back to earlier + initial re-reading
Overall: 66 ms ambiguity effect for animate nouns, 34 ms ambiguity effect for inanimate nouns (sig. smaller, but > 0)

13. Now see very simple, neat pattern of results – big cost to ambiguity, regardless of animacy or preview; big cost of preview; and big cost of animacy (sentences with animate initial noun hard regardless of ambiguity – relatively implausible)
One conclusion: FP times did not provide adequate evidence that inanimacy eliminated or reduced disruption; data consistent with a GP model that says structural factors determine initial parse, plausibilty affects later integration
But admittedly, data also consistent with interactive models, as long as they give strong weight to the high frequency or simplicity of simple active sentence. Only say that we shouldn’t take TTG data as disconfirmation of GP model.
Overall: 68 ms ambiguity effect for animate nouns, 83 ms ambiguity effect for inanimate nouns (n.s. different)

14. Interim Conclusion and Question
Looking only at first pass times gave an incomplete picture of sentence comprehension
RPD measure indicated garden-pathing even with inanimate initial noun
Trueswell et al. experiment not an adequate basis to reject modular serial parsing model
But do the increased RPDs reflect the usual behavior of the eyes?
Accomplished one goal – established that TTG does not show that meaning, plausibility overrides syntactic bias
But what’s really going on? Would like to be able to say that the relative speed of ambig/inanimate items in Fptime reflected a high proportion of regressions out of the disambiguating region – the eyes frequently went back to re-read the offending noun or verb. BUT….

15. First pass regressions out: extremely infrequent, especially in inanimate condition, where I would like to say that disruption prompted re-reading of earlier material. With normal preview, appeared to happen on 5% of the trials at best!
Did get the expected pattern for the no preview case – more regressions in ambiguous case than in full relatvve case. But power problem – interactions not significant.
More regressions for No Preview than Preview, and for Animate than Inanimate, but apparent effect of Reduction and interactions not significant.

16. A similar pattern for: regression path duration
A similar pattern for: regression path duration (outside the region) second pass on target (coming from left)
Very similar pattern if look at 3 related measures – prop reg out (as before), time spent re-reading earlier material, time spent re-reading disambiguating region after regressing out of it. In all cases, sig cost of lack ov preview, sig cost of animate (implausible) noun, and marginal (at best) cost of ambiguity, with no interactions.
Perhaps a power problem. Combine the data from the preview condition with data gathered in parallel at Univ South Carolina (also preview, high and low span readers). Look at prop reg out (similar results, other measures):

17. Combined data: First pass regressions
Combine data with parallel experiment conducted at U South Carolina
Always preview of “by the” phrase
Vary high vs low span readers
Total n = 68, more stable data

18. Significant effects of animacy, ambiguity, no interaction
Now get clearly more regressions for ambiguous RedRel than unamb FullRel (and for animate than inanimate) But no interaction – no evidence that the eye responded differently to disambiguation following inanimate than animate noun (so don’t have understanding of FP/RPD difference) And stuck with saying that this particular response to difficulty – regression – is only happening about 10% of the time at best.
Does this mean that 90% of the time the eye deals with difficulty by lingering in the disambiguating region? But then, given the small effect of disambiguation on first pass time, we would conclude that 90% of the time inanimacy overcame the normal parsing preference for active sentences – and conclude that interactive models are basically right.
But this conclusion is based on a very simplistic model of eye movements. The fact may be that the eyes are USUALLY driven by recognizing words. Processing difficulty may sometimes show up as long fixation or regression, but may sometimes show up as noncomprehension, and sometimes not show up at all. Unhappy conclusion – high level processing shows up only dimly in theET record. May have to swallow it.
Significant effects of animacy, ambiguity, no interaction

19. Summary
Eye movement measures show comparable ambiguity effects for sentences with animate and with inanimate subjects
It is premature to conclude that inanimacy blocks garden-pathing
Other data are needed to evaluate models that give logical priority to syntactic structure
Don’t lose clear message: ET does show equivalent-size effects of ambiguity regardless of plausibility and subject animacy.
Need more tests – perhaps ones like Binder, Duffy & Rayner, JML 2001 – look at (lack of) effect of context on main clause sentences.
But spend rest of time worrying about why we can’t tell a simpler story about how garden-paths and other types of processing difficulty are reflected in eye movements.
Present a little data about other instances of RRCs, some which should be easy, some which should be terribly hard.

20. A problem for understanding eye movements
Regression path durations are inflated by processing difficulty
This inflation can only come from trials where there were regressions
But regressions were very infrequent, < 10% of the trials

21. A similar effect in other experiments
Reduced relative clauses with various classes of verbs (manner of motion, theta-grid changing, unaccusatives)
No garden-pathing in first pass times, big garden-pathing in RPD, but < 30% regressions

22. Conclusions
Sentence processing difficulty is only variably mirrored in the ET record
Doesn’t always show up in FPT
One clear response to difficulty: a regression
But only happens 5-30% of time
A common response to difficulty: fuggettabottit
Just get the words, `mam

23. I’d like to be shown wrong, but…
ET response to comprehension difficulty is variable: linger, regress, nothing
Effects seen in means may actually occur on only a minority of trials
Eye movements may be controlled by word recognition, and only exceptionally affected by higher-level factors

24. Put on those rose-colored glasses
Maybe all these problems will disappear if people read texts they really care about, not just single sentences
But I wouldn’t bet on it

25. Stevenson & Merlo, 1997
The troops marched across the fields all day resented the general.
Manner of movement verbs hard in reduced relatives; transitive use of these verbs requires an operation of “syntactic causativisation”
But…The witch melted in the Wizard of Oz was played by a famous actress.
Seems easy. Suggestion: Unaccusative verbs OK in reduced relatives
Proposal: RRCs hard with certain types of verbs (MOM), easy with others (Unaccuastives). Unacc, same thematic rrelations in active and passive (the witch is the theme of melting in each case). MOM, have to change causal structure (from troops to someone else as the causal agent). Former easy, latter hard – subjectively.

26. Eyetracking Experiment
Sentences with Manner of Movement (MOM) verbs
Sentences with verbs whose thematic grid changes (agent-theme to goal-theme) with passivization
Sentences with unaccusative verbs (theme in both active and passive subject position)

27. MOM (manner of movement) sentences
The horse raced along the beach frightened the onlookers
The craft sailed to the island served as a makeshift ferry
Controls
The horse that was raced to to the beach frightened the onlookers. (full relative clause)
The horse raced along the beach and frightened the onlookers. (main clause)
Disambiguating region indicated in boldface
Note, controls here both unambiguous full RC and main clause continuations.

28. Thematic Grid Change Sentences
The secretary brought the forms filled them out.
The client asked the question received no thanks.
Main clause structure: Agent – verb – theme
Relative clause structure: Goal – verb – theme
Control: The client asked the question but received no thanks
Critical change: from a verb whose frame contains an agent and a theme to one whose frame contains a goal and a theme

29. Unaccusative Verbs The leaves burned today smoked a lot.
The potatoes baked in the oven smelled terrific.
Main clause structure: theme – verb
Relative clause structure: theme – verb
Control: The potatoes baked in the oven and smelled terrific.
Should be easy – thematic relations don’t really change.

30. Predictions Manner of movement sentences: Hard, on everybody’s story
Thematic grid change sentences: Comparably hard, if thematic grid, not causal structure, the root
Unaccusative sentences: Relatively easy, if thematic grid change not syntactic structure change the underlying problem

31. First Pass Times
No sig effect of sentence form, verb type, or their interaction in FPT.

32. Regression Path Durations
Highly sig difficulty for reduced relative sentence. No interaction though – comparable size difficulty effects for all three types of sentences
Echoed in prop regressions out – high prop (mean .32) for red rel, each sentence form
But do we want to conclude that readers only had difficulty with these sentences on 32% of the trials? No way.

33. Comprehension Accuracy
Questions were asked following 2/3 of the items (half of these were relevant to the RC)
MOM questions: The boy marched to school looked unhappy.
How did the boy get to school?
Someone marched him there/He marched there on his own
Theta-grid changing questions: The client asked the question received no thanks.
Who asked the question?
The client / someone
Unaccusative questions: The potatoes baked in the oven smelled terrific
What was happening with the potatoes?
Someone had baked them in the oven / They were sitting in the oven, baking.
Problem is – comprehension uncertain at best. Asked questions, but couldn’t get good disambiguating questions for unaccusatives.

34. Our subjects don’t understand passives!
Note: accuracy for Unaccusative verb sentences is not shown. The questions were bad – only 67% correct for main clause items.

35. Conclusions
Intuitions of sentence processing difficulty not transparently mirrored in the ET record
Typical response to difficulty: a regression
May be specific to this experiment; lots of hard, tricky RC sentences
But even here, only about 1/3 of the trials
Another response to difficulty: fughetabottit
Just get the words, `mam

36. For the future: I’d like to be shown wrong, but…
ET response to comprehension difficulty seems to be variable: linger, regress, nothing
Eye movements may generally be controlled by word recognition, and only exceptionally affected by higher-level factors
Effects seen in means may actually occur on only a minority of trials
Maybe all these problems will disappear if people read texts they really care about, not just single sentences
But I wouldn’t bet on it

39. MacDonald, Pearlmutter, Seidenberg, 1994
In all cases, the examples cited here were not the only reduced relatives in these articles.
The pattern of results obtained with global reading times is reflected by differences located on the critical disambiguating phrase.
From psycholinguists writing

40. Ferreira & Clifton 1986 – An early attempt to find easy RRCs
Ambiguous animate: The defendant examined by the lawyer was unreliable
Ambiguous inanimate: The evidence examined by the lawyer was unreliable
Controls: The defendant/evidence who/that was examined by the lawyer was unreliable.
Disambiguating region indicated in boldface
TTG actually a response to an earlier paper. FF and I were convinced that animacy of the first noun would guide parsing, show g-p wrong. ET experiment, basically like the TTG expt just described. Expected their results. But…