2. PSY 369: Psycholinguistics Language Comprehension: Sentence comprehension

3. Announcements Getting dates set Homework 4 due Tuesday March 27 Three articles to choose from, pick one Quiz on chapters 6&9 posted, due March 29 Moving Exam 3 back a week to Thursday, April 12 The rest of the homework assignment dates should be correct on blackboard now More quizzes will be posted as we upload the questions, I will announce them as they get posted

4. Overview of comprehension The cat chased the rat. Input cat dog cap wolf tree yarn cat claw fur hat Word recognition Language perception c a t /k/ /ae/ /t/ Syntactic analysis cat S VP ratthe NP chased V the NP Semantic & pragmatic analysis

5. Eye-movements in reading Clothes make the man. Naked people have little or no influence on society. One of the most common measures used in sentence comprehension research is measuring Eye-movements Eye-movements in reading are saccadic rather than smooth Eye-movements in reading Video examples: 1 | 2 | 3 | 4 | 5 1234 5

6. The eye and how it behaves At its center is the fovea, a pit that is most sensitive to light and is responsible for our sharp central vision. The central retina is cone- dominated and the peripheral retina is rod-dominated. Limitations of the visual field 130 degrees vertically, 180 degrees horizontally (including peripheral vision)

7. Eye-movements in reading Limitations of the visual field 130 degrees vertically, 180 degrees horizontally (including peripheral vision Perceptual span for reading: 7-12 spaces Clothes make the man. Naked people have little or no influence on society.

8. Eye Behaviors Within the visual field, eye movements serve two major functionseye movements Saccades to Fixations – Position target objects of interest on the fovea Tracking – Keep fixated objects on the fovea despite movements of the object or head

9. Fixations The eye is (almost) still – perceptions are gathered during fixations 90% of the time the eye is fixated duration: 150ms - 600ms In reading, the assumption is that the length of fixation is correlated with amount/type of processing being done at that point (on that word, at that point in the syntactic parse)

10. Saccades Saccades are used to move the fovea to the next object/region of interest. Connect fixations Duration 10ms - 120ms Very fast (up to 700 degrees/second) No visual perception during saccades Vision is suppressed Evidence that some cognitive processing may also be suppressed during eye-movements (Irwin, 1998) Video examples: 1 | 2 | 3 | 4 | 5 1234 5

11. Saccades Move to here

12. Saccade w/o suppression Video example

13. Saccades Move to here Video example

14. Saccades Video example

15. Saccades Saccades are used to move the fovea to the next object/region of interest. Connect fixations Duration 10ms - 120ms Very fast (up to 700 degrees/second) No visual perception during saccades Vision is suppressed Ballistic movements (pre-programmed) About 150,000 saccades per day

16. Smooth Pursuit Smooth movement of the eyes for visually tracking a moving object Cannot be performed in static scenes (fixation/saccade behavior instead)

17. Smooth Pursuit versus Saccades Saccades Jerky No correction Up to 700 degrees/sec Background is not blurred (saccadic suppression) Smooth pursuit Smooth and continuous Constantly corrected by visual feedback Up to 100 degrees/sec Background is blurred

18. Purkinje Eye Tracker Laser is aimed at the eye. Laser light is reflected by cornea and lens Pattern of reflected light is received by an array of light- sensitive elements. Very precise Also measures pupil accommodation (switching between looking far or close) No head movements Measuring Eye Movements

19. Video-Based Systems Infrared camera directed at eye Image processing hardware determines pupil position and size (and possibly corneal reflection) Good spatial precision (0.5 degrees) for head-mounted systems Good temporal resolution (up to 500 Hz) possible

20. S NP Ndet Themanhit dog withtheleash.the Theman Eye movements in reading

21. S NPVP V Ndet Themanhit dog withtheleash.the Themanhit Eye movements in reading

22. S NPVP VNP NdetN Themanhit dog withtheleash.the Themanhitdogthe Eye movements in reading

23. S NPVP VNP NdetN Themanhit dog withtheleash.the Themanhitdog PP withtheleash the Modifier Eye movements in reading

24. S NPVP VNP NdetN Themanhit dog withtheleash.the Themanhitdog PP withtheleash the Instrument Eye movements in reading

25. Themanhit dog withtheleash.the How do we know which structure to build? Eye movements in reading S NPVP VNP NdetN Themanhitdog PP withtheleash the Instrument S NPVP VNP NdetN Themanhitdog PP withtheleashthe Modifier

26. Parsing The syntactic analyser or “parser” Main task: To construct a syntactic structure from the words of the sentence as they arrive Main research question: how does the parser “make decisions” about what structure to build?

27. Different approaches Immediacy Principle: access the meaning/syntax of the word and fit it into the syntactic structure Serial Autonomous Analysis: Build just one based on syntactic information and continue to try to add to it as long as this is still possible Parallel Autonomous Analysis: Build all possible syntactic structures based on syntax, then use semantics to pick the right one Interactive Analysis: Use multiple levels (both syntax and semantics) of information to build the “best” structure Minimal Commitment: Stop building - and wait until later material clarifies which analysis is the correct one.

28. Sentence Comprehension Autonomous (modular) Interactive models

29. Sentence Comprehension A vast amount of research focuses on: Garden path sentences A garden path sentence invites the listener to consider one possible parse, and then at the end forces him to abandon this parse in favor of another.

30. Real Headlines  Juvenile Court to Try Shooting Defendant  Red tape holds up new bridge  Miners Refuse to Work after Death  Retired priest may marry Springsteen  Local High School Dropouts Cut in Half  Panda Mating Fails; Veterinarian Takes Over  Kids Make Nutritious Snacks  Squad Helps Dog Bite Victim  Hospitals are Sued by 7 Foot Doctors

31. Sentence Comprehension Garden path sentences The horse raced past the barn fell. VP S NP The horse

32. Sentence Comprehension Garden path sentences The horse raced past the barn fell. VP V raced S NP The horse

33. Sentence Comprehension Garden path sentences The horse raced past the barn fell. VP VPP PNP racedpast S NP The horse

34. Sentence Comprehension Garden path sentences The horse raced past the barn fell. VP VPP PNP racedpastthe barn S NP The horse

35. Sentence Comprehension Garden path sentences The horse raced past the barn fell. VP VPP PNP racedpastthe barn S NP The horse fell

36. Sentence Comprehension Garden path sentences The horse raced past the barn fell. VP VPP PNP racedpastthe barn S NP The horse raced is initially treated as a past tense verb

37. Sentence Comprehension Garden path sentences The horse raced past the barn fell. VP VPP PNP racedpastthe barn S NP The horse fell raced is initially treated as a past tense verb This analysis fails when the verb fell is encountered

38. Sentence Comprehension Garden path sentences The horse raced past the barn fell. VP VPP PNP racedpastthe barn S NP The horse fell raced is initially treated as a past tense verb This analysis fails when the verb fell is encountered raced can be re-analyzed as a past participle. VP V raced PP PNP pastthe barn S NP The horsefell NPRR V

39. A serial model Formulated by Lyn Frazier (1978, 1987) Related to work by Kimball (1973) Build trees using syntactic cues: phrase structure rules plus two parsing principles Minimal Attachment Late Closure Go back and revise the syntax if later semantic information suggests things were wrong

40. A serial model Minimal Attachment Prefer the interpretation that is accompanied by the simplest structure. simplest = fewest branchings (tree metaphor!) Count the number of nodes = branching points The girl hit the man with the umbrella.

41. S NP the girl VP V hit NP the man PP P with NP the umbrella S NP the girl VP V hit NP the man PP P with NP the umbrella The girl hit the man with the umbrella. 8 Nodes 9 nodes Minimal attachment Preferred

42. A serial model Late Closure Incorporate incoming material into the phrase or clause currently being processed. OR Associate incoming material with the most recent material possible. She said he tickled her yesterday

43. Parsing Preferences.. late closure She said he tickled her yesterday S np she vp v said S' np he vp v tickled np her adv yesterday S np she vp v said S' np he vp v tickled np her adv yesterday Preferred (Both have 10 nodes, so use LC not MA)

44. Minimal attachment Garden path sentences The spy saw the cop with a telescope. minimal attach non- minimal attach Modular prediction Build this structure first Interactive prediction Build this structure first (Rayner & Frazier, 1983)

45. Minimal attachment Garden path sentences The spy saw the cop with a revolver. minimal attach non- minimal attach Modular prediction Build this structure first Interactive prediction Build this structure first Lexical/semantic information rules this one out (Rayner & Frazier, 1983)

46. Minimal attachment Garden path sentences (Rayner & Frazier, 1983) S NP the spy VP V saw NP the cop PP P with NP the revolver S’S’ but the cop didn’t see him S NP the spy VP V saw NP the cop PP P with NP the revolver S’S’ but the cop didn’t see him MANon-MA The spy saw the cop with the binoculars.. The spy saw the cop with the revolver Conclusion: participants didn’t use semantic information initially, built the wrong structure and had to reanalyze. Supports a serial model. <- takes longer to read

47. Interactive Models The evidence examined by the lawyer … The defendant examined by the lawyer… evidence typically gets examined, but can’t do the examining Other factors (e.g., semantic context, co-occurrence of usage & expectation) may provide cues about the likely interpretation of a sentence Trueswell et al (1994) But See Ferreira & Clifton (1986)

48. Interactive Models Other factors (e.g., semantic context, co-occurrence of usage & expectation) may provide cues about the likely interpretation of a sentence The evidence examined by the lawyer … The defendant examined by the lawyer … A defendant can be examined but can also do examining. Trueswell et al (1994)

49. Semantic expectations Taraban & McCelland (1988) Expectation The couple admired the house with a friend but knew that it was over-priced. The couple admired the house with a garden but knew that it was over-priced. Other factors (e.g., semantic context, co-occurrence of usage & expectation) may provide cues about the likely interpretation of a sentence

50. The Non-MA structure may be favored Semantic expectations Taraban & McCelland, 1988 The couple admired the house with a friend but knew that it was over- priced. The couple admired the house with a garden but knew that it was over- priced.

51. What about spoken sentences? All of the previous research focused on reading, what about parsing of speech? Methodological limits – ear analog of eye-movements not well developed Auditory moving window Reading while listening Looking at a scene while listening Some research on use of intonation

52. Intonation as a cue A: I’d like to fly to Davenport, Iowa on TWA. B: TWA doesn’t fly there... B1: They fly to Des Moines. B2: They fly to Des Moines.

53. Chunking, or “phrasing” A1: I met Mary and Elena’s mother at the mall yesterday. A2: I met Mary and Elena’s mother at the mall yesterday.

54. Phrasing can disambiguate I met Mary and Elena’s mother at the mall yesterday Mary & Elena’s mother mall One intonation phrase with relatively flat overall pitch range.

55. Phrasing can disambiguate I met Mary and Elena’s mother at the mall yesterday Mary mall Elena’s mother Separate phrases, with expanded pitch movements.

56. Summing up Is ambiguity resolution a problem in real life? Yes (Try to think of a sentence that isn’t partially ambiguous) Many factors might influence the process of making sense of a string of words. (e.g. syntax, semantics, context, intonation, co- occurrence of words, frequency of usage, …)