1. universiteit van amsterdam 9th Szklarska Poreba Workshop On the Roots of Pragmasemantics. February 21-25, 2008 | 1 Reinhard Blutner http://www.blutner.de blutner@uva.nl a note on the simulationist framework and motor theories of perception Institute for Logic, Language and Computation

2. universiteit van amsterdam February 21-25, 2008 | 2 Outline ›Motor theories of perception and the simulationist framework ›Caramazza & Mahon's criticism ›Experiments on asymmetries of interpretation/ production ›Conclusions & challenges

3. universiteit van amsterdam 9th Szklarska Poreba Workshop On the Roots of Pragmasemantics. February 21-25, 2008 | 3 1 Motor Theories of Perception and the Simulationist framework

4. universiteit van amsterdam February 21-25, 2008 | 4 James 1890 An educated man... on entering his house one day he received a shock from crushing the finger of one of his little children in the door. At the moment of his fright he felt a violent pain in the corresponding finger of his own body, and this pain abode with him three days. (James, 1890, p. 712)

5. universiteit van amsterdam February 21-25, 2008 | 5 Lotze 1852 The spectator accompanies the throwing of a billiard ball, or the thrust of a swordsman, with slight movements of his arm; the untaught narrator tells his story with many gesticulations; the reader, while absorbed in the perusal of a battle scene, feels a slight tension run through his muscular system, keeping time as it were with the actions he is reading of. (Lotze, 1852; as quoted in James, 1890, p. 525)

6. universiteit van amsterdam February 21-25, 2008 | 6 A mirror neuron is a neuron which fires both when performing an action and when observing the same action performed by another creature ›Monkey grasp a nut ›Monkey sees how another creature grasp a nut Mirror Neurons

7. universiteit van amsterdam February 21-25, 2008 | 7 A mirror neuron is a neuron which fires both when performing an action and when observing the same action performed by another creature ›Monkey grasp a nut ›Monkey sees how another creature grasp a nut Mirror Neurons

8. universiteit van amsterdam 9th Szklarska Poreba Workshop On the Roots of Pragmasemantics. February 21-25, 2008 | 8 2 Caramazza& Mahon's Critizism

9. universiteit van amsterdam February 21-25, 2008 | 9 Thesis 1 (weak motor theory of perception) The perception of actions is constrained by the implicit knowledge that the central nervous system has concerning the movements that it is capable of producing itself. -The mechanism by which humans perceive others differs greatly from how humans perceive inanimate objects. -Unlike inanimate objects, humans have the distinct property of being “like me” in the eyes of the observer. -This allows us to use the same systems that process knowledge about self-performed actions, self-conceived thoughts, and self-experienced emotions to understand actions, thoughts, and emotions in others.

10. universiteit van amsterdam February 21-25, 2008 | 10 Thesis 2 (strong thesis of simulation) The perception of actions/emotions is realized by performing simulations of the relevant movements/ emotions (analysis-by-synthesis mechanism). Proving thesis 2 requires demonstrating that -production programs are necessarily run in the course of perception/recognition/understanding -production programs are sufficient to ground the conceptual content of perception/recognition/ understanding

11. universiteit van amsterdam February 21-25, 2008 | 11 Are Production Programs Needed for Perception? ›3- to 5-month-old infants can discriminate between a point light walker and similar figures with scrambled spatial relationships between the moving dots. -Given that 3- to 5-month-old babies do not have experience with walking, what are they simulating in order that they can recognize walking? ›Subjects who had bilateral paralysis of the face from infancy (Mobius syndrome) were not impaired on a test of facial affect recognition. -What are they simulating?

12. universiteit van amsterdam February 21-25, 2008 | 12 Are Production Programs Sufficient for Perception? ›Perceiving a person that drinks a glass of water -Depending on the context you can infer that the person was thirsty -In another context you may infer the person was nervous -In both cases the motor programs (for drinking water) are the same! ›Hence, intentions cannot be inferred by using simulations

13. universiteit van amsterdam February 21-25, 2008 | 13 Preliminary Conclusions ›Caramazza concludes from such observations that simulations are epiphenomenal. In contrast, I think the play a real role in forming conceptual knowledge ›Simulations may contribute in an important way to the full “meaning” of object and event concepts (such as hammer and drinking) ›However, I doubt that perception/understanding is realized by actually running simulations. Rather, their role consist in restricting perception/understanding via the forming of the relevant knowledge structures.

14. universiteit van amsterdam 9th Szklarska Poreba Workshop On the Roots of Pragmasemantics. February 21-25, 2008 | 14 3 Experiments on Asymmetries of Interpretation/Production

15. universiteit van amsterdam February 21-25, 2008 | 15 Zeevat‘s Metaphor is Misleading but True “The situation can be fruitfully compared to the habit of hiding easter eggs for one's children. The parents engaged in hiding the eggs balance the amount of effort with the desired amount of difficulty in finding the egg. (They also picture the child looking for it and try to keep it possible for the child of finding the egg, without spoiling the fun.) For the child it is another matter. They just have to throw in the effort required for finding the eggs. Not more of course, but definitely not less. It is not a complicated balancing act.” (Zeevat 2000: 245)

16. universiteit van amsterdam February 21-25, 2008 | 16 A Production Problem for R-Expressions ›Hendriks, Englert, Wubs & Hoeks (to appear): Age differences in adults’ use of referring expression ›Children and very old adults produce pronouns where R-expressions are more appropriate

17. universiteit van amsterdam February 21-25, 2008 | 17 Sentence Elicitation Study A woman hold-ing an ice cream cone is walking past a road sign. The woman comes across a girl. She gives the girl an ice cream cone. The girl is eating from the ice cream cone. Well, the woman passes again an ice cream van. The woman buys another ice cream come. Topic shift Target Picture she

18. universiteit van amsterdam February 21-25, 2008 | 18 Results ›Elderly adults produce (non-recoverable) pronouns significantly more often than young adults when refer- ring to the old topic in the presence of a new topic. ›With respect to the comprehension task, no significant differences were found between elderly and young adults.

19. universiteit van amsterdam February 21-25, 2008 | 19 Discussion of the Results ›In principle, I see two possibilities to overcome the production problem: -Fossilization (via bidirectional learning) -The producer learns to reason bidirectionally ›The observation that elderly people have a likewise production problem excludes the fossilization idea ›Defossilization doesn‘t make any sense ›Hence, the speaker seems to take the hearer into account!

20. universiteit van amsterdam February 21-25, 2008 | 20 The Pronoun Interpretation Problem ›Young children have problems in interpreting pronouns but not in interpreting reflexives. ›Hendriks and Spenader (2004) give a new interpretation of children‘s delay of the comprehension of pronouns. ›I discuss the validity of this interpretation and present an alternative account in terms of iterated learning ›The conclusion will be that there is no empirical evidence that he hearer takes the speaker into account (in incremental natural language understanding)

21. universiteit van amsterdam February 21-25, 2008 | 21 Children’s Comprehension of Reflexives* ›Here is an elephant and an alligator. The elephant is hitting himself. ›Question: Does the sentence match the picture? ›Children from age 4 on: Yes * I thank Petra Hendriks for allowing me to use her slides

22. universiteit van amsterdam February 21-25, 2008 | 22 Children’s Interpretation of Pronouns ›Here is an elephant and an alligator. The elephant is hitting him. ›Question: Does the sentence match the picture? ›Children until at least the age of 6 or 7: Yes E.g., Chien & Wexler, 1990; Grimshaw & Rosen, 1990; Jakubowicz, 1984; Koster, 1993; McDaniel, Smith Cairns, & Hsu, 1990; McDaniel & Maxfield, 1992; McKee, 1992.; Spenader, Smits & Hendriks, subm.

23. universiteit van amsterdam February 21-25, 2008 | 23 Children’s Production of Pronouns ›Task: Describe what you see on the picture. ›Children between 4;6 and 7: The elephant is hitting him. Cf. De Villiers, Cahillane, & Altreuter, 2006; Spenader, Smits, & Hendriks, subm.

24. universiteit van amsterdam February 21-25, 2008 | 24 Children’s Production of Reflexives ›Task: Describe what you see on the picture. ›Children between 4;6 and 7: The elephant is hitting himself *The elephant is hitting him

25. universiteit van amsterdam February 21-25, 2008 | 25 H & S ‘s explanation: Delayed Bidirection ›The proposal is that children begin with unidirectional optimization, and only later acquire the ability to optimize bidirectionally. ›Optimizing bidirectionally inherently involves reasoning about alternatives not present in the current situation, which may be a skill acquired very late, thus explaining the lag in acquisition. ›Optimizing bidirectionally requires more processing resources than unidirectional optimization

26. universiteit van amsterdam February 21-25, 2008 | 26 Alternative Explanation: Fossilization ›The fossilization view applies OT learning theory as a mechanism of conventionalization/automatization/ routinization. It is related to an instance theory of automatization (Logan 1988) ›Conceptual advantage of the fossilization account: no mind reading capacities are required for the processing tasks ›Fossilization accounts for the fact hat elderly people (or stressed preople) have no problems with pronoun interpretation.

27. universiteit van amsterdam 9th Szklarska Poreba Workshop On the Roots of Pragmasemantics. February 21-25, 2008 | 27 4 Conclusions and Challenges

28. universiteit van amsterdam February 21-25, 2008 | 28 Online Processing, Offline processing & Learning ›In NNs & OT, online processing is basically unidirectional optimization: expressive vs. interpretive optimization. ›Under special conditions the speaker can take the hearer into account (however, normal incremental generation processes produce garden path sentences; e.g. “The coach knew you missed practice” (Feirerra & Dell 2000) ›I don‘t see any empirical evidence that the hearer takes the speaker into account (in online processing) ›Offline processing can be bidirectional (grammaticality judgments) ›Iterated learning and individual/cultural fossilization is a really existing phenomenon.

29. universiteit van amsterdam February 21-25, 2008 | 29 Invited Inferencing Theory of Semantic Change (Traugott & Dasher 2005; picture taken from Benz 2007) Coded meaning (source stage) New coded meaning (target stage) Global Interpretation Mechanism (Particularised Implicature) Conventionalization (Generalised Implicature) Semantic Reanalysis

30. universiteit van amsterdam February 21-25, 2008 | 30 A Small List With Challenges ›What is the best solution to the rad/rat problem? ›The treatment of garden path effects in incremental interpretation (the listener‘s view taken in OT semantics is basically motivated by such examples) -The coach knew you missed practice ›Accounting for the findings in experimental pragmatics (scalar implicatures, interpretation of reflexives, pronouns, R- expressions) ›Taking over the sentence production in mid sentence by the hearer. Why does this phenomenon exclude other models of incremental processing? ›Reinhard Blutner & Anatoli Strigin: Asymmetric OT, Bidirection and Fossilization