1. Theory of Mind and an application to ECA interaction Christopher Peters LINC University of Paris 8 WP6 workshop, 10-11th March 2005, Paris

2. Overview What is Theory of Mind? Some approaches –Cognitive –Philisophical –Evolutionary Psychological Adopted approach –Baron-Cohen (1994) Application to interaction and agents –Interaction initiation

3. Theory of Mind Specific cognitive ability to understand others as intentional agents –Interpret their behaviour –Attribute mental states –Form theories of their intentions, desires and beliefs –Allows prediction Reproduced from REF. 72 © 2000 Elsevier Science

4. Some Approaches Cognitive approach (Leslie 1987) –Pretend play in 2 year olds –Meta-representations Philisophical approach (Dennett 1987) –Design stance and physical stance –Intentional stance Evolutionary psychological approach (Baron- Cohen 1994)

5. One approach Evolutionary approach to theory of mind –Selective advantages for organisms with an ability to interpret and predict behaviour on the basis of mental states –Provides the ‘why?’ and facilitates prediction Perrett and Emery (1994) Direction of Attention Detector Mutual Attention Mechanism Baron-Cohen (1994) Eye Direction and Intentionality Detectors Theory of Mind Module

6. Intentionality Detector (ID) Represents behaviour in terms of volitional states (goal and desire) Based on visual, auditory and tactile cues Attributes intentionality characteristic to objects based on the presence of certain cues Process dyadic representations Goal-desire relationships with objects –A bee wants to go to a colourful flower –A brick wants to fall onto the ground!?

7. Detect the presence of eye-like stimuli –Present in a large number of species Visual modality Detect and represent eye direction as an “Agent” seeing the Self or something else Process dyadic representations –Agent-relation-Self, Agent-relation-Object –Agent 1 -relation-Agent 2, Self-relation-Object Eye Direction Detector (EDD)

8. Shared Attention Mechanism (SAM) Represent if the Self and another agent are attending to the same object or event Triadic relationships –Agent and self are both attending to same object –Self-relation-(Agent-relation-Object) –Self-relation-(Agent 1 -relation-Agent 2 ) Links ID to EDD –Allows eye direction to be read in terms of volitional states

9. Theory of Mind Mechanism (ToMM) Represent the full range of mental states including the episdemic ones Integrate mental state knowledge into a coherent theory of interpreting action M-Representations –Agent-Attitude-“Proposition” Allow one to: –Make sense of an agents current behaviour –Predict an agents future action Interpretation of behaviour in mentalistic ways

10. Details Modules are special purpose computational systems in the social brain All four modules (ID, EDD, SAM, ToMM) are functioning by 4 years of age in humans –EDD most basic and found in many vertebrates as a sensitivity to predators looking at the animal –ID appears to be present in many primates –SAM only partially in great apes –Much less evidence for ToMM outside of humans

11. Useful Information processing approach Modular Emphasis on the eyes and direction of attention of the agent –Special relationship between EDD and SAM –Triadic representations built more easily in the visual modality –When the goal of an action is uncertain, first place young children and adults look for information to disambiguate the goal is the eyes

12. Importance of the Eyes Eyes play an important role in social interaction Looking is noticeable –Social hierarchies and attention structure for cohesion Copyright © Steve McCurry http://www.unep-wcmc.orgCopyright © 2003 Wild Horizons Publishing

13. Eye Contact Psychological Arousal –Aggressive cue Threatening displays in many non-human primates Dominance –Affiliative cue Youngsters who can secure eye contact … Gain attention of their parent Being fed or having other needs satisfied www.adventures.com

14. Conversation initiation Problems with interaction initiation: –Constrained by distance Close proximity required –E.g. Touch screen, enter a certain proximity, etc –Explicit cues must be exchanged User speaks to ECA, or vice versa to initialise ‘All-or-nothing’ approach ECA interactions end up with interactants in close proximity… –…but should not have to begin that way –More subtle and flexible

15. Application Conversation initiation for agents –How do humans start interactions? –How can we let somebody know that we: Want to interact with them Do not want to interact with them –Speech Explicit statement of ones intention, but risky Avoid social embarrassment of engaging in discourse with an unwilling participant (Goffman) Nice to predict ones intention to interact before committing

16. Application –Gesture and facial expression Less explicit/risky But how to know who the expression is directed at? –Direction of attention Eye behaviour, gaze, body orientation, locomotion Allows earlier subtle ‘negotiation’ Probe for intention to interact before committing

17. Application Idea –Use ToM to formulate theory on the intention of the other to interact –Base theory on direction of attention behaviours –Inherently uncertain process Perception and Interpretation –Full model should also account for utterance, gesture and facial expression Baron-Cohen doesn’t include these

18. Direction of Attention Extension by Perrett and Emery (1994) More general Direction of Attention Detector (DAD) –Eye, head, body and locomotion direction

19. Mutual Attention Extension by Perrett and Emery (1994) SAM relationships triadic –But attention can also be shared between two agents Mutual Attention Mechanism (MAM) –Activates only with mutual gaze and eye contact –Dyadic relationship X observes Y and Y observes X simultaneously

20. Integrated Model Schematic of final model including extensions Modules are building blocks –Foundation of an agent perception and interpretation system

21. Possible Uses Dual role –Allow ECA’s to detect humans who want to interact Do so in a more sociable manner Subtle interaction initialisation rather than ‘all-or-nothing’ approach Interaction starts at a distance Allows a more human-like interface with the ECA –Allow ECA’s to generate behaviours that show humans their intention to interact Do so in a more human-like manner i.e. not always sudden and explicit Orienting of attention direction to match attentive behaviours of users or to elicit attention from viewers

22. Our Model Base on simple, high-level theories: –Have they seen me? ID, DAD and MAM modules –Have they seen me looking? ID, DAD and MAM modules –Are they interested in interacting? Level of Interest metric Theories are uncertain and represent best guesses based on perception

23. Our Model Model involves other important mechanisms –Visual Attention –Memory and integration of percepts Implementation in development –Virtual Environment –Interaction between agents –Covered as part of tomorrows talk on Perception of Attention

24. Conclusion Presented a model of theory of mind based on perception of direction of attention –Potential application to improving interface with ECAs –Perhaps provide some insight into how users may change their perception towards ECAs (ID) Baron-Cohens model useful for computational modelling THANK YOU!