1. Empathy in Educational Synthetic Agents: a first glimpse Ana Paiva

2. Vienna June 2003 Why empathy?

3. Vienna June 2003 Empathy  Def. "An observer reacting emotionally because he perceives that another is experiencing or about to experience an emotion“ (Stotland 1978)  Def 2 “The capacity of participating in or vicarious experiencing of another’s feeling, volitions, or ideas and sometimes another’s movements to the point of executing bodily movements resembling his.

4. Vienna June 2003 Constructs of Empathy  The first is the mediation of empathy (how does empathy occur?) and the second the outcome of the empathic process (what does it lead to?).  Elements: Observer (experiences empathy) and the Target (the perceived person)

5. Vienna June 2003 Constructs of Empathy: Mediation (1) via the situation- the observer concludes the emotional state of the target from the situation the target is dealing with. (2) via emotional expressions - the observer interprets the behaviour of the target, like for example, when a target cries means that he/she is probably sad.

6. Vienna June 2003 Constructs of Empathy: Outcome  cognitive or affective(Davis, 1994). cognitive outcome involves cognitive activity of the observer, like obtaining more information about the target, or helping the target. affective outcomes (which is the one we normally consider as empathy) means that the observer experiences an emotion because of his/her perception of the target.

7. Vienna June 2003 Empathic Synthetic Agents User’s experience Empathic emotions Character’s show empathy towards users and other characters "empathic synthetic agent" as "an agent that is able to perceive and have an internal representation of the other agent's emotions and/or experiencing an appropriate emotion as a consequence empathic synthetic agent" "empathic synthetic agent" as "an agent that is able to, by its behaviour and features, allow the users to build an empathic relation with it"

8. Vienna June 2003 Empathy and Synthetic Charactares  First-order Empathic Characters- to build characters that, by their behaviours, they are able to show empathy (or not) for other characters and thus become more believable; and  Relational Empathic Characters- to build characters that, by their appearance, situation, and behaviour, are able to trigger empathic emotions in the user.

9. Vienna June 2003 First Problem: First Problem: How build a synthetic character that is perceived as empathic?

10. Vienna June 2003 Characters that show empathy First Problem: First Problem: How build a synthetic character that is perceived as empathic? Look at the outcomes of empathy… Cognitive (design of the behaviour of the character and the situation) Affective (through emotional expressions in face, body, gestures and voice).

11. Vienna June 2003 Cognitive aspects/behaviour  Behaviour/ architecture of the agent  Emotion (need to have emotional processing)  Modeling of others’ emotions (and of the user)  Familiarity with the others (and the user) Characters that show empathy

12. Vienna June 2003 Affective In order for the Empathic Agent to show empathy which is perceived by the others and the user: 1. Facial Expression (expressive facial expressions that include emotions and communication signals) 2. Voice 3. Body expression, posture and gestures Characters that show empathy

13. Vienna June 2003 Educational Context: the Empathic Synthetic Tutor 

14. Vienna June 2003 Educational Context: the Empathic Synthetic Tutor  Open  Warm  Relaxed  Good-humored  Explains how things work without criticizing  Smiles frequently  Has lots of eye contact  Has an expressive face which shows emotions  Tries to reflect the other's emotions  Gives clear directions  Supportive  Animated  Moves around  Listens carefully  Elicits understanding  Uses humor  Knows the user interest and is aware of individual differences  Forms a "personal" relationship with the user Characters that show empathy

15. Vienna June 2003 2nd Problem: How to trigger empathy?

16. Vienna June 2003 2nd Problem: How to trigger empathy?  Via the situations: by creating situations that the user can perceive the emotional state of the characters.  Via emotional expression: by giving the characters affective expressions, through face, voice, body, sound.

17. Vienna June 2003 VICTEC: Empathy for Personal and Social Education  Bullying (a constant and very frequent problem in schools…) Direct physical bullying (hitting, kicking, punching…) Verbal bullying including name calling, cruel teasing, taunting and threatening Relational/indirect bullying related to social exclusion, malicious rumour spreading and deliberate withdrawal of friendship

18. Vienna June 2003 VICTEC: Bullying  Victimisation rates in the rate of 8% to 46% and bullying others from 3% to 23%  Consequences involve conduct disorder, hyperactivity, physical health problems, sickness, depression, anxiety and low self- esteem.

19. Vienna June 2003 Triggering Empathy in VICTEC  Building a Virtual Storytelling environment where children are confronted with bullying situations.  Characters must be empathic (relational empathic).  The aim is to both trigger empathy in children and to make them aware for the problem.

20. Vienna June 2003 Designing Characters in VICTEC: Mediating empathy through proximity  Based on characters that children already know and interact with (“proximity” aspect)

21. Vienna June 2003 Designing Characters in VICTEC: Mediating empathy though facial expression

22. Vienna June 2003 Designing Characters in VICTEC: Mediating empathy though body expression

23. Vienna June 2003 Building Characters in VICTEC: Mediating empathy though exaggeration Through animation Through posture and facial expression

24. Vienna June 2003 Believable behaviour for the agents: emotional aspects Approach: using an adaptation of Scherer’s theory  “Emotion is defined as an episode of temporary synchronization of all major subsystems of organismic functioning in response to the evaluation of an internal or external stimulus event as relevant to central concerns of the organism” (Scherer) Emotion is a continuous process Emotion is a dynamic process Emotion involves the interaction of dynamic and continuous processes   Process modelling approach: VERY ambitious! 

25. Vienna June 2003 Stimulus Evaluation Checks  Scherer suggests a minimal set of criteria for appraisal, the SECs.  Four appraisal objectives: Relevance Implications Coping Potential Normative Significance  Each SEC may have several sub-checks

26. Vienna June 2003 Architecture

27. Vienna June 2003 Example (1)  Luke’s aggresion to John – Luke’s appraisal  Assessment of Coping Potential check: control and high power. ANS: -Increase in depth of respiration -Slight heart rate decrease -Increased blood flow to head, chest and hands -Papilary consctrictions SNS: -Balanced tone -Tension increase in head and neck

28. Vienna June 2003 Example (2)  Visible manifestations: Voice: -Chest register phonations – “full voice” Body: -Agonistic hand/arm movements -Erect postures -Body lean forward -Approach locomotion

29. Vienna June 2003 Building Situations and Episodes in VICTEC Mediating empathy through situations  Inspired in “real” bullying situations  Teachers workshop and constant interaction with children of selected schools.  Tested using Kartouche.  Using ChildLine situations.

30. Vienna June 2003 Building Situations and Episodes in VICTEC: Goal- Emergent Narrative  Episodes are described as “constraints” determining the “type of event” and the outcome.  The actions of the characters are determined by their architecture and are not scripted.  The episode itself emerges from the actions of the characters (taking the constraints imposed).  A stage manager determines episode by episode, which characteristics are the nest for the next episode, taking into account certain heuristics.

31. Vienna June 2003 Building Situations and Episodes in VICTEC: architecture of the system

32. Vienna June 2003 Episodes  Episodes are constraints on the actions of the characters.  Episodes are used by the episode sequencer to ….. Playground hassle Conflict Playground Victim Bully Hit(Bully, Victim)

33. Vienna June 2003 1st (Very Draft) Demonstrator  Built on top of Wild Tangent  Use of the INESC-ID framework for building synthetic agents in Virtual worlds.  Runs on as an applet on Explorer.

34. Vienna June 2003 Some initial tests  Conducted at the ChildLine conference in London 25th of March.  Using a questionnaire (8 sections measured according to a 5 point Likert scale) Section1. Cartoon versus real. And which character they liked Section2. Voices, and content of conversation Section 3 Character’s movements and realism Section 4 Nature of school environment (attractiveness of the environment) and adequacy to the characters Section 5. Storyline Section 6. Feelings after the demo Usability  76 questionnaires (19 male and 55 female). Age from 10 to 55.

35. Vienna June 2003 Some initial results  51% favoured cartoons and 49% realistic  Younger age group preferred cartoons 70.6% compared with 36.8%  46% attractive environment (15% negative views)  No character preferred. Significant bias towards not preferring Luke (age 8-12).  No preference reported with Martinha.  Older group (+40) 50% preferred Luke whilst 22.7 preferred John.

36. Vienna June 2003 Interaction issues  Episodes must have their narrative impact (creating empathy), thus children must see them as a whole!  Interaction occurs between episodes (the child is the invisible friend of the victim)  Does Interactivity lead to more empathy????  Trying to use physical devices (SenToy) to see the impact in empathy and relation with the characters…

37. Vienna June 2003 SenToy

38. Vienna June 2003 The team VICTEC Partners Prof. Ruth Aylett (in Salford), UK Sarah Woods, Kerstin Dautenhahn in Hertsforshire, UK Carsten Zoll, Bamberg, Germany INESC-ID People Isabel Machado, Catarina Gouveia, Daniel Sobral, Nuno Otero Raquel César Rui Ferreira Fernando Rebelo Marco Costa Rui Prada Marco Vala

39. Vienna June 2003 Final comments