1. Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands Scalable Computational Models of Emotion for Virtual Characters Joost Broekens, Doug DeGroot {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands.

2. Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands Contents What’s an emotion? Psychological theories of emotion. Appraisal Theory and BDI Agents. Why use emotions in agents? Problem definition. FeelMe systems Context Sensitive Appraisal Banks Experiment Conclusion

3. Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands What’s an Emotion Common emotions: fearful, angry, happy, sad, surprised, disgusted Short episode triggered by an (internal/external) event composed of: –subjective feelings –inclinations to act –facial expressions –cognitive evaluation, –(and some other things) Heuristic relating events to goals, needs, desires, beliefs of an agent. –Evaluates personal relevance and helps decision-making (Neurological evidence: Damasio) Communication medium. –Communicate internal state (Sociological evidence: Darwin, Ekman)

4. Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands How is an emotion produced? James-Lange –Emotion results from the evaluation of the bodily reactions that are provoked by events. Schacter-Singer (Two Factor Theory) –Emotion results from the cognitive evaluation labeling the arousal of the organism. Arousal results directly from events. StimulusResponseFeedback StimulusArousal EmotionEvaluation Emotion

5. Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands Cognitive Appraisal Theory and BDI Agents Appraisal Theory (Frijda, Lazarus, Scherer, etc): –Emotion: is a result of the evaluation of the environment in relation to the agent’s goals, needs, beliefs and desires (=appraisal). –Evaluation in terms of appraisal dimensions: variables expressing a certain emotional aspect of a situation, e.g. valence or arousal. –Appraisal Assumption: evaluation is both necessary and sufficient for an emotion to occur. BDI based agents: –Agent’s thinking based on beliefs, desires and intentions. –Possess basics to which appraisal based emotions can be added. PerceptionAppraisalEmotion Beliefs/Desire/Goals/Etc.

6. Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands Why Use Emotions in Agents Virtual Agents (NPCs, Tutor agents) are enriched with emotions for, e.g., the following reasons: –Enhance sense of realism (VR Training). –Entertainment (Games) –Enhance communication between agent/robot and human (HCI). Examples: SIMS2, Mission Rehearsal Exercise (Marsella and Gratch), Kismet (Braezeal).

7. Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands Computational Models of Emotion in Virtual Agents Mostly Cognitive Appraisal Theory Based –Appraisal assumptions “evaluation in terms needs and goals of the agent is necessary and sufficient” makes Cognitive Appraisal Theory suitable for Virtual Agents based on BDI architecture. Integrated into BDI structure / Architecture of the agent. –Because emotions result from cognitive evaluation, computational models of emotion are integrated in BDI architecture. –Not built as extendible module or add-on but often deeply integrated in the BDI architecture.

8. Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands Problem How to incrementally add sophistication to a computational model of emotion, while keeping it consistent with the emotions produced by the simpler version of the model? Why model-scalability: –Different mechanisms might give good emotion results in different situations (e.g. event-based emotion encoding / appraisal-based emotion). –Development benefits (selling upgrades/ debugging and evaluation). How to make scalable computational models of emotion? Why runtime-scalability: –Resources in games are limited, but very variable (high-end/low-end PCs). –Ideally users can trade-off the amount of emotional detail for e.g. frame- rate (analogous with e.g. Graphical detail). –Other situations,e.g. viewing distance.

9. Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands FeelMe system, a Dynamic Modular Approach Used FeelMe system (DeGroot) to work on this problem. FeelMe system (limited overview). DSS: Decision Support System AS: Appraisal System, emotionally evaluates the environment EMS: Emotion Maintenance System, maintains emotional state BMS: Emotional expression AS EMS BMS DSS Agent's environment/Agent's internal state PerceptionAppraisalEmotion

10. Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands Appraisal System / Emotion Maintenance System. AS continuously emotionally evaluates the situation as constructed by the Decision Support System. AS interprets the situation in terms of appraisal dimension values. AS sends a stream of n-dimensional vectors of these values to the EMS. Such a vector is called an appraisal result. EMS maintains the emotional state as a point in the n-dimensional space of appraisal dimensions. (e.g. a point in the pleasure, novelty space) EMS assumes appraisal results are changes, or “deltas” to the emotional state. EMS integrates these appraisal results with the existing emotional state. –EMS is “pushed” towards a direction. I.E: Signal-based approach AS EMS BMS DSS Agent's environment/Agent's internal state

11. Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands Multiple “Appraisal Banks” What if multiple independent appraisal subsystems could send appraisal results? –Appraisal results can be produced by multiple appraisal mechanism, including event encoding (e.g. events have a fixed emotional meaning). –Modular and scalable approach to appraisal? Context Sensitive Appraisal Banks (i.e. appraisal modules) How to integrate the result of different concurrent “appraisal banks”? AS ASM EMS BMS DSS Agent's environment/Agent's internal state AS EMS BMS DSS Bank 1 Bank n Agent's environment/Agent's internal state

12. Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands Constraints for the Integration of Results from Appraisal Banks EMS integrates appraisal results: –Appraisal-results defined at interval scale –Banks together must produce non-zero positive and negative values. Context Sensitivity in Appraisal Banks –Sensitive to mutually exclusive situations. Either predefined (based on context), or Dynamic (based on appraisal of the current situation of the agent). –Appraising on different detail level: One bank: big picture Second bank: details AS EMS BMS DSS Bank 1 Bank n Agent's environment/Agent's internal state

13. Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands Experiment: PacMan To find out if such integration of appraisal bank information is feasible using the signal-based approach FeelMe system, and To find out if this integration permits the use of these banks as separate appraisal modules that add value to each other. –Allowing incremental design  model-scalability –Dynamic appraisal adaptation during run-time  run-time scalability PacMan (Chow) simulation: compare the traces of the resulting emotional state while playing one level of PacMan (controlled by the human) of an Appraisal Systems with 1, or with 2 appraisal banks. AS EMS Bank 1 AS EMS Bank 1 Bank 2

14. Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands PacMan’s Appraisal System Appraisal Banks relate to PacMan’s following goals: –Points: all events related to gathering points. –Survival: all events related to survival. The output of Appraisal Banks is dependent on the appraisal intensity of other banks but is independent from the appraisal mechanism of other banks. –Survival bank inhibits the effect of points bank. –Rational: survival more basic and important than points –Example of mutual exclusive banks, dynamically based on the current situation as evaluated by the appraisal banks.. AS EMS: Integrate Survival Points Inhibition metric

15. Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands PacMan’s Emotional State Based on three appraisal dimensions (Mehrabian): –Pleasure: related to goal congruency. –Arousal: related to novelty and attention needed for the event. –Dominance: related to the influence PacMan has on its environment, i.e. PacMan’s power. EMS simply integrates the appraisal results from the Appraisal Banks

16. Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands Results Emotional state (emotion) behaves more sophisticated to the conditions of the environment in the 2-bank (survival, points) configuration (e.g. eating a ghost is positive in the 2-bank case). Emotional state (emotion) is still consistent with the emotional state of those situations in which the 1-bank (survival) produced emotions that were meaningful.

17. Joost Broekens, Doug DeGroot, {broekens, degroot}@liacs.nl LIACS, Leiden University, The Netherlands Conclusions Context Sensitive Appraisal Banks in a dynamic, signal-based approach facilitate model-scalability (i.e. the ability to incrementally add sophistication to a computational model of emotions while staying consistent with the earlier version of that model) –If: appraisal banks are mutually exclusive either pre-programmed or dynamic (experiment), and –If results from appraisal appraisal banks can be integrated in a meaningful way. Runtime-scalability can be achieved using the appraisal bank setup. –Switching the “point” appraisal bank on/off would only add/remove emotional sophistication, and not result in inconsistent emotional state behavior. –Triggering the emotion system at a (slightly) different rates did not introduce differences in the resulting emotion (see paper).