1. Expressive Emotional ECA ✔ Catherine Pelachaud ✔ Christopher Peters ✔ Maurizio Mancini

2. Expressive Emotional ECA ✔ Emotion behavior modeling: ✔ Signals ✔ Temporal course of signals ✔ Expressive qualities ✔ Often in ECA systems: ✔ only modeling of signals is considered ✔ 6 ‘universal’ expressions

3. Large variety of facial expressions (Kollias et al) ✔ Archetypal expressions modeling using FAPs MPEG-4 ✔ Estimation of the range of variations of FAPs in archetypal expressions ✔ Intermediate expressions: rule-based computational model based on ✔ Whissell’s representation: ✔ Activation, evaluation, ✔ Plutchik’s angular measure

4. Laban Movement Analysis ✔ 5 major components: ✔ Body: part of the body in used ✔ Space: description of the directions and paths of motion ✔ Shape: changing forms of the body ✔ Effort: dynamics of the body moves ✔ Relationship: modes of interaction with oneself, others and the environment (facing, contact…) ✔ Textual and symbolic language of movement description

5. EMOTE (Badler et al) ✔ EMOTE model for Effort and Shape ✔ Based on Movement Observation Science ✔ Laban Movement Analysis ✔ Computational model of Effort and Shape components

6. Effort ✔ Four motion factors: space, weight, time, flow ✔ Space ✔ Weight ✔ Time ✔ Flow ✔ Each factor ranges from: ✔ Indulging in the quality ✔ Fighting against the quality

7. Effort ✔ Space: indirect  direct ✔ Waving away bugs  pointing to a particular spot ✔ Weight: light  strong ✔ Feather movement  punching ✔ Time: sustained  sudden ✔ Stretching  grabbing a falling object ✔ Flow: free  bound ✔ Waving widely  moving in slow motion

8. Shape ✔ Three distinct qualities of change in the form of movement: ✔ Shape flow: mover’s attitude toward the changing relationship among body parts ✔ Directional movement: mover’s intent to bridge the action to a point in the environment ✔ Shaping: mover’s carving or molding attitude with the environment

9. Shape ✔ Three dimensions: ✔ Horizontal: spreading  enclosing ✔ Opening arms to embrace  clasping someone in a hug ✔ Vertical: rising  sinking ✔ Reaching for something in a high shelf  stamping the floor with indignation ✔ Sagittal: advancing  retreating ✔ Reaching out to shake hand  Avoiding a punch

10. Expressive Communicative Gestures (Hartmann et al) ✔ Expressivity parameters: ✔ Spatial: amplitude of movement ✔ Temporal: duration of movement ✔ Power: dynamic property of movement ✔ Fluidity: smoothness and continuity of movement ✔ Repetitiveness: tendency to rhythmic repeats ✔ Overall Activation: quantity of movement across modalities ✔ Expressivity modeling: modal and intermodal

11. Emotional Modeling Carmen's Bright IDEAS ( Stacy Marsella, USC) Interactive drama Agent exhibits gestures based on their emotional states and personality traits

12. FantasyA (Ana Paiva, Inesc) Role-playing game The emotion detected from user drives the behavior of one of the agents in the video game

13. SenToy (Ana Paiva, Inesc) SenToy is a tangible interface device, used to influence emotional behaviour in FantasyA SAFIRA: EU project

14. Behavior Modeling ✔ Facial expression: ✔ Use facial analysis techniques/motion capture to look at: ✔ Value of facial animation parameters (muscular contraction, FAP…) ✔ Pattern of movement for a given parameter ✔ Relation of facial animation parameters within a same expression ✔ Coarticulation of movements between expressions

15. Mocap

16. Mocap

17. FAPs – MPEG-4

18. Example

19. Gesture ✔ Gesture specification often based on sign language gesture specification: ✔ Hand description ✔ Palm orientation ✔ Wrist position ✔ Movement: wrist, elbow, finger… ✔ Parameters characterizing quality of movement: Laban, sub-set…? ✔ Parameters act on overall animation (Emote) or on gesture phases (preparation, stroke, …)?

20. Behavior ✔ Gaze: ✔ Direction, gaze length, mutual gaze, ratio of gaze avoidance/gaze look at ✔ Body: ✔ Body posture, overall movement quality ✔ Voice: ✔ Paralinguistic parameters ✔ Text

21. Signals analysis/generation ✔ Use analysis method to: ✔ Get information on parameters ✔ Multimodal integration ✔ Signals synchrony across modalities ✔ Build computational model for generation ✔ Get the expressive quality of behavior ✔ Signals generation should not be done on one frame only but on a temporal interval

22. Conclusion ✔ Emotion is shown through: ✔ Signals ✔ Their temporal course ✔ Which parameters to look at? Which are the perceptually relevant ones? ✔ How to describe: ✔ The signals? ✔ Their evolution through time? ✔ Correlation between modalities?

23. Conclusion ✔ Relation between intensity of emotion and behaviors? ✔ Selection of behaviors ✔ Quality of behaviors ✔ Duration of behaviors ✔ Behaviors across modalities