1. Body Expression of Emotion (BEE)
Computational Model for
Body Expression of Emotion
(BEE) by
Marina Ousov-Fridin
Tamar Flash
Faculty of Mathematics and Computer Science
The Weizmann Institute of Science, Israel

2. Statement of the problem
Main Aim: To provide compact math. representation of BEE for 4 basic emotions: Sadness, Joy, Fear, Anger and build computational model. Based on it to define primitives and synergisms.
Correlation between Primitives and Perception.

3. Theory of organization of the motor system: Primitives and Synergies
Motor primitives: the entire repertoire of man actions could be constructed from limited set of building blocks (Mussa-Ivaldi [2004]):
universal
defined in terms of different state variables, coordinate frames and
may exist at different levels of representation
static, kinematical, dynamic or combined
Examples: Troje - PCA ; Mataric - static set of joint angles;….
Synergies: the coordinated control over several limb segments or multiple effectors.
coordination between different leg angles
coordination between hand and leg trajectories.
                                                                      
                                                          
                                                                                                                                                                                                                                                                                    

4. The BEE: input stream (1)
Type of Input Data
Static
Dynamic
Video
Sensors
Single person
Context-Free Simulations
Performing
Stimulating Emotional Behavior
Arts
Multiple persons
Performing, Stimulating Emotional Behavior and Arts
Acquiring spontaneous emotional behavior ۷ ۷ ۷ ۷ ۷ ۷ ۷
Naturalness ۷ ۷
Computational Complexity ۷ ۷ ۷ ۷ ۷ ۷

5. The BEE input stream (1) Type: Genuine range of subjects :
Single person ,Context-Free Simulations, Static Photos
Naturalness
Low Computational Complexity
Genuine range of subjects :
Ordinary people and actors;
Genders, ages, social backgrounds and cultural
influences;
Build in

7. The BEE input stream (2) Main Problem: Uncertainty and uniqueness
Observer
Performer
Performers present the apex of the expression by their opinion in the portrayed BEE
RecognitionRatepicture
Subjects: 27 (R.V. Lab)
18 (~20 y. old)
24 (14-17 y. old)
72 (artists)
IntensityRatepicture
Subjects: 21 (R.V. Lab)

8. Human Body Parts Position Estimation
Vision Processing
Extract candidate features :
Human Vision-based analysis
Tracking
Human Body Parts Position Estimation
Tracking Body (Segmentation)
Tracking Body Parts (Labeling)
Body model estimation
Tracking Heads/Faces
Head Position Estimation
Tracking Hands
Gesture Recognition

9. Select important features/primitives
Information measurement criteria:
I(C; F) = H(C) – H (C|F)
is amount of information delivered by a candidate primitive about the class of emotion
Primitive definition (F)
Bank of primitives (B)
data set belong to class
otherwise
Class
Non Class

10. Similarity Function Similarity measurement (S)
Normalized Cross Correlation
Distance between turning
function of polygon
Primitive as binary variables (Fi)

11. Mutual Information Algorithm
MI and associated threshold
The mutual information between the primitive and the class of a emotion (I)
Similarity Threshold ( )
Max-Min Algorithm
First Primitive
K-primitive: max additional information

12. General Scheme S Max - Min Anger : Fear : Joy : Sadness : . . . . . .

13. Results: Body Joy (Join Angles) Recognition Rate MI Max-Min JA
Anger
Fear
Joy
Sadness
0.1755
0.1484
0.1755
0.0108 JA
Sim. Measur. TA
0.1467
0.1431
0.0827
0.0960
0.1408
0.1408
0.1215
0.0781

14. Sadness: Right Gesture
Results: Gesture
Sadness: Right Gesture
Recognition Rate
Anger
Fear
Joy
Sadness
Right Hand
Extract Edge : primitives are gesture polygon
Similarity Measurement : distance between turning function
Left Hand
Max-Min

15. Additional Feature (1) Head Body Feature Vision processing Pith
Body boundary box normalized by silhouette
Body tendency
Vision processing
Segmentation: background subtraction
Find Body Silhouette: morphology and edge detection
Head
Pith
Roll
Tracking head: Skin detection
Approximation to ellipse
on HSV color space

16. How to combine all possible
Additional Feature (2)
Anger
Fear
Joy
Sadness
Head
How to combine all possible
feature?
Body

17. BEE Perception and correlation to Computational model
Rating database; influence of rating on computational model.
Class differences:
Gender
Actors
Culture influences
Contributes to emotion theory
Response time – Wrong/Right Recognition
Response Time - Emotion

18. Conclusion
exists computational model, describes static BEE (for 4 basic emotion) by compact representation, when features (primitives) are selected by computational measurement (MI)

19. Thank you for your attention!