1. Akshay Asthana, Jason Saragih, Michael Wagner and Roland G öcke ANU, CMU & U Canberra In part funded by ARC grant TS0669874

2. Background Thinking Head project http://thinkinghead.edu.au/ 5-year multi-institution (Canberra, UWS, Macquarie, Flinders) project in Australia Develop a research platform for human communication sciences “An Approach for Automatically Measuring Facial Activity in Depressed Subjects”, McIntyre, G öcke, Hyett, Green, Breakspear, ACII 2009

3. Aim for this Study Active Appearance Models (AAM) have become a popular tool for markerless face tracking in recent years A number of different AAM fitting methods exist Which one should we use? We wanted to evaluate these in the context of facial expression recognition (FER) How well do AAMs generalise? How robust are these methods w.r.t. initialisation error? How does their fitting accuracy affect the FER accuracy?

4. AAM Shape: Texture:

5. AAM – Shape Variation Shape variation Mean

6. AAM – Texture Variation Texture variation Mean

7. AAM – Modelling Appearance Appearance = Shape + Texture Mean

8. AAM (cont.) Alignment based on finding model parameters that iteratively fit learnt model to the image InitialisationAfter 5 iterationsConverged

9. AAM Fitting Methods Compared in this Study Fixed Jacobian (FJ): Cootes, Edwards & Taylor, 1998 Project-Out Inverse Compositional (POIC): Baker & Matthews, 2001 Simultaneous Inverse Compositional (SIC): Baker, Gross & Matthews, 2003 Robust Inverse Compositional (RIC): Gross, Matthews & Baker, 2005 Iterative Error-Bound Minimisation (IEBM), aka Linear Discriminative-Iterative: Saragih & Goecke, 2006 Haar-like Feature Based Iterative-Discriminative Method (HFBID): Saragih & Goecke, 2007

10. System Overview 1 2

11. Experiments (1) Generalisation, (2) Robustness to initialisation error Person-dependent models (PDFER): individual models Person-independent models (PIFER): general models Not for POIC as has previously been shown to not generalise well across different people Cohn-Kanade database: Subset of 30 subjects (15f / 15m) Total of 3424 images: 992 images for Neutral, 448 images for Anger, 296 images for Disgust, 346 images for Fear, 532 images for Joy, 423 images for Sorrow and 387 images for Surprise.

12. Initialisation Traditionally, beside generalisation, one of the most challenging problems for AAMs has been robustness to initialisation error Common face detectors, e.g. Viola-Jones, often give you an error (translation) of up to 30 pixels We simulate this by deliberately misaligning the initial AAM: ±5, ±10, ±20, ±25 (PIFER) / ±30 pixels (PDFER) Multi-class SVM using a linear kernel for PDFER and a Radial Basis Function kernel for PIFER Classify expressions as Neutral or one of the ‘Big 6’ (7- class problem

13. Facial Expression Recognition In this study, we were interested in recognising the ‘Big 6’ + Neutral expressions Since the scope of most of the vision based expression recognition systems is based on changes in appearance, we grouped AUs together on a ‘regional basis’ In that way, we did not have to recognise individual AUs but analysed movement patterns in various facial regions, which made the FER process more robust

14. FER (2)

15. FER Results - Video Ground truth

16. Results – Person-dependent Models Stable “Unstable”

17. Results – Person-independent Models Stable “Unstable”

18. Conclusions Investigate the utility of different AAM fitting algorithms in the context of real-time FER Iterative-Discriminative (ID) approach adopted in IEBM and HFBID boosts the fitting performance significantly and thus leads to improved FER results More robust to initialisation error than other methods IEBM and HFBID generalise well Rapid fitting (real-time capable) ~ as fast as POIC Future work: Pose-invariant FER