1. Selective Transfer Machine for Personalized Facial Action Unit Detection Wen-Sheng Chu, Fernando De la Torre and Jeffery F. Cohn Robotics Institute, Carnegie Mellon University July 9, 2013 1

2. AU 6+12 Facial Action Units (AU) 2

3. Main Idea 3

4. Related Work: Features 4

5. Related Work: Classifiers 5

6. Feature Bias Person specific! 6

7. Occurrence Bias 7

8. Selective Transfer Machine (STM) Formulation Maximizes margin of penalized SVM Minimize distribution mismatch 8

9. Goal (1): Maximize penalized SVM margin margin penalized loss 9

10. Goal (2): Minimize Distribution Mismatch Kernel Mean Matching (KMM)* 10 * “Covariate shift by kernel mean matching”, Dataset shift in machine learning, 2009.

11. Goal (2): Minimize Distribution Mismatch Groundtruth Bad estimator for testing data! 11

12. Better fitting! Groundtruth Selection by reweighting training data 12 Goal (2): Minimize Distribution Mismatch

13. 13

14. 14 Optimization: Alternate Convex Search

15. 15 Optimization: Alternative Convex Search

16. Compare with Relevant Work 16 [1] "Covariate shift by kernel mean matching," Dataset shift in machine learning, 2009. [2] "Transductive inference for text classification using support vector machines," In ICML 1999. [3] "Domain adaptation problems: A DASVM classification technique and a circular validation strategy," PAMI 2010.

17. Experiments Features – SIFT descriptors on 49 facial landmarks – Preserve 98% energy using PCA 17 Datasets#Subjects#Videos#Frm/vidContent CK+123593~20Neutral  Peak GEMEP-FERA78720~60Acting RU-FACS29 5000~7500Interview

18. Experiment (1): Synthetic Data 18

19. Two protocols – PS 1 : train/test are separate data of the same subject – PS 2 : training subjects include test subject (same protocol in [2]) GEMEP-FERA Experiment (2): Comparison with Person- specific (PS) Classifiers 19

20. Experiment (2): Selection Ability of STM 20

21. 123 subjects, 597 videos, ~20 frames/video Experiment (3): CK+ 21

22. Experiment (4): GEMEP-FERA 22 7 subjects, 87 videos, 20~60 frames/video

23. 29 subjects, 29 videos, 5000~7000 frames/vid Experiment (5): RU-FACS 23

24. Summary Person-specific biases exist among face- related problems, esp. facial expression We propose to alleviate the biases by personalizing classifiers using STM Next – Joint optimization in terms of – Reduce the memory cost using SMO – Explore more potential biases in face problems, e.g., occurrence bias 24

25. Questions? [1] "Covariate shift by kernel mean matching," Dataset shift in machine learning, 2009. [2] "Transductive inference for text classification using support vector machines," In ICML 1999. [3] "Domain adaptation problems: A DASVM classification technique and a circular validation strategy," PAMI 2010. [4] “Integrating structured biological data by kernel maximum mean discrepancy”, Bioinformatics 2006. [5] “Meta-analysis of the first facial expression recognition challenge,” IEEE Trans. on Systems, Man, and Cybernetics, Part B, 2012. 25 http://humansensing.cs.cmu.edu/wschu/