1. Automatic Pose Estimation of 3D Facial Models Yi Sun and Lijun Yin Department of Computer Science State University of New York at Binghamton Binghamton, New York, 13902 USA 19 th International Conference on Pattern Recognition December 8 th, 2008

2. 19th International Conference on Pattern Recognition 2 Introduction Pose estimation plays an essential role in many computer vision applications, such as human computer interaction (HCI), monitoring driver attentiveness, face recognition, and automatic model editing. 2D image/Videos based. Active infra-red illumination based. 3D facial models based

3. December 8th, 200819th International Conference on Pattern Recognition 3 Motivation Motivation Problem? Invariant to illumination, robust to pose variations, deal with different expressions, none-facial outliers, noise, partial facial data missing, etc. Geometric surface representation Identify facial features (inner eye corners, nose tip). Machine learning plus structure based to estimate pose orientation.

4. December 8th, 200819th International Conference on Pattern Recognition 4 3D facial expressions database – samples First row: raw faces; second row: clean faces First row: 3D textured faces; second row: 3D shaded faces. From left to right: neutral, anger, disgust, fear, happy, sad, and surprise.

5. December 8th, 200819th International Conference on Pattern Recognition 5 Observation

6. December 8th, 200819th International Conference on Pattern Recognition 6 System framework

7. December 8th, 200819th International Conference on Pattern Recognition 7 Candidate eye inner corner Use decision tree method to determine the proper threshold Use decision tree method to determine the proper threshold

8. December 8th, 200819th International Conference on Pattern Recognition 8 Noise erase (1) Erase candidate far away from others Erase candidate far away from others Erase points having limited number of neighboring candidates Erase points having limited number of neighboring candidates

9. December 8th, 200819th International Conference on Pattern Recognition 9 Inner eye corner clustering Apply 2-means clustering approach to find the two inner eye corners Apply 2-means clustering approach to find the two inner eye corners

10. December 8th, 200819th International Conference on Pattern Recognition 10 Sparse candidate elimination

11. December 8th, 200819th International Conference on Pattern Recognition 11 Noise tip determination (1) Fit a flat surface (reference plane) to: to two clusters by solving the optimization problem: Where, n is the number of candidates in two clusters

12. December 8th, 200819th International Conference on Pattern Recognition 12 Noise tip determination (2)

13. December 8th, 200819th International Conference on Pattern Recognition 13 Symmetry plane

14. December 8th, 200819th International Conference on Pattern Recognition 14 Automatic frontal view transform

15. December 8th, 200819th International Conference on Pattern Recognition 15 Experiments Tested with 2500 range models from BU-3DFE database Tested with 2500 range models from BU-3DFE database Tested with both raw facial models and clean facial models Tested with both raw facial models and clean facial models Estimated pose orientation less than 5 degrees - correct Estimated pose orientation less than 5 degrees - correct Raw data: 92.1% Raw data: 92.1% Clean data: 96.4% Clean data: 96.4%

16. December 8th, 200819th International Conference on Pattern Recognition 16 Sample results (1) Same subject, different poses

17. December 8th, 200819th International Conference on Pattern Recognition 17 Sample results (2) Different subjects, different poses

18. December 8th, 200819th International Conference on Pattern Recognition 18 Sample results (3) Same subject with clean and raw models Same subject with clean and raw models

19. December 8th, 200819th International Conference on Pattern Recognition 19 Sample results (4) Different expressions Different expressions

20. December 8th, 200819th International Conference on Pattern Recognition 20 Conclusion Propose a fully automatic 3D face pose estimation approach. Based on 3D wire-frame model. Feasible with respect to various subjects, large pose variations, different expressions, and data with noise/none-facial outliers.

21. December 8th, 200819th International Conference on Pattern Recognition 21 Acknowledgement This material is based upon the work supported in part by the National Science Foundation under grants IIS-0414029 and IIS-0541044, and the NYSTAR's James D. Watson Investigator Program.

22. December 8th, 200819th International Conference on Pattern Recognition 22 Thank you!