1. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Topic 5. Human Faces Human face is extensively studied in vision. Depending on the applications, there are a long list of tasks [5]: 1.Detection and Recognition: Face detection (finding all faces in a picture), facial feature detection (eyes, lips, …), Face localization (detecting a single face in image), Face recognition or identification (from a database, classification) Face authentication (verifying claim, bank id), Age/gender recognition, Face tracking (location and pose over time) Facical expression recognition (affective states), aesthetic study. 2.Modeling and Photorealistic Synthesis: Appearance models, deformable templates, lighting models, facial action units, face hallucination (high resolution from low resolution), pose adjustment, image editing (removing wrinkles, eye glass, red-eye etc.) 3. Artistic rendering Sketch, portrait, caricature, cartoon, painting, …

2. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Face Image Databases The CMU Rowley dataset

3. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Face Image Databases The CMU Schneidrman and Kanade Dataset

4. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu References. 1. P. Hallinan, G. Gordon, A. Yuille, P. Giblin, and D. Mumford, 2D and 3D Patterns of the Face, A.K. Peters, Ltd. Book chapters 2-4. (handouts). 2. D.H. Ballard, "Generaling the Hough transform to detect arbitrary shapes", (in handbook). 3. P. Viola and M. Jones, "Robust Real Time Object Detection", 4. F. Fleuret and D. Geman, " Coarse-to-fine face detection", IJCV 41(1/2),2001. 5. M.H. Yang, D. Kriegman, N. Ahuja, “Detecting faces in images, a survey”, PAMI vol.24,no.1, January, 2002. 6 T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", ECCV 1998 7. C. Liu, S. C. Zhu, and H. Y. Shum, "Learning inhomogeneous Gibbs models of faces by minimax entropy", ICCV 2001. 8. Y. Tian, T. Kanade, and J. Cohn, "Recognizing action units for facial expression analysis" PAMI, Feb, 2001. 9. H. Chen, Y. Q. Xu, H. Y. Shum, S. C. Zhu, and N. N. Zhen, "Example-based facial sketch generation with non-parametric sampling", ICCV 2001.

5. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Outline We proceed in three steps: A survey on face detection and recognition techniques 2.Mathematical models of face images 3. Face synthesis: photorealistic and non-photorealistic.

6. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Face Detection Methods [5]

7. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Face vs non-face Clsutering 6 clusters in a 19 x19 space (Sung and Poggio)

8. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Distance Measure D1D1 D2D2 For each input image, it measures two distances for each cluster center: D1 is the Mahalanobis distance and D2 is the Euclidean distance. Thus Sung and poggio have 2 x 6 x 2 = 24 features for classification in a multiple layer perceptron.

9. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Deformable Face Template Deformable face template by Fishler and Elschlager 1973. M. Fishler and R. Elschlager, “The representation and matching of pictorial structures”, IEEE Trans. on Computer. Vol.C-22, 67-92, 1973.

10. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Local Deformation and Global Transform Geometric variations of faces: (Hallinan, Yuille, Mumford et al)

11. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Deformable Model of Facial Features Eye template using parabolic curves by Yuille et al 1989-92. A.L.Yuille, D. Cohen, and P.Hallinan, “Feature extraction from faces using deformable templates”, CVPR 89, IJCV 92. We can derive meaningful diffusion equations from the energy functionals.

12. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Upper Face Action Units

13. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Lower Face Action Units

14. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Templates for Various States

15. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Templates for Various States

16. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Features for Action Unit Recognition

17. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Classification from Feature Vector

18. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Recognition Rate

19. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Apparence Model: Landmarks on a face 400 images each labeled with 122 points.

20. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Eigen-vectors for Geometry and Photometry

21. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Apparence Model

22. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Face Localization and Recognition

23. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu A Linear HMM Model for Face

24. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Face Detection

25. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Sample of the 4D space

26. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Multi-scale Detection

27. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Edge Features

28. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Decision Tree

29. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Examples of Decision Trees

30. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Bounds Analysis

31. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Some Examples

32. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Face Prior Learning: Experimental Details 83 key points defined on face 720 individuals with all kinds of types Dimension reduced to 33 by PCA 40000 samples drawn by the inhomogeneous Gibbs sampler in each Monte Carlo integration 50 features pursuit Total runtime: about 5 days on a PIII 667, 256MB PC

33. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Obs & Syn Samples (1) Observed faces Synthesized faces without any features

34. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Synthesis Samples Synthesized faces with 20 features Synthesized faces with 10 features

35. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu Synthesis Samples Synthesized faces with 30 features Synthesized faces with 50 features

36. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu 50 Observed Histograms

37. TTH 1:30-2:48 Winter 01-02 DL266 http://www.cis.ohio-state.edu/~szhu/cis788_2002/ CIS 788v04 Zhu 50 Synthesized Histograms