1. Faces: Analysis and Synthesis Vision for Graphics CSE 590SS, Winter 2001 Richard Szeliski

2. 2/14/2001Vision for Graphics2 What can we do with faces? Modeling (reconstruction): manual [Pighin et al. 1998] Automated [Zhang et al. 2000]

3. 2/14/2001Vision for Graphics3 What can we do with faces? Analysis principal components and deformation modes [Turk & Pentland 1991] [Rowland & Perrett 1995] [Guenter et al. 1998] [Blanz & Vetter 1999]

4. 2/14/2001Vision for Graphics4 What can we do with faces? Tracking and synthesis tracking [Toyama 1998] animation [Pighin et al. 1999] [Buck et al. 2000] Recognition [Turk & Pentland 1991; Lanitis et al. 1997]

5. 2/14/2001Vision for Graphics5 Synthesizing Realistic Facial Expressions from Photographs Frederic Pighin Jamie Hecker Dani Lischinski v David Salesin Richard Szeliski * SIGGRAPH’98

6. Animated Face Modeling From Video Images Zhengyou Zhang, Zicheng Liu, Michael Cohen Vision Group & Graphics Group Microsoft Research

7. Manipulating Facial Appearance through Shape and Color Duncan A. Rowland and David I. Perrett St Andrews University IEEE CG&A, September 1995

8. 2/14/2001Vision for Graphics8 Principal component analysis Compute average faces (color and shape) Compute deviations between male and female (vector and color differences)

9. 2/14/2001Vision for Graphics9 Changing gender Deform shape and/or color of an input face in the direction of “more female” original shape colorboth

10. 2/14/2001Vision for Graphics10 Enhancing gender more same original androgynous more opposite

11. 2/14/2001Vision for Graphics11 Changing age Face becomes “rounder” and “more textured” and “grayer” original shape colorboth

12. A Morphable Model For The Synthesis Of 3D Faces Volker Blanz Thomas Vetter SIGGRAPH’99

13. 2/14/2001Vision for Graphics13 Morphable model of 3D faces Start with a catalogue of 200 3D Cyberware scans Build a model of average shape and texture, and principal variations

14. 2/14/2001Vision for Graphics14 Morphable model of 3D faces Divide face into 4 regions (eyes, nose, mouth, head) For each new prototype, find amount of deviation from the reference shape and texture.

15. 2/14/2001Vision for Graphics15 Morphable model of 3D faces Adding some variations

16. 2/14/2001Vision for Graphics16 Reconstruction from single image

17. 2/14/2001Vision for Graphics17 Modifying a single image

18. 2/14/2001Vision for Graphics18 Animating from a single image

19. 2/14/2001Vision for Graphics19 Resulting animation

20. Tracking Face Orientation Kentaro Toyama Vision–Based Interaction Group Microsoft Research

21. 2/14/2001Vision for Graphics21 1 University of Washington 2 Microsoft Research ICCV’99 Resynthesizing Facial Animation through 3D Model-Based Tracking Frédéric Pighin 1 Richard Szeliski 2 David Salesin 1,2

22. Performance-Driven Hand-Drawn Animation Ian Buck Adam Finkelstein Charles Jacobs Allison Klein David H. Salesin Joshua Seims Richard Szeliski Kentaro Toyama

23. 2/14/2001Vision for Graphics23 Go from video to “cartoon”

24. 2/14/2001Vision for Graphics24 Hand-drawn expressions

25. 2/14/2001Vision for Graphics25 Expression correspondence

26. 2/14/2001Vision for Graphics26 Morphing and tracking

27. 2/14/2001Vision for Graphics27 More examples

28. 2/14/2001Vision for Graphics28 Final animations

29. 2/14/2001Vision for Graphics29 Bibliography F. Pighin, J. Hecker, D. Lischinski, D. H. Salesin, and R. Szeliski. Synthesizing realistic facial expressions from photographs. In SIGGRAPH'98 Proceedings, pages 75--84, Orlando, July 1998. Z. Liu, Z. Zhang, C. Jacobs, and M. Cohen. Rapid modeling of animated faces from video. Technical Report MSR-TR-2000-11, Microsoft Research, February 2000. B. Guenter et al. Making faces. Proceedings of SIGGRAPH 98, pages 55- -66, July 1998. V. Blanz and T. Vetter. A morphable model for the synthesis of 3d faces. Proceedings of SIGGRAPH 99, pages 187--194, August 1999. K. Toyama. Prolegomena for robust face tracking. Technical Report MSR-TR-98-65, Microsoft Research, November 1998. F. Pighin, D. H. Salesin, and R. Szeliski. Resynthesizing facial animation through 3D model-based tracking. In Seventh International Conference on Computer Vision (ICCV'99), pages 143--150, Kerkyra, Greece, September 1999.

30. 2/14/2001Vision for Graphics30 Bibliography I. Buck et al. Performance-driven hand-drawn animation. In Symposium on Non Photorealistic Animation and Rendering, pages 101--108, Annecy, June 2000. ACM SIGGRAPH. D. A. Rowland and D. I. Perrett. Manipulating facial appearance through shape and color. IEEE Computer Graphics and Applications, 15(5):70- -76, September 1995. M. Turk and A. Pentland. Face recognition using eigenfaces. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'91), pages 586--591, Maui, Hawaii, June 1991. IEEE Computer Society Press. P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman. Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):711-- 720, July 1997. A. Lanitis, C. J. Taylor, and T. F. Cootes. Automatic interpretation and coding of face images using flexible models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):742--756, July 1997.