Faces: Analysis and Synthesis Vision for Graphics CSE 590SS, Winter 2001 Richard Szeliski

2/14/2001Vision for Graphics2 What can we do with faces? Modeling (reconstruction): manual [Pighin et al. 1998] Automated [Zhang et al. 2000]

2/14/2001Vision for Graphics3 What can we do with faces? Analysis principal components and deformation modes [Turk & Pentland 1991] [Rowland & Perrett 1995] [Guenter et al. 1998] [Blanz & Vetter 1999]

2/14/2001Vision for Graphics4 What can we do with faces? Tracking and synthesis tracking [Toyama 1998] animation [Pighin et al. 1999] [Buck et al. 2000] Recognition [Turk & Pentland 1991; Lanitis et al. 1997]

2/14/2001Vision for Graphics5 Synthesizing Realistic Facial Expressions from Photographs Frederic Pighin Jamie Hecker Dani Lischinski v David Salesin Richard Szeliski * SIGGRAPH’98

Animated Face Modeling From Video Images Zhengyou Zhang, Zicheng Liu, Michael Cohen Vision Group & Graphics Group Microsoft Research

Manipulating Facial Appearance through Shape and Color Duncan A. Rowland and David I. Perrett St Andrews University IEEE CG&A, September 1995

2/14/2001Vision for Graphics8 Principal component analysis Compute average faces (color and shape) Compute deviations between male and female (vector and color differences)

2/14/2001Vision for Graphics9 Changing gender Deform shape and/or color of an input face in the direction of “more female” original shape colorboth

2/14/2001Vision for Graphics10 Enhancing gender more same original androgynous more opposite

2/14/2001Vision for Graphics11 Changing age Face becomes “rounder” and “more textured” and “grayer” original shape colorboth

A Morphable Model For The Synthesis Of 3D Faces Volker Blanz Thomas Vetter SIGGRAPH’99

2/14/2001Vision for Graphics13 Morphable model of 3D faces Start with a catalogue of 200 3D Cyberware scans Build a model of average shape and texture, and principal variations

2/14/2001Vision for Graphics14 Morphable model of 3D faces Divide face into 4 regions (eyes, nose, mouth, head) For each new prototype, find amount of deviation from the reference shape and texture.

2/14/2001Vision for Graphics15 Morphable model of 3D faces Adding some variations

2/14/2001Vision for Graphics16 Reconstruction from single image

2/14/2001Vision for Graphics17 Modifying a single image

2/14/2001Vision for Graphics18 Animating from a single image

2/14/2001Vision for Graphics19 Resulting animation

Tracking Face Orientation Kentaro Toyama Vision–Based Interaction Group Microsoft Research

2/14/2001Vision for Graphics21 1 University of Washington 2 Microsoft Research ICCV’99 Resynthesizing Facial Animation through 3D Model-Based Tracking Frédéric Pighin 1 Richard Szeliski 2 David Salesin 1,2

Performance-Driven Hand-Drawn Animation Ian Buck Adam Finkelstein Charles Jacobs Allison Klein David H. Salesin Joshua Seims Richard Szeliski Kentaro Toyama

2/14/2001Vision for Graphics23 Go from video to “cartoon”

2/14/2001Vision for Graphics24 Hand-drawn expressions

2/14/2001Vision for Graphics25 Expression correspondence

2/14/2001Vision for Graphics26 Morphing and tracking

2/14/2001Vision for Graphics27 More examples

2/14/2001Vision for Graphics28 Final animations

2/14/2001Vision for Graphics29 Bibliography F. Pighin, J. Hecker, D. Lischinski, D. H. Salesin, and R. Szeliski. Synthesizing realistic facial expressions from photographs. In SIGGRAPH'98 Proceedings, pages , Orlando, July Z. Liu, Z. Zhang, C. Jacobs, and M. Cohen. Rapid modeling of animated faces from video. Technical Report MSR-TR , Microsoft Research, February B. Guenter et al. Making faces. Proceedings of SIGGRAPH 98, pages , July V. Blanz and T. Vetter. A morphable model for the synthesis of 3d faces. Proceedings of SIGGRAPH 99, pages , August K. Toyama. Prolegomena for robust face tracking. Technical Report MSR-TR-98-65, Microsoft Research, November F. Pighin, D. H. Salesin, and R. Szeliski. Resynthesizing facial animation through 3D model-based tracking. In Seventh International Conference on Computer Vision (ICCV'99), pages , Kerkyra, Greece, September 1999.

2/14/2001Vision for Graphics30 Bibliography I. Buck et al. Performance-driven hand-drawn animation. In Symposium on Non Photorealistic Animation and Rendering, pages , Annecy, June ACM SIGGRAPH. D. A. Rowland and D. I. Perrett. Manipulating facial appearance through shape and color. IEEE Computer Graphics and Applications, 15(5): , September M. Turk and A. Pentland. Face recognition using eigenfaces. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'91), pages , Maui, Hawaii, June IEEE Computer Society Press. P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman. Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7): , July A. Lanitis, C. J. Taylor, and T. F. Cootes. Automatic interpretation and coding of face images using flexible models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7): , July 1997.