1. Learning the Appearance of Faces: A Unifying Approach for the Analysis and Synthesis of Images. Thomas Vetter Germany University of Freiburg http://graphics.informatik.uni-freiburg.de

2. Computer Vision & Computer Graphics Computer Graphics can help to solve Computer Vision! | G (p) - I | 2 = min Parameters G ( image ) Parameters V ision ( image ) parameters image G raphics ( parameters )

5. Analysis by Synthesis 3D World Image Analysis Synthesis Image Model Image Description model parameter

6. Synthesis of Faces Input Image Modeler Result Database Face Analyzer 3D Head Morphable Face Model

7. Approach: Example based modeling of faces 2D Image 3D Face Models = w 1 * + w 2 * + w 3 * + w 4 * +...

8. Cylindrical Coordinates red(h,  ) green(h,  ) blue(h,  ) red(h,  ) green(h,  ) blue(h,  )  h radius(h,  ) h 

9. Morphing 3D Faces 3D Blend 3D Morph 1 __ 2 1 2 + =

10. Correspondence: A two step process! Correspondence between 1. two examples ( Optical Flow like algorithms). 2. many examples ( Morphable Model ) Reference Example 2 nd Example

11. = a 1 * + a 2 * + a 3 * + a 4 * +... b 1 * + b 2 * + b 3 * + b 4 * +... Vector space of 3D faces. A Morphable Model can generate new faces.

12. Manipulation of Faces Modeler

13. Modelling in Face Space Caricatur Original Average

14. Modelling the Appearance of Faces A face is represented as a point in face space. Which directions code for specific attributes ?

15. Learning from Labeled Example Faces Fitting a (linear) regression function

16. Facial Attributes WeightWeight OriginalOriginal Subjective Attractiveness

17. Transfer of Facial Expressions = Smile - - Originals: + Smile = Novel Face:

18. Facial Expressions OriginalOriginal

19. 3D Shape from Images Face Analyzer 3D Head Input Image

20. Matching a Morphable 3D-Face-Model = R Optimization problem! a 1 * + a 2 * + a 3 * + a 4 * +.. b 1 * + b 2 * + b 3 * + b 4 * +..

21. Error Function Image difference Plausible parameters Minimize

22. Optimization Strategies Stochastic Gradient Decent Difference Decomposition

23. Future Challenges Which Object Classes are linear ? How to built them automatically?