1. Face Model Fitting with Generic, Group-specific, and Person- specific Objective Functions Chair for Image Understanding and Knowledge-based Systems Institute for Informatics Technische Universität München Sylvia Pietzsch sylvia.pietzsch@cs.tum.edu

2. 2008, January 22 nd 2/15 Technische Universität München Sylvia Pietzsch Overview  Model-based Image Interpretation  Generic Objective Functions  Specific Objective Functions  Experimental Results  Conclusion and Outlook

3. 2008, January 22 nd 3/15 Technische Universität München Sylvia Pietzsch Model-based image interpretation  The model The model contains a parameter vector that represents the model’s configuration.  The objective function Calculates a value that indicates how accurately a parameterized model matches an image.  The fitting algorithm Searches for the model parameters that describe the image best, i.e. it minimizes the objective function.

4. 2008, January 22 nd 4/15 Technische Universität München Sylvia Pietzsch Local Objective Functions

5. 2008, January 22 nd 5/15 Technische Universität München Sylvia Pietzsch Ideal Objective Functions P1:Correctness property: Global minimum corresponds to the best fit. P2:Uni-modality property: The objective function has no local extrema. ¬ P1 P1 ¬P2 P2  Don’t exist for real-world images  Only for annotated images: f n ( I, x ) = | c n – x |

6. 2008, January 22 nd 6/15 Technische Universität München Sylvia Pietzsch Learning the Objective Function x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x  Ideal objective function generates training data  Machine Learning technique generates calculation rules

7. 2008, January 22 nd 7/15 Technische Universität München Sylvia Pietzsch Learning the Objective Function (1)

8. 2008, January 22 nd 8/15 Technische Universität München Sylvia Pietzsch Learning the Objective Function (2)

9. 2008, January 22 nd 9/15 Technische Universität München Sylvia Pietzsch Learning the Objective Function (3) 6 styles · 3 sizes · (5 · 5) locations = 450 features

10. 2008, January 22 nd 10/15 Technische Universität München Sylvia Pietzsch Specific Objective Functions  Single Images The objective function has to take any appearance of a human face into consideration. ➱ moderate accuracy  Image Sequence The appearance of a person‘s face only changes slightly.  Consider particular characteristics of the visible person, e.g. beard, glasses, bald head,... ➱ increase of accuracy  Challenges:  Learn specific objective functions for groups of persons offline.  Detect the correct group online.

11. 2008, January 22 nd 11/15 Technische Universität München Sylvia Pietzsch Evaluation: Fitting Results 45 persons from news broadcasts on TV

12. 2008, January 22 nd 12/15 Technische Universität München Sylvia Pietzsch Group-specific Objective Functions  automatically subdivide persons into partitions  learn objective function for each partition  best possible partitioning: minimize the mean fitting error when applying the group-specific objective functions to the partitions  number of partitions influences specificity

13. 2008, January 22 nd 13/15 Technische Universität München Sylvia Pietzsch Experimental Results P26 P44 P42 λ = 3.9

14. 2008, January 22 nd 14/15 Technische Universität München Sylvia Pietzsch Conclusion and Outlook  large variation in facial appearance challenges model fitting  trade-off between generality and accuracy  group-specific objective functions perform substantially better for persons within the group  of avail for tracking face models through image sequences  Further work: automatic classification to determine on-line, which partition a person belongs to

15. 2008, January 22 nd 15/15 Technische Universität München Sylvia Pietzsch Thank you!