Matthias Wimmer, Sylvia Pietzsch, Freek Stulp and Bernd Radig Chair for Image Understanding Institute for Computer Science Technische Universität München Learning Robust Objective Functions with Application to Face Model Fitting Christoph Mayer

/13 Technische Universität München Christoph Mayer Facial Expression Recognition Natural Human-Computer Interaction tactile channel visual channel audatory channel olfactory channel gustatory channel auditory channel visual channel tactile channel olfactory channel gustatory channel

/13 Technische Universität München Christoph Mayer Model-based Image Interpretation  Objective Function Calculates how well a parameterized model matches an image. The model contains a parameter vector p hat represents the model’s configuration.  Fitting Algorithm Searches for the model that matches the image best by minimizing the objective function.

/13 Technische Universität München Christoph Mayer Ideal Objective Functions P1:Correctness Property: The global minimum corresponds to the best model fit. P2:Uni-modality Property: The objective function has no local extrema. ¬ P1 P1 ¬P2 P2

/13 Technische Universität München Christoph Mayer Introducing Objective Functions a) image b) along perpendicular c) edge values d) designed objective function e) ideal objective function f) training samples g) learned objective function

/13 Technische Universität München Christoph Mayer Traditional Approach Shortcomings:  Requires domain knowledge.  Based on designer’s intuition.  Time-consuming. Manually design the objective function Manually evaluate on test images designed objective function good not good

/13 Technische Universität München Christoph Mayer Learning the Objective Function (Step 1) Manually annotate images with ideal parameterization Ideal objective function

/13 Technische Universität München Christoph Mayer Learning the Objective Function (Step 2) Manually annotate images with ideal parameterization Ideal objective function Automatically generate further image annotations result = 0 result = 0.2 result = 0.3

/13 Technische Universität München Christoph Mayer Learning the Objective Function (Step 3) Number of features: 6 styles · 3 sizes · 25 locations = 450 Locations Styles Sizes Manually annotate images with ideal parameterization Ideal objective function Automatically generate further image annotations Manually specify a set of image features

/13 Technische Universität München Christoph Mayer Learning the Objective Function (Step 4+5)  Automatically obtain calculation rules of objective function. Mapping of feature values to the value of the objective function.  Machine learning by Model Trees. Select the most relevant features. Manually annotate images with ideal parameterization Ideal objective function Automatically generate further image annotations Manually specify a set of image features Automatically generate training data learned objective function

/13 Technische Universität München Christoph Mayer Evaluation of local objective functions DesignedLearned  Evaluation of displacement and face turning.  Weak global minimum using the designed objective function.  Strong global minimum using the learned objective function.

/13 Technische Universität München Christoph Mayer Evaluation of the global function  95% of models are located at 0.12 using the learned objective function.  95% of models are located at 0.16 using a state-of-the-art approach.

/13 Technische Universität München Christoph Mayer Conclusion  Evaluation in natural dialog situations.  Application for Daimler-Crysler in car-driving situations.  Integration of a three-dimensional face model.

/13 Technische Universität München Christoph Mayer Thank you!