Sylvia Pietzsch Chair for Image Understanding Computer Science Technische Universität München Learning Generic and Person Specific Objective Functions Diplomarbeit

/16 Technische Universität München Sylvia Pietzsch Overview  Model-based Image Interpretation  Generic Objective Functions  Traditional Approach  Ideal Objective Functions  Learning Objective Functions  Experimental Evaluation  Person-specific Objective Functions  Experimental Evaluation

/16 Technische Universität München Sylvia Pietzsch Model-based Image Interpretation  Model Contains a parameter vector p that represents the model‘s configurations.  Objective Function Calculates how well a parameterized model fits to an image.  Fitting Algorithm Searches for the model that fits the image best by minimizing the objective function.

/16 Technische Universität München Sylvia Pietzsch Traditional Approach  Designer selects salient features from the image and composes them.  Based on designer‘s intuition and implicit knowledge of the domain.  shortcomings:  time-consuming  resulting objective function is not ideal

/16 Technische Universität München Sylvia Pietzsch Ideal Objective Functions P1:Correctness Property: The global minimum of the objective function corresponds to the best model fit. P2:Uni-Modality Property: The objective function has no local extrema or saddle points.

/16 Technische Universität München Sylvia Pietzsch Example: Comparing Objective Functions a) image b) along perpendicular c) edge values d) designed objective function e) ideal objective function f) training samples g) learned objective function

/16 Technische Universität München Sylvia Pietzsch Learning the Objective Function (1)

/16 Technische Universität München Sylvia Pietzsch Learning the Objective Function (2)

/16 Technische Universität München Sylvia Pietzsch Learning the Objective Function (3) 6 styles · 3 sizes · (5 · 5) locations = 450 features

/16 Technische Universität München Sylvia Pietzsch Evaluation 1: Used Features  Model trees tend to select the most relevant features.  Edge-based features are hardly used at all.

/16 Technische Universität München Sylvia Pietzsch Evaluation 2: Robustness Indicators measure the fulfillment of P1 and P2: I1:Correctness Indicator Distance between the ideal position of the contour point and the global minimum of the objective function I2:Uni-Modality Indicator Total number of local minima divided by the size of the considered region

/16 Technische Universität München Sylvia Pietzsch Evaluation 3: Learning Distance

/16 Technische Universität München Sylvia Pietzsch Person Specific Objective Functions  Single Images The objective function has to take any appearance of a human face into consideration. ➱ moderate accuracy  Image Sequence The appearance of a person‘s face only changes slightly.  Consider particular characteristics of the visible person, e.g. beard, glasses, bald head,... ➱ increase of accuracy  Challenges:  Learn specific objective functions for groups of persons offline.  Detect the correct group online.

/16 Technische Universität München Sylvia Pietzsch Evaluation: Fitting Results 45 persons from news broadcasts on TV

/16 Technische Universität München Sylvia Pietzsch Outlook  Learning objective functions for 3D-Models  Integration of further image features  Compute the image features on the fly  Automatic detection of the visible person: e.g. via AAM parameters

/16 Technische Universität München Sylvia Pietzsch The End

/16 Technische Universität München Sylvia Pietzsch Improving the Accuracy of the Training Step  model tree algorithm M5‘ uses standard deviation reduction as splitting criterion  shortcomings:  solution: taking not only the distance to the ideal feature point into consideration but also the direction  bim: Diese Folie passt hier nicht. Ich würde sie ganz weglassen. Oder am Schluss nur kurz ansprechen.

/16 Technische Universität München Sylvia Pietzsch Learning Person-specific Objective Functions offline step:  subdivide persons into groups  best possible grouping: minimum error between annotated model parameters and model parameters obtained by fitting the model to a set of test images using the objective function of this grouping  learning group-specific objective functions online step:

/16 Technische Universität München Sylvia Pietzsch The End