Wangfei Ningbo University A Brief Introduction to Active Appearance Models

Wangfei Ningbo University Topics of the talk IntroductionAAM Future and Related works Reference

Wangfei Ningbo University Introduction What is AAM? Non-linear, generative, parametric models What can AAM do? Statistical models Depend on the problem Computer Vision Image Interpretation Face Recognition Medical image analysis

Wangfei Ningbo University Introduction-Application Face Recognition Figure: Example face image annotated with landmarks

Wangfei Ningbo University Introduction-Application Medical image analysis Figure: Example MR image of knee with carilage outlined

Wangfei Ningbo University Introduction-History History Snake (Active Contour Models) ASM (Active Shape Models) AAM (Active Appearance Models)

Wangfei Ningbo University Snake- Active Contour Models Start with a curve near the object Discrete snake: spline with n control points Evolve the curve to fit the boundary Minimize the energy function Original formulation

Wangfei Ningbo University Snake Weakness weak constraints high compute cost can not search inside boundary not optimal for known shape because of no prior knowledge

Wangfei Ningbo University ASM-Active Shape Models Use prior knowledge from the training set Variable parameters Statistical Shape Models Allow formal statistical techniques to be applied to sets of shapes, making possible analysis of shape differences and changes

Wangfei Ningbo University ASM Variable parameters position shape parameters scale orientation

Wangfei Ningbo University ASM Shape define the shapes as the coordinates of the v vertices that make up the mesh: AAM allow linear shape variation the shape parameters

Wangfei Ningbo University ASM The linear shape model of an independent AAM

Wangfei Ningbo University ASM Build the model Get shapes from a set of annotated images of typical examples NormalizePCA

Wangfei Ningbo University ASM Use the model for locating Given a rough starting approximation instance Examine a region around, find the best nearby match for each point Update the parameters to best fit the new point Repeat until convergence

Wangfei Ningbo University ASM Figure: Search using Active Shape Model of a face

Wangfei Ningbo University AAM-Active Appearance Models ShapeAppearance Model Instantiation Fitting

Wangfei Ningbo University AAM Appearance Warp each example image SampleNormalizePCA

Wangfei Ningbo University AAM Warp each example image Its control points match the mean shape Using Piecewise affine warping (Delaunay Triangulation algorithm) Thin plate splines Sample The intensity information from shape-normalized image to form a texture vector

Wangfei Ningbo University AAM Figure: a ‘shape-free’ image patch

Wangfei Ningbo University AAM Normalize To minimize the effect of global lighting variation PCA The appearance expression the appearance parameters

Wangfei Ningbo University AAM Figure: The linear appearance variation of an independent AAM

Wangfei Ningbo University AAM Model Instantiation The two equations describe the shape and the appearance variation Given the shape parameters Given the appearance parameters Create warping appearance A from the base mesh S 0 to the model shape S

Wangfei Ningbo University AAM Figure: An example of AAM instantiation

Wangfei Ningbo University AAM Fitting Naturally, we want to minimize the error between and Denote as:

Wangfei Ningbo University AAM Fitting Algorithms Inefficient Gradient Descent Algorithms Efficient Ad-Hoc Fitting Algorithms Efficient Gradient Descent Image Alignment Lucas-Kanade Image Alignment Forwards Compositional Image Alignment Inverse Compositional Image Alignment...

Wangfei Ningbo University Future and Related works Alignment algorithms Automatic landmark View-Based appearance models Applications…

Wangfei Ningbo University Reference T.F. Cootes and C.J. Taylor Statistical Models of Appearance for computer vision Active Appearance Models Active Shape Models-Their Training and Application Iain Matthews and Simon Baker Active Appearance Models Revisited …