Active Appearance Models theory, extensions & cases Mikkel B. Stegmann Master thesis presentation IMM September 12th 2000

Aim & Method High precision segmentation of non-rigid objects in digital images Active Appearance Models: A deformable template model A priori knowledge learned through statistical analysis of a training set

Presentation outline Method Case Contributions Results Discussion

Shape analysis Shape is represented by a linear spline of landmarks X = ( x1, x2, … , xn, y1, y2, … , yn)T Alignment w.r.t. position, scale, orientation Principal component analysis Compact shape representation

Shape analysis - cont’d Principal component analysis on N aligned shapes Mean shape Covariance matrix Eigen problem Shape deformation

Texture analysis Texture – the intensities across the object – is sampled inside the shape using a suitable warp function Warp function: A piece-wise affine warp using the Delaunay triangulation g = ( x1, … , xm)T Principal component analysis Compact texture representation

Combined model Shape and texture is combined into a compact model representation capable of deforming in a similar manner to what is observed in the training set Shape & texture deformation Model instance construction Obtaining PCA scores

Metacarpals — a case study 24 x-ray images of the human hand with metacarpal 2, 3, 4 annotated using 50 landmarks each

Modes of variation Shape deformation 1st shape mode Texture deformation 1st texture mode Combined deformation 1st combined mode

Search

Contributions Handling of homogeneous shapes Handling of heterogeneous shapes Search-based initialisation Fine-tuning of the model fit Applying robust statistics to the optimisation Unification of Finite Element Models and AAMs

Homogeneous shapes Increases texture specificity by adding a suitable neighborhood region

Heterogeneous shapes Models the outer border only to avoid large-scale texture noise Normal AAM Border AAM Registration

Search-based initialisation Performs a sparse search using the standard AAM search Parameters: c, position, scale, rotation (constrained) Survival of the fittest: Generates a set of candidates, which are further investigated until one candidate remains (evolved guessing) Image sampling point (candidate) Actual dy (pixels) Predicted dy (pixels)

Fine-tuning of the model fit Using general purpose optimisation techniques: Gradient based methods Steepest descent Conjugate gradient Quasi-Newton (BFGS) Non-gradient based methods Pattern search Random-sampling based methods Simulated annealing (SA) Evaluation criterion The similarity measure between model and image: Parameters c, position, scale, rotation

Robust optimisation Sensitivity to texture outliers reduced using robust similarity measures in the optimisation Quadratic Lorentzian Quadratic Lorentzian

Unification of Finite Element Models and AAMs Wanted: flexibility inside shapes independently of landmark movement Accomplished using artificial interior points deformed by a finite element model

Results Radiographs — the human hand Cardiac MRI — left ventricle 1st combined mode Radiographs — the human hand Landmark accuracy Basic AAM 0.88 pixels Neighbourhood + SA 0.82 pixels 23 images / 150 landmarks / ~13.000 pixels Cardiac MRI — left ventricle Basic AAM 1.18 pixels SA 1.06 pixels 13 images / 66 landmarks / ~2.200 pixels Perspective Images — pork carcasses Basic AAM 1.12 pixels Border AAM 0.86 pixels 13 images / 83 landmarks / ~3.500 pixels

Discussion “Hidden” benefits Requirements Analyses of Variance (group/longitudinal studies) Classification/interpretation using the PCA parameters Automatic registration Requirements Landmarks (point correspondence) Distinct features The AAM approach extends to 3D and multivariate imaging

Conclusion AAM has been explored, documented and extended as a fully automated and data-driven approach towards image segmentation Thorough evaluation has shown that AAM performs well on different segmentation problems and different image modalities (x-ray, MRI etc.) Properties General Specific Robust Non-parametric (no knobs) Self-contained Fast

fin http://www.imm.dtu.dk/~aam