1. Active Appearance Models theory, extensions & cases
Mikkel B. Stegmann
Master thesis presentation
IMM September 12th 2000

2. Aim & Method
High precision segmentation of non-rigid objects in digital images
Active Appearance Models:
A deformable template model
A priori knowledge learned through statistical analysis of a training set

3. Presentation outline
Method
Case
Contributions
Results
Discussion

4. Shape analysis Shape is represented by a linear spline of landmarks
X = ( x1, x2, … , xn, y1, y2, … , yn)T
Alignment w.r.t. position, scale, orientation
Principal
component
analysis
Compact shape representation

5. Shape analysis - cont’d
Principal component analysis on N aligned shapes
Mean shape
Covariance matrix
Eigen problem
Shape deformation

6. Texture analysis
Texture – the intensities across the object – is sampled inside the shape using a suitable warp function
Warp function: A piece-wise affine warp using the Delaunay triangulation
g = ( x1, … , xm)T
Principal
component
analysis
Compact texture
representation

7. Combined model
Shape and texture is combined into a compact model representation capable of deforming in a similar manner to what is observed in the training set
Shape & texture deformation
Model instance construction
Obtaining PCA scores

8. Metacarpals — a case study
24 x-ray images of the human hand with metacarpal 2, 3, 4 annotated using 50 landmarks each

9. Modes of variation Shape deformation 1st shape mode
Texture deformation
1st texture mode
Combined deformation
1st combined mode

10. Search

11. Contributions Handling of homogeneous shapes
Handling of heterogeneous shapes
Search-based initialisation
Fine-tuning of the model fit
Applying robust statistics to the optimisation
Unification of Finite Element Models and AAMs

12. Homogeneous shapes
Increases texture specificity by adding a suitable neighborhood region

13. Heterogeneous shapes
Models the outer border only to avoid large-scale texture noise
Normal AAM
Border AAM
Registration

14. Search-based initialisation
Performs a sparse search using the standard AAM search
Parameters: c, position, scale, rotation (constrained)
Survival of the fittest:
Generates a set of candidates, which are further investigated until one candidate remains (evolved guessing)
Image
sampling point (candidate)
Actual dy (pixels)
Predicted dy (pixels)

15. Fine-tuning of the model fit
Using general purpose optimisation techniques:
Gradient based methods
Steepest descent
Conjugate gradient
Quasi-Newton (BFGS)
Non-gradient based methods
Pattern search
Random-sampling based methods
Simulated annealing (SA)
Evaluation criterion
The similarity measure between model and image:
Parameters
c, position, scale, rotation

16. Robust optimisation
Sensitivity to texture outliers reduced using robust similarity measures in the optimisation
Quadratic
Lorentzian
Quadratic
Lorentzian

17. Unification of Finite Element Models and AAMs
Wanted: flexibility inside shapes independently of landmark movement
Accomplished using artificial interior points deformed by a finite element model

18. Results Radiographs — the human hand Cardiac MRI — left ventricle
1st combined mode
Radiographs — the human hand
Landmark accuracy
Basic AAM pixels
Neighbourhood + SA 0.82 pixels
23 images / 150 landmarks / ~ pixels
Cardiac MRI — left ventricle
Basic AAM pixels
SA pixels
13 images / 66 landmarks / ~2.200 pixels
Perspective Images — pork carcasses
Basic AAM pixels
Border AAM pixels
13 images / 83 landmarks / ~3.500 pixels

19. Discussion “Hidden” benefits Requirements
Analyses of Variance (group/longitudinal studies)
Classification/interpretation using the PCA parameters
Automatic registration
Requirements
Landmarks (point correspondence)
Distinct features
The AAM approach extends to 3D and multivariate imaging

20. Conclusion
AAM has been explored, documented and extended as a fully automated and data-driven approach towards image segmentation
Thorough evaluation has shown that AAM performs well on different segmentation problems and different image modalities (x-ray, MRI etc.)
Properties
General
Specific
Robust
Non-parametric (no knobs)
Self-contained
Fast

21. fin