1. Active Appearance Models for Face Detection
Rocío Cabrera, Guillaume Lemaître, Mojdeh Rastgoo

2. Presentation Outline Introduction Database Used Models Implementation
The IMM Face Database
Models
Statistical Shape Models
Statistical Models of Appearance
Active Appearance Models
Implementation
Training Stage
Testing Stage
Conclusions
3D Digitization - Active Appearance Models for Face Detection
1/14/2011

3. Introduction Non-trivial Applications in Machine Vision
“Understand” the presented images
Recover image structure
Know what this structure means
Real applications include complex/variable structures
Faces Detection
Model-based Methods
Prior knowledge of the problem
Expected Shapes of Structures
Their Spatial Relationship
Greylevel Appearance
Restrict Automated Search to Plausible Interpretations
3D Digitization - Active Appearance Models for Face Detection
1/14/2011

4. Introduction Model-based Methods Generative Models Deformable Models
Are able to generate realistic images of target objects
Deformable Models
Are able to deal with object variability
Two main desired characteristics
General – capable of generating plausible examples of the class they represent
Specific – capable of generating only legal/valid example
Model-based Methods
Top-down Strategy
Prior Model of Expected Class
Find Best Match in Image
“Measure” if the target is actually present
3D Digitization - Active Appearance Models for Face Detection
1/14/2011

5. The IMM Face Database An Annotated Dataset of 240 Face Images
40 different human subjects (7 females vs. 33 males)
All without glasses or accessories
Manual Annotation of 58 landmarks
Six Different Positions
Full frontal face, neutral/happy expression, diffuse light
Face rotated (30° right/left), neutral expression, diffuse light
Full frontal face, neutral expression, spot light added at the person's left side.
Full frontal face, arbitrary expression, diffuse light.
Eyebrows
Eyes
Nose
Mouth
Jaw
3D Digitization - Active Appearance Models for Face Detection
1/14/2011

6. Statistical Shape Models
Shape alignment
Procrustes Analysis : Aligning the images onto the same reference axes
Translation, Rotation and Scaling Transformations
Procrustes Analysis minimizes the distance between a reference shape and each shape in the dataset
Modelling Shape Variation
Computation of the mean shape
Computation of the scatter (covariance) matrix
Sorting the eigenvectors and keeping the first k eigenvectors , based on the largest eigenvalues
Eigen decomposition of the shapes where ,
Value of k is based on
3D Digitization - Active Appearance Models for Face Detection
1/14/2011

7. Statistical Shape Models
Mean Shape and Largest Deformation
3D Digitization - Active Appearance Models for Face Detection
1/14/2011

8. Statistical Shape Models
3D Digitization - Active Appearance Models for Face Detection
1/14/2011

9. Statistical Models of Appearance
Texture mapping is required to generate the photo realistic synthetic images
Combination of a shape variation model with texture variation model
Configuration of landmarks
Texture is the pattern of intensities or color across the image patch
In order to build the statistical appearance model , we require training set of label images where key landmarks are marked on each examples
1 . Statistical shape model which was already done in the previous step  mean shape
2. Wrapping each training example to the mean shape to obtain shape free patch , Using PCA on free patches (Eigen faces basically) 
shape model
Texture model
3D Digitization - Active Appearance Models for Face Detection
1/14/2011

10. Statistical Models of Appearance
Training set of label Images
Computation of statistical shape models -PCA
Statistical Shape models – Mean shapes
Computation of Free-Patch Images – Image wrapping
Appearance models
PCA
Applying PCA on Free-Patch Images
Statistical texture Models
3D Digitization - Active Appearance Models for Face Detection
1/14/2011

11. Statistical Models of Appearance – Image wrapping
Piece Wise Affine
Performing the Delaunay triangulation on each shape model
Affine Transformation which maps the corner of the triangles to their new positions in new Image
In general consider the problem such as we have image I with set of control points xi , we want to map this points to new points x’i with the mapping function f , Using f function we can project each pixel in image I to image I’.
3D Digitization - Active Appearance Models for Face Detection
1/14/2011

12. Statistical Models of Appearance – Texture modeling
Training set of shape-free normalized image patches
Performing PCA
Model of texture:
Set of orthogonal modes of variations
Set of gray level parameters
Mean normalized gray level I
3D Digitization - Active Appearance Models for Face Detection
1/14/2011

13. Statistical texture Models
Eigen-faces decomposition
3D Digitization - Active Appearance Models for Face Detection
1/14/2011

14. Statistical Models of Appearance – Combined Image
Shape parameter vector and texture parameter vector might have correlation
Performing PCA
Appearance Model:
Diagonal matrix of weight for each shape parameter
The elements of shape vector have the unit of distance , and the element of the texture will have intensity units so they can not be compared directly that s why we needs the weight factor .
Controlling both shape and texture
Eigenvectors
Shape and texture will be a function of c
3D Digitization - Active Appearance Models for Face Detection
1/14/2011

15. Statistical Appearance Models
Combination of texture model and shape model
Difference Image
Texture model
3D Digitization - Active Appearance Models for Face Detection
1/14/2011

16. Implementation Consists of two main stages Training Stage
Multi-scale implementation to obtain an AAM model
N scales implementation
Testing Stage
Searches for the object (face) in a test image
3D Digitization - Active Appearance Models for Face Detection
1/14/2011

17. Implementation – Training Stage
Load Training Data
for SCALE = 1:N
Make Shape Model
Align shapes with Procrustes Analysis
Obtain main directions of variations with PCA
Keep the 98% most significant eigenvectors
Grey-level appearance Model
Transform face image into mean texture image
Normalize the greyscale, to compensate for illumination
Perform PCA
Keep the 99% most significant eigenvectors
Combined Shape-Appearance Model
Addition of the shape and appearance models
Keep only 99% of all eigenvectors
Search Model
Find the object location in a test set
Training done by translation and intensity difference computation (keep position with smallest difference)
Transform the image to a coarser scale
end
Make Shape Model
Grey-level Appearance Model
Combined Shape-Appearance Model
Search Model
Transform Image to a Coarser Scale
3D Digitization - Active Appearance Models for Face Detection
1/14/2011

18. Implementation – Testing Stage
Manual Initialization
For SCALE = 1:N (start in coarser scale)
Get Model for Current Scale
Image Scaling
Search Iterations
Sample Image Intensities
Compute difference between model and real image intensities
If Errorold < Errorcurrent
Go to previous location
Else
Update Errorold
End
Go to next finer scale
Show Detection Results
Manual Initialization
Search Iterations
Show Detection Results
3D Digitization - Active Appearance Models for Face Detection
1/14/2011

19. Results Lower Scale Higher Scale
3D Digitization - Active Appearance Models for Face Detection
1/14/2011

20. Results Highest Scale Texture map found at this scale
3D Digitization - Active Appearance Models for Face Detection
1/14/2011

21. Problems Faced during Implementation
Memory issues during the training
Problem of the reconstruction of the appearance
Not real-time application
3D Digitization - Active Appearance Models for Face Detection
1/14/2011

22. Conclusion
Face detection and face tracking are non-trivial applications in machine vision
Model-based methods
Prior knowledge of the problem
Expected Shapes of Structures
Their Spatial Relationship
Grey-level Appearance
Active Appearance Models
Are built from a set of training examples
Should account for class variabilty
They heavily rely on Principal Component/ Eigenvalue Analysis
Through a search algorithm we seek to interpret a new target image with the optimal model parameters which best describe the target image
The extension to Face Detection was not yet achieved but it is expected to work for the deliverable due date
The use of AAM seem like a promising method to perform face detection and/or recognition
3D Digitization - Active Appearance Models for Face Detection
1/14/2011

23. References
[1] Ginneken B. et al. "Active Shape Model Segmentation with Optimal Features", IEEE Transactions on Medical Imaging [2] T.F. Cootes, G.J Edwards, and C,J. Taylor "Active Appearance Models", Proc. European Conference on Computer Vision 1998 [3] T.F. Cootes, G.J Edwards, and C,J. Taylor "Active Appearance Models", IEEE Transactions on Pattern Analysis and Machine Intelligence 2001 [4] Vazeos Ioannis. Active Appearance Models (AAM). MASTER THESIS REPORT. Master of Science in Information Networking. Athens Information Technology [5] T.F. Gootes and C.J. Taylor Statistical Models of Appearance for Computer Vision. Imaging Science and Biomedical Engineering, University of Manchester. Technical Report [6] F. Dornaika and J. Ahlberg . Efficient Active Appearance Model for Real-Time Head and Facial Feature Tracking Proceedings of the IEEE International Workshop on Analysis and Modeling of Faces and Gestures (AMFG’03) [7] Akshay Asthana, Jason Saragih, Michael Wagner and Roland Goecke. Evaluating AAM Fitting Methods for Facial Expression Recognition IEEE [8] Mingcai Zhou, Yangsheng Wang, Xiaoyan Wang and Xuetao Feng. A Two-Stage Approach for AAM Fitting. Eighth International Conference on Intelligent Systems Design and Applications IEEE [9] Fangqi Tang and Benzai Deng Facial Expression Recognition using AAM and Local Facial Features. Third International Conference on Natural Computation (ICNC 2007)
3D Digitization - Active Appearance Models for Face Detection
1/14/2011