Active Appearance Models for Face Detection Rocío Cabrera, Guillaume Lemaître, Mojdeh Rastgoo

Presentation Outline Introduction Database Used Models Implementation The IMM Face Database Models Statistical Shape Models Statistical Models of Appearance Active Appearance Models Implementation Training Stage Testing Stage Conclusions 3D Digitization - Active Appearance Models for Face Detection 1/14/2011

Introduction Non-trivial Applications in Machine Vision “Understand” the presented images Recover image structure Know what this structure means Real applications include complex/variable structures Faces Detection Model-based Methods Prior knowledge of the problem Expected Shapes of Structures Their Spatial Relationship Greylevel Appearance Restrict Automated Search to Plausible Interpretations 3D Digitization - Active Appearance Models for Face Detection 1/14/2011

Introduction Model-based Methods Generative Models Deformable Models Are able to generate realistic images of target objects Deformable Models Are able to deal with object variability Two main desired characteristics General – capable of generating plausible examples of the class they represent Specific – capable of generating only legal/valid example Model-based Methods Top-down Strategy Prior Model of Expected Class Find Best Match in Image “Measure” if the target is actually present 3D Digitization - Active Appearance Models for Face Detection 1/14/2011

The IMM Face Database An Annotated Dataset of 240 Face Images 40 different human subjects (7 females vs. 33 males) All without glasses or accessories Manual Annotation of 58 landmarks Six Different Positions Full frontal face, neutral/happy expression, diffuse light Face rotated (30° right/left), neutral expression, diffuse light Full frontal face, neutral expression, spot light added at the person's left side. Full frontal face, arbitrary expression, diffuse light. Eyebrows Eyes Nose Mouth Jaw 3D Digitization - Active Appearance Models for Face Detection 1/14/2011

Statistical Shape Models Shape alignment Procrustes Analysis : Aligning the images onto the same reference axes Translation, Rotation and Scaling Transformations Procrustes Analysis minimizes the distance between a reference shape and each shape in the dataset Modelling Shape Variation Computation of the mean shape Computation of the scatter (covariance) matrix Sorting the eigenvectors and keeping the first k eigenvectors , based on the largest eigenvalues Eigen decomposition of the shapes where , Value of k is based on 3D Digitization - Active Appearance Models for Face Detection 1/14/2011

Statistical Shape Models Mean Shape and Largest Deformation 3D Digitization - Active Appearance Models for Face Detection 1/14/2011

Statistical Shape Models 3D Digitization - Active Appearance Models for Face Detection 1/14/2011

Statistical Models of Appearance Texture mapping is required to generate the photo realistic synthetic images Combination of a shape variation model with texture variation model Configuration of landmarks Texture is the pattern of intensities or color across the image patch In order to build the statistical appearance model , we require training set of label images where key landmarks are marked on each examples 1 . Statistical shape model which was already done in the previous step  mean shape 2. Wrapping each training example to the mean shape to obtain shape free patch , Using PCA on free patches (Eigen faces basically)  shape model Texture model 3D Digitization - Active Appearance Models for Face Detection 1/14/2011

Statistical Models of Appearance Training set of label Images Computation of statistical shape models -PCA Statistical Shape models – Mean shapes Computation of Free-Patch Images – Image wrapping Appearance models PCA Applying PCA on Free-Patch Images Statistical texture Models 3D Digitization - Active Appearance Models for Face Detection 1/14/2011

Statistical Models of Appearance – Image wrapping Piece Wise Affine Performing the Delaunay triangulation on each shape model Affine Transformation which maps the corner of the triangles to their new positions in new Image In general consider the problem such as we have image I with set of control points xi , we want to map this points to new points x’i with the mapping function f , Using f function we can project each pixel in image I to image I’. 3D Digitization - Active Appearance Models for Face Detection 1/14/2011

Statistical Models of Appearance – Texture modeling Training set of shape-free normalized image patches Performing PCA Model of texture: Set of orthogonal modes of variations Set of gray level parameters Mean normalized gray level I 3D Digitization - Active Appearance Models for Face Detection 1/14/2011

Statistical texture Models Eigen-faces decomposition 3D Digitization - Active Appearance Models for Face Detection 1/14/2011

Statistical Models of Appearance – Combined Image Shape parameter vector and texture parameter vector might have correlation Performing PCA Appearance Model: Diagonal matrix of weight for each shape parameter The elements of shape vector have the unit of distance , and the element of the texture will have intensity units so they can not be compared directly that s why we needs the weight factor . Controlling both shape and texture Eigenvectors Shape and texture will be a function of c 3D Digitization - Active Appearance Models for Face Detection 1/14/2011

Statistical Appearance Models Combination of texture model and shape model Difference Image Texture model 3D Digitization - Active Appearance Models for Face Detection 1/14/2011

Implementation Consists of two main stages Training Stage Multi-scale implementation to obtain an AAM model N scales implementation Testing Stage Searches for the object (face) in a test image 3D Digitization - Active Appearance Models for Face Detection 1/14/2011

Implementation – Training Stage Load Training Data for SCALE = 1:N Make Shape Model Align shapes with Procrustes Analysis Obtain main directions of variations with PCA Keep the 98% most significant eigenvectors Grey-level appearance Model Transform face image into mean texture image Normalize the greyscale, to compensate for illumination Perform PCA Keep the 99% most significant eigenvectors Combined Shape-Appearance Model Addition of the shape and appearance models Keep only 99% of all eigenvectors Search Model Find the object location in a test set Training done by translation and intensity difference computation (keep position with smallest difference) Transform the image to a coarser scale end Make Shape Model Grey-level Appearance Model Combined Shape-Appearance Model Search Model Transform Image to a Coarser Scale 3D Digitization - Active Appearance Models for Face Detection 1/14/2011

Implementation – Testing Stage Manual Initialization For SCALE = 1:N (start in coarser scale) Get Model for Current Scale Image Scaling Search Iterations Sample Image Intensities Compute difference between model and real image intensities If Errorold < Errorcurrent Go to previous location Else Update Errorold End Go to next finer scale Show Detection Results Manual Initialization Search Iterations Show Detection Results 3D Digitization - Active Appearance Models for Face Detection 1/14/2011

Results Lower Scale Higher Scale 3D Digitization - Active Appearance Models for Face Detection 1/14/2011

Results Highest Scale Texture map found at this scale 3D Digitization - Active Appearance Models for Face Detection 1/14/2011

Problems Faced during Implementation Memory issues during the training Problem of the reconstruction of the appearance Not real-time application 3D Digitization - Active Appearance Models for Face Detection 1/14/2011

Conclusion Face detection and face tracking are non-trivial applications in machine vision Model-based methods Prior knowledge of the problem Expected Shapes of Structures Their Spatial Relationship Grey-level Appearance Active Appearance Models Are built from a set of training examples Should account for class variabilty They heavily rely on Principal Component/ Eigenvalue Analysis Through a search algorithm we seek to interpret a new target image with the optimal model parameters which best describe the target image The extension to Face Detection was not yet achieved but it is expected to work for the deliverable due date The use of AAM seem like a promising method to perform face detection and/or recognition 3D Digitization - Active Appearance Models for Face Detection 1/14/2011

References [1] Ginneken B. et al. "Active Shape Model Segmentation with Optimal Features", IEEE Transactions on Medical Imaging 2002. [2] T.F. Cootes, G.J Edwards, and C,J. Taylor "Active Appearance Models", Proc. European Conference on Computer Vision 1998 [3] T.F. Cootes, G.J Edwards, and C,J. Taylor "Active Appearance Models", IEEE Transactions on Pattern Analysis and Machine Intelligence 2001 [4] Vazeos Ioannis. Active Appearance Models (AAM). MASTER THESIS REPORT. Master of Science in Information Networking. Athens Information Technology. 2004 -2005 [5] T.F. Gootes and C.J. Taylor Statistical Models of Appearance for Computer Vision. Imaging Science and Biomedical Engineering, University of Manchester. Technical Report. 2004. [6] F. Dornaika and J. Ahlberg . Efficient Active Appearance Model for Real-Time Head and Facial Feature Tracking Proceedings of the IEEE International Workshop on Analysis and Modeling of Faces and Gestures (AMFG’03). 2003 [7] Akshay Asthana, Jason Saragih, Michael Wagner and Roland Goecke. Evaluating AAM Fitting Methods for Facial Expression Recognition. 2009 IEEE [8] Mingcai Zhou, Yangsheng Wang, Xiaoyan Wang and Xuetao Feng. A Two-Stage Approach for AAM Fitting. Eighth International Conference on Intelligent Systems Design and Applications. 2008 IEEE [9] Fangqi Tang and Benzai Deng Facial Expression Recognition using AAM and Local Facial Features. Third International Conference on Natural Computation (ICNC 2007). 2007. 3D Digitization - Active Appearance Models for Face Detection 1/14/2011