Real-Time Detection, Alignment and Recognition of Human Faces Rogerio Schmidt Feris Changbo Hu Matthew Turk Pattern Recognition Project June 12, 2003

Overview Introduction Introduction FERET Dataset FERET Dataset Face Detection Face Detection Face Alignment Face Alignment Face Recognition Face Recognition Conclusions Conclusions

Introduction DetectionAlignmentRecognition

Introduction  Why this is a difficult problem? Facial Expressions, Illumination Changes, Pose, etc.  Assumption: Frontal view faces  Objectives: Develop a fully automatic system, suitable for real-time applications. Develop a fully automatic system, suitable for real-time applications. Evaluate it on a large dataset. Evaluate it on a large dataset.

FERET DataSet  1196 different individuals  Probe Sets: FB: Different facial expressions FB: Different facial expressions FC: Different illumination conditions FC: Different illumination conditions DUP1: Different days DUP1: Different days DUP2: Images taken at least 1 year after DUP2: Images taken at least 1 year after

Face Detection  State-of-the-art: Learning-based approaches  Neural Nets [Rowley et al, PAMI 98]  SVMs [Heisele and Poggio, CVPR 01]  Boosting [Viola and Jones, ICCV 01]  Want to know more? Detecting Faces in Images: a Survey [M. Yang, PAMI 02]

Face Detection [Viola and Jones, 2001]  Simple features, which can be computed very fast.  A variant of Adaboost is used both to select the features and to train the classifier.  Classifiers are combined in a “cascade” which allows background regions of the image to be quickly discarded.

Face Detection Time: 100ms (PIV 1.6Ghz)

Face Alignment  State-of-the-art: Deformable Models  Bunch-Graph approach [Wiskott, PAMI 98]  Active Shape Models [Cootes, CVIU 95]  Active Appearance Models [Cootes, PAMI 01]

Face Alignment Active Appearance Model (AAM) Active Appearance Model (AAM) Statistical Shape Model (PCA) Statistical Texture Model (PCA) AAM Search AAM Search

Face Alignment  Problem: Partial Occlusion  Active Wavelet Networks (AWN) (submitted to BMVC’03) Main idea: Replace AAM texture model by a wavelet network

Face Alignment Similar performance to AAM in images under normal conditions. More robust against partial occlusions.

Face Alignment Using 9 wavelets, the system requires only 3 ms per iteration (PIV 1.6Ghz). In general, at most 10 iterations are sufficient for good convergence.

Face Recognition  State-of-the-art: Subspace Techniques  PCA, FDA, Kernel PCA, Kernel FDA, ICA, etc.  Want to know more? Face Recognition: a Literature Survey [W. Zhao, 2000]

Face Recognition   Preprocessing Line up eyes, histogram equalization, masking  Subspace Training (PCA)  Classification (Nearest-neighbor)

Face Recognition

Conclusions  An efficient, fully automatic system for face recognition was presented and evaluated.  Future Work:  Alignment: multiresolution search  View-based face recognition  Explicit illumination model  Live demo

Face Recognition