Face Recognition Summary –Single pose –Multiple pose –Principal components analysis –Model-based recognition –Neural Networks

Single Pose Standard head-and-shoulders view with uniform background Easy to find face within image

Aligning Images Alignment –Faces in the training set must be aligned with each other to remove the effects of translation, scale, rotation etc. –It is easy to find the position of the eyes and mouth and then shift and resize images so that are aligned with each other

Nearest Neighbour Once the images have been aligned you can simply search for the member of the training set which is nearest to the test image. There are a number of measures of distance including Euclidean distance, and the cross- correlation

Principal Components PCA reduces the number of dimensions and so the memory requirement is much reduced. The search time is also reduced

Two ways to apply PCA (1) We could apply PCA to the whole training set. Then each face is represented by a point in the PC space We could then apply nearest neighbour to these points

Two ways to apply PCA (2) Alternatively we could apply PCA to the set of faces belonging to each person in the training set Each class (person) is then reprented by a different ellipsoid and Mahalanobis distance can be used to classify a new unknown face You need a lot of images of each person to do this

Problems with PCA The same person may sometimes appear differently due to –Beards, moustaches –Glasses, –Makeup These have to be represented by different ellipsoids

(2) (3) (4) (5) (6) (7) (8) (9) (10)

Problems with PCA Facial expressions –Differing facial expressions Opening and closing the mouth Raised eyebrows Widening the eyes Smiling, frowing etc, These mean that the class is no longer ellipsoidal and must be represented by a manifold

Facial Expressions There are six types of facial expression We could use PCA on the eyes and mouth – so we could have eigeneyes and eigenmouths Anger Fear Disgust Happy Sad Surprise

Multiple Poses Heads must now be aligned in 3D world space Classes now form trajectories in feature space It becomes difficult to recognise faces because the variation due to pose is greater than the variation between people

Model-based Recognition We can fit a model directly to the face image Model consists of a mesh which is matched to facial features such as the eyes, nose, mouth and edges of the face. We use PCA to describe the parameters of the model rather than the pixels.

Model-based Recognition The model copes better with multiple poses and changes in facial expression.