Evaluation of Image Pre-processing Techniques for Eigenface Based Face Recognition Thomas Heseltine york.ac.uk/~tomh

2 Introduction Growing interest in biometric authentication National ID cards, Airport security, Surveillance, Site access. Face recognition offers several advantages over other biometrics: Can be used without subjects knowledge. Human readable media. No association with crime, as with fingerprints. Data required is easily obtained and readily available. Approaches include: Neural networks, Feature analysis, Graph matching, Information theory.

3 Principal Component Analysis A 256 by 256 pixel image of a face, represented as a vector occupies a single point in 65,536-dimensional image space. Images of faces occupy a relatively small region of this large image space. A training set of face images is used to create a covariance matrix, from which the eigenvectors (and eigenvalues) are computed using standard methods. The eigenvectors define an image subspace, known as face space, which maximises the spread of face images within this space. Selecting the eigenvectors with the highest eigenvalues (the principle components) reduces the dimensionality of face space. Two face images are compared by measuring the (Euclidean) distance between their positions in face space. Used to create an image subspace (face space) with reduced dimensionality, while maintaining a high level of discrimination between faces.

4 Eigenfaces The eigenvectors are sorted in order of descending eigenvalues and the 30 greatest eigenvectors are chosen to represent face space. This reduces the dimensionality of the image space to 30, yet maintains a high level of variance between face images throughout the image subspace. Any face image can then be represented as a vector of coefficients, corresponding to the ‘contribution’ of each eigenface. Each eigenvector can be displayed as an image and due to the likeness to faces, Turk and Pentland refer to these vectors as eigenfaces.

5 Limitations Variations in lighting conditions. Different lighting conditions for enrolment and query. Bright light causing image saturation. Differences in pose – Head orientation. 2D feature distances appear to distort. Image quality. CCTV, Web-cams etc. are often not good enough. Expression (change in feature location and shape). Partial covering (Hats, scarves, glasses). The system effectiveness is highly dependant on the image capture conditions. Meaning face recognition systems are usually not as accurate as other biometrics.

6 Possible Solution There are many image representations and filtering techniques that can reduce the effect of lighting conditions and improve image quality e.g. Colour normalisation Histogram equalisation. Edge detection. Noise reduction. Could such methods as these be used to improve eigenface based face recognition?

7 Test Database 960 bitmap images of 120 individuals (60 male, 60 female) extracted from the AR Face Database provided by Martinez and Benavente [10]. All images are translated, rotated and scaled, such that the centres of the eyes are aligned. The database is separated into two disjoint sets: The training set, (60 different people, natural lighting conditions, neutral expression). The test set, (900 images - 15 images of 60 people, captured under a variety of conditions). After a series of initial investigations, we select a resolution of 25 pixels between the eyes, with a width and height of 75 and 112 pixels respectively comparisons are made to calculate false rejection rates and 7080 comparisons to calculate false acceptance rates.

8 Test Procedure

9 Output FAR The percentage of distance measures below the threshold, when images of different people are being compared. FRR The percentage of distance measures above the threshold when images of the same person are being compared. By varying the threshold we obtain error rate pairs describing a curve. The EER is used to compare pre-processing techniques. However, it should not be used as a guideline to the system performance in a real world situation.

10 Image Filters Convolution Methods Colour Representation Methods Statistical Methods Method Combinations

11 Results

12 Tweaking A few final adjustments are made to determine the optimum image region. Reducing the ERR by another 1.6%. The first five eigenfaces generated from the pre-processed training set. OriginalProcessedAverage

13 Conclusion The eigenface-based method of face recognition can be significantly improved by introducing a simple image pre-processing step. An EER of 22% percent can be achieved (a reduction of 12%) on a set of extremely difficult images (20% partially obscured, 40% taken under extreme lighting conditions). The remaining 22% error shows that some factors are not compensated for by the single image pre-processing techniques. The effect of some filters may be mutually exclusive and could therefore be used in parallel for improved results (further experimentation required). Other variations of PCA based face recognition (Pentland et al’s modular eigenface system [8], Belhumeur et al’s comparison to Fisher faces [9]) have also shown significant improvements, without image pre-processing. It is likely that such systems could also benefit from the filtering techniques described, resulting in greatly improved face recognition systems.

14 References [1] M. Turk, A. Pentland. Eigenfaces for Recognition. Vision and Modelling Group, Massachusetts Institute of Technology. [2] M. Turk, A. Pentland. Face Recognition Using Eigenfaces. Vision and Modelling Group, Massachusetts Institute of Technology. [3] G.D. Finlayson, B. Schiele, J. L. Crowley. Comprehensive Colour Image Normalisation. The Colour and Imaging Institute, The University of Derby. [4] G. Finlayson, G. Schaefer. Hue that is Invariant to Brightness and Gamma. School of Information Systems, University of East Anglia. [5] Y. Adini, Y. Moses, S. Ullman. Face Recognition: the Problem of Compensating for Changes in Illumination Direction. Department of Applied Mathematics and Computer Science, The Weizmann Institute of Science. [6] W. Zhao, R. Chellappa. 3D Model Enhanced Face Recognition. Sarnoff Corporation, Princeton. [7] R. Cutler. Face Recognition Using Infrared Images and Eigenfaces. Department of Computer Science, University of Maryland. [8] A. Pentland, B. Moghaddom, T. Starner. View-Based and Modular Eigenfaces for Face Recognition. The Vision and Modelling Group, Massachusetts Institute of Technology. [9] P. Belhumeur, J. Hespanha, D. Kriegman. Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection. Department of Electrical Engineering, Yale University. [10] A.M. Martinez, R. Benavente. The AR Face Database. CVC Technical Report #24, June 1998.