1. Face Recognition Under Varying Illumination Erald VUÇINI Vienna University of Technology Muhittin GÖKMEN Istanbul Technical University Eduard GRÖLLER Vienna University of Technology

2. Erald Vuçini - Vienna University of Technology 2 Face Recognition System Image Capture Face Identification Face Detection Database Feature Projection

3. Erald Vuçini - Vienna University of Technology 3 Face Recognition Approaches Principal Component Analysis (PCA) Linear Discriminant Analysis (LDA) Local Feature Analysis Active Appearance Model Hidden Markov Model Support Vector Machine …

4. Erald Vuçini - Vienna University of Technology 4 Face Recognition – Problems The variations of the same face due to illumination viewing direction are almost always larger than image variations due to changes in the face identity

5. Erald Vuçini - Vienna University of Technology 5 Handling Variable Illumination Extract illumination invariant features Transform variable illumination to canonical representation Model 2D illumination variations Utilize 3D face models whose shapes and albedos are obtained in advance

6. Erald Vuçini - Vienna University of Technology 6 I. Dimensionality Reduction - LDA better than PCA regarding illumination II. Image Synthesis - Solve the Small Sample Size (SSS) problem III. Reconstruction – Restore frontal illumination Outline of proposed approach

7. Erald Vuçini - Vienna University of Technology 7 I. Dimensionality Reduction - LDA better than PCA regarding illumination II. Image Synthesis - Solve the Small Sample Size (SSS) problem III. Reconstruction – Restore frontal illumination Dimensionality Reduction

8. Erald Vuçini - Vienna University of Technology 8 Principal Component Analysis (PCA) One of the most commonly used methods in Face Recognition Maximizes the scattering of all projected samples PCA x1x1 x2x2 x1x1 x2x2 z1z1 z2z2

9. Erald Vuçini - Vienna University of Technology 9 PCA under Varying Illumination PCA fails with variant illumination The scatter being maximized is due to Between-class scatter Within-class scatter Discard 3 most significant principal components to reduce lighting variation

10. Erald Vuçini - Vienna University of Technology 10 LDA Interpretation LDA is a class specific method LDA

11. Erald Vuçini - Vienna University of Technology 11 LDA Problems LDA maximizes the ratio of Between-class scatter and Within-class Scatter Within-class Scatter singularity problem Fisher LDA (FLDA) removes Null Space FLDA handles best the variation in lighting, having lower error rate than PCA

12. Erald Vuçini - Vienna University of Technology 12 I. Dimensionality Reduction - LDA better than PCA regarding illumination II. Image Synthesis - Solve the Small Sample Size (SSS) problem III. Reconstruction – Restore frontal illumination Image Synthesis

13. Erald Vuçini - Vienna University of Technology 13 Image Synthesis - Motivation Face Recognition Systems Performance related with training database LDA require many samples per class In many systems only one image per person is provided Quotient Image makes possible the synthesis of the image space of a given input image

14. Erald Vuçini - Vienna University of Technology 14 Lambertian Objects - Faces The image space lives in a 3D linear subspace Three images are sufficient for generating the image space of the object Albedo Surface Normal Light Source Direction

15. Erald Vuçini - Vienna University of Technology 15 Quotient Image (Definitions) Ideal class of faces Same shape Different albedos Synthesis Problem: Given 3N images of N faces of the same class, illuminated under 3 lighting conditions Synthesize image space of new input

16. Erald Vuçini - Vienna University of Technology 16 Quotient Image (Definitions) Given objects y and a we define quotient image Q by the ratio of their albedos Q is illumination invariant Image space of y can be generated with Quotient Q 3 images of a Generalization: Use bootstrap of 3N images

17. Erald Vuçini - Vienna University of Technology 17 Quotient Image - Examples Quotient

18. Erald Vuçini - Vienna University of Technology 18 Quotient Image – Image Space Synthesis

19. Erald Vuçini - Vienna University of Technology 19 10 person Bootstrap 5 person Bootstrap 1 person Bootstrap

20. Erald Vuçini - Vienna University of Technology 20 I. Dimensionality Reduction - LDA better than PCA regarding illumination II. Image Synthesis - Solve the Small Sample Size (SSS) problem III. Reconstruction – Restore frontal illumination Reconstruction

21. Erald Vuçini - Vienna University of Technology 21 YaleB Testing Database Yaleb Database 450 images of 10 persons Divided in 4 subsets Subset1 up to 10˚ Subset2 up to 25˚ Subset3 up to 45˚ Subset4 up to 75˚ Normalized

22. Erald Vuçini - Vienna University of Technology 22 Histogram Equalization Histogram equalization(HE) done as preprocessing increases the recognition rate Adaptive HE(AHE) is used as a preprocessing step in the iterative face recognition approach Results with the YaleB Database (PCA used) No PreprocessingHEAHE Recognition Rate(%)43.47481.5

23. Erald Vuçini - Vienna University of Technology 23 Illumination Restoration Approach A face image with arbitrary illumination is restored to having frontal illumination. It has the following advantages: No need to estimate face surface normals No need to estimate light source directions and albedos No need to perform image warping Face images will be visually natural looking

24. Erald Vuçini - Vienna University of Technology 24 Algorithm Outline Compute mean face image and eigenspace Compute initial restored images Create iteration by replacing B ro with blurred H io Continue iteration until stopping criteria satisfied Input Image Blurred Reference Image Blurred Input Image Restored Image

25. Erald Vuçini - Vienna University of Technology 25 Iteration Steps

26. Erald Vuçini - Vienna University of Technology 26 Experimental Results (Subset 3) Restoration

27. Erald Vuçini - Vienna University of Technology 27 Experimental Results (Subset 4) Restoration

28. Erald Vuçini - Vienna University of Technology 28 Results with YaleB Database Subset1Subset2Subset3Subset4Overall HE+PCA(%) 10097.566.444.274 HE+New(%) 100 92.891.795.6 Subset1Subset2Subset3Subset4Overall AHE+PCA(%) 100 83.5750.081.6 AHE+New(%) 100 95.098.7

29. Erald Vuçini - Vienna University of Technology 29 Thank you for the attention! Proposed Method Dimension Reduction Image Synthesis Reconstruction