1. 1 Numerical geometry of non-rigid shapes Expression-invariant face recognition Expression-invariant face recognition Lecture 8 © Alexander & Michael Bronstein tosca.cs.technion.ac.il/book Numerical geometry of non-rigid shapes Stanford University, Winter 2009

2. 2 Numerical geometry of non-rigid shapes Expression-invariant face recognition Biometrics in the age of Patriarchs “The voice is the voice of Jacob, but the hands are the hands of Esau”

3. 3 Numerical geometry of non-rigid shapes Expression-invariant face recognition Biometrics in fairy tales

4. 4 Numerical geometry of non-rigid shapes Expression-invariant face recognition Biometrics in opera

5. 5 Numerical geometry of non-rigid shapes Expression-invariant face recognition First face recognition Galton’s mechanical face recognition machine I contrieved an apparatus […] which I call a mechanical selector. Its object is to find which set out of a standard collection of many sets of measures, resembles any one of given set within specified degree of unlikeness. “ ” Nature, 1888 Sir Francis Galton (1822-1911)

6. 6 Numerical geometry of non-rigid shapes Expression-invariant face recognition Face recognition today = ? Is this the same person?

7. 7 Numerical geometry of non-rigid shapes Expression-invariant face recognition Some terminology Gallery Instances of faces of a person with known identity Probe A face unseen before with unknown identity Impostor False rejection Probe deemed dissimilar to gallery of same identity False acceptance Probe of different identity deemed similar to gallery

8. 8 Numerical geometry of non-rigid shapes Expression-invariant face recognition The coin that betrayed Louis XVI Gallery Probe

9. 9 Numerical geometry of non-rigid shapes Expression-invariant face recognition How face recognition works Gallery Probe FACE DISTANCE > threshold? REJECTACCEPT NoYes

10. 10 Numerical geometry of non-rigid shapes Expression-invariant face recognition Recognition accuracy False acceptance rate (FRR) False rejection rate (FAR) High threshold (loose) Low threshold (conservative) Equal Error Eate (EER) FAR = FRR

11. 11 Numerical geometry of non-rigid shapes Expression-invariant face recognition Face distance Discriminative Large between faces of different persons Invariant Small between faces of same person in different conditions

12. 12 Numerical geometry of non-rigid shapes Expression-invariant face recognition + GEOMETRIC (3D) PHOTOMETRIC (2D) What is a face? =

13. 13 Numerical geometry of non-rigid shapes Expression-invariant face recognition What is more important: 2D or 3D? + =

14. 14 Numerical geometry of non-rigid shapes Expression-invariant face recognition What is more important: 2D or 3D? + =

15. 15 Numerical geometry of non-rigid shapes Expression-invariant face recognition Acco Turkish bath museum, Israel Live statue

16. 16 Numerical geometry of non-rigid shapes Expression-invariant face recognition Conclusion 1 3D data encodes valuable information about person’s identity Less sensitive to external factors (light, pose, makeup) More difficult to forge

17. 17 Numerical geometry of non-rigid shapes Expression-invariant face recognition The curse of expressions

18. 18 Numerical geometry of non-rigid shapes Expression-invariant face recognition Is geometry sensitive to expressions? x x’ y y’ Euclidean distances

19. 19 Numerical geometry of non-rigid shapes Expression-invariant face recognition Is geometry sensitive to expressions? x x’ y y’ Geodesic distances

20. 20 Numerical geometry of non-rigid shapes Expression-invariant face recognition Conclusion 2 Distance distortion distribution Extrinsic geometry sensitive to expressions Intrinsic geometry insensitive to expressions

21. 21 Numerical geometry of non-rigid shapes Expression-invariant face recognition Isometric model of expressions Facial expressions are approximate isometries of the facial surface Identity = intrinsic geometry Expression = extrinsic geometry A. M. Bronstein et al., IJCV, 2005

22. 22 Numerical geometry of non-rigid shapes Expression-invariant face recognition

23. 23 Numerical geometry of non-rigid shapes Expression-invariant face recognition How to canonize a person? 3D scan Smooting Canonization Cropping

24. 24 Numerical geometry of non-rigid shapes Expression-invariant face recognition Canonical forms of faces

25. 25 Numerical geometry of non-rigid shapes Expression-invariant face recognition SCANNED FACE CANONICAL FORM DISTANCES

26. 26 Numerical geometry of non-rigid shapes Expression-invariant face recognition Telling identical twins apart Extrinsic similarity Intrinsic similarity MichaelAlex

27. 27 Numerical geometry of non-rigid shapes Expression-invariant face recognition Telling identical twins apart MichaelAlex

28. 28 Numerical geometry of non-rigid shapes Expression-invariant face recognition

29. 29 Numerical geometry of non-rigid shapes Expression-invariant face recognition

30. 30 Numerical geometry of non-rigid shapes Expression-invariant face recognition Comparing photometric properties Facial surface with texture 3D canonical form 2D canonical form Two-dimensional canonical form can be used as a common parametrization of the facial textures

31. 31 Numerical geometry of non-rigid shapes Expression-invariant face recognition Spherical embedding R = 80 mmR = 100 mmR = 150 mm

32. 32 Numerical geometry of non-rigid shapes Expression-invariant face recognition Partial similarity