1. Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et al. Faces in the Wild

2. 2 Computer Science Is Face Recognition Solved?

3. 3 Computer Science Is Face Recognition Solved? “100% Accuracy in Automatic Face Recognition” [!!!] Science 25 January 2008

4. 4 Computer Science Is Face Recognition Solved? “100% Accuracy in Automatic Face Recognition” [!!!] Science 25 January 2008 A history of overstated results.

5. 5 Computer Science The Truth  Many different face recognition problems Out of context, accuracy is meaningless!  Many problems are REALLY HARD! For some problems state of the art is 70% or worse!  We have a long way to go!

6. 6 Computer Science Face Recognition at UMass  Labeled Faces in the Wild  The Detection-Alignment-Recognition pipeline  Congealing and automatic face alignment  Hyper-features for face recognition  New directions in recognition

7. 7 Computer Science Labeled Faces in the Wild http://vis-www.cs.umass.edu/lfw/

8. 8 Computer Science The Many Faces of Face Recognition Labeled Faces in the Wild

9. 9 Computer Science The Many Faces of Face Recognition Labeled Faces in the Wild

10. 10 Computer Science The Many Faces of Face Recognition Labeled Faces in the Wild

11. 11 Computer Science The Many Faces of Face Recognition Labeled Faces in the Wild

12. 12 Computer Science The Many Faces of Face Recognition Labeled Faces in the Wild

13. 13 Computer Science Labeled Faces in the Wild  13,233 images, with name of each person  5749 people  1680 people with 2 or more images  Designed for the “unseen pair matching problem”. Train on matched or mismatched pairs. Test on never-before-seen pairs.  Distinct from problems with “galleries” or training data for each target image.  Best accuracy: currently about 73%!

14. 14 Computer Science Detection-Alignment-Recognition Pipeline Detection RecognitionAlignment “Same”

15. 15 Computer Science Detection-Alignment-Recognition Pipeline Detection RecognitionAlignment “Same” Parts should work together.

16. 16 Computer Science Labeled Faces in the Wild  All images are output of a standard face detector.  Also provides aligned images.  Consequence: any face recognition algorithm that works well on LFW can easily be turned into a complete system.

17. 17 Computer Science Congealing (CVPR 2000)

18. 18 Computer Science Criterion of Joint Alignment  Minimize sum of pixel stack entropies by transforming each image. A pixel stack

19. 19 Computer Science Congealing Complex Images Window around pixelSIFT vector and clusters SIFT clusters vector representing probability of each cluster, or “mixture” of clusters

21. 21 Computer Science Crash Course on Martian Identification ? Test: Find Bob after one meeting Martian training set = = = Bob

22. 22 Computer Science Training Data “same” “different”

23. 23 Computer Science General Approach to Hyper-feature method  Carefully align objects  Develop a patch-based model of image differences.  Score match/mismatch based on patch differences.

24. 24 Computer Science Three Models 1.Universal patch model: P(patchDistance|same) P(patchDistance|different) 2.Spatially dependent patch model: P(patchDistance |same,x,y) P(patchDistance |different,x,y) 3.Hyper-feature dependent model: 1.P(patchDistance |same,x,y,appearance) 2.P(patchDistance |different,x,y,appearance)

25. 25 Computer Science Universal Patch Model A single P(dist | same) for all patches Different blue patches are evidence against a match!

26. 26 Computer Science Spatial Patch Model P(dist|same,x 1,y 1 ) estimated separately from P(dist|same,x 2,y 2 ) Greatly increases discriminativeness of model.

27. 27 Computer Science Hyper-Feature Patch Model Is the patch from a matching face going to match this patch?

28. 28 Computer Science Hyper-Feature Patch Model Is the patch from a matching face going to match this patch? Probably yes

29. 29 Computer Science Hyper-Feature Patch Model What about this patch?

30. 30 Computer Science Hyper-Feature Patch Model What about this patch? Probably not.

31. 31 Computer Science Ridiculous Errors from the World’s Best Unconstrained Face Recognition System

32. 32 Computer Science Ridiculous Errors from the World’s Best Unconstrained Face Recognition System

33. 33 Computer Science The New Mission: Estimate Higher Level Features

34. 34 Computer Science The New Mission: Estimate Higher Level Features Can we guess pose?

35. 35 Computer Science The New Mission: Estimate Higher Level Features Can we guess gender?

36. 36 Computer Science The New Mission: Estimate Higher Level Features Can we guess degree of balding, beardedness, moustache?

37. 37 Computer Science The New Mission: Estimate Higher Level Features Can we say that none of these individuals are the same person?

38. 38 Computer Science What can we do with a good segmentation?

39. 39 Computer Science CRF Segmentations

40. 40 Computer Science CRF Segmentations

41. 41 Computer Science Who’s This?

42. 42 Computer Science Who’s This?

43. 43 Computer Science Who’s This? from www.coolopticalillusions.com

44. Computer Science Department Thanks