1. Recognition – PCA and Templates

2. Recognition Suppose you want to find a face in an imageSuppose you want to find a face in an image One possibility: look for something that looks sort of like a face (oval, dark band near top, dark band near bottom)One possibility: look for something that looks sort of like a face (oval, dark band near top, dark band near bottom) Another possibility: look for pieces of faces (eyes, mouth, etc.) in a specific arrangementAnother possibility: look for pieces of faces (eyes, mouth, etc.) in a specific arrangement

3. Recognition Suppose you want to recognize a particular faceSuppose you want to recognize a particular face How does this face differ from average faceHow does this face differ from average face

4. Templates Model of a “generic” or “average” faceModel of a “generic” or “average” face – Learn templates from example data For each location in image, look for template at that locationFor each location in image, look for template at that location – Optionally also search over scale, orientation

5. Templates In the simplest case, based on intensityIn the simplest case, based on intensity – Template is average of all faces in training set – Comparison based on e.g. SSD More complex templatesMore complex templates – Outputs of feature detectors – Color histograms – Often combine position and frequency information (wavelets)

6. Face Detection Results Sample Images Wavelet Histogram Template Detection of frontal / profile faces

7. More Face Detection Results Schneiderman and Kanade

8. How to Recognize Specific People? Consider variation from average faceConsider variation from average face Not all variations equally importantNot all variations equally important – Variation in a single pixel relatively unimportant If image is high-dimensional vector, want to find directions in this space along which variation is highIf image is high-dimensional vector, want to find directions in this space along which variation is high

9. Principal Components Analaysis Principal Components Analysis (PCA): approximating a high-dimensional data set with a lower-dimensional subspacePrincipal Components Analysis (PCA): approximating a high-dimensional data set with a lower-dimensional subspace Original axes * * * * * * * *** * * *** * * * * * * * * * Data points First principal component Second principal component

10. Principal Components Computing PCA:Computing PCA: – Subtract out mean (“whitening”) – Find eigenvalues of covariance matrix – Equivalently, compute SVD of data matrix

11. PCA on Faces: “Eigenfaces” Average face First principal component Other components For all except average, “gray” = 0, “white” > 0, “black” < 0

12. Using PCA for Recognition Store each person as coefficients of projection onto first few principal componentsStore each person as coefficients of projection onto first few principal components Compute projections of target image, compare to databaseCompute projections of target image, compare to database

13. Recognition Using Relations Between Templates Often easier to recognize a small featureOften easier to recognize a small feature – e.g., lips easier to recognize than faces – For articulated objects (e.g. people), template for whole class usually complicated So, identify small pieces and look for spatial arrangementsSo, identify small pieces and look for spatial arrangements – Many false positives from identifying pieces

14. Graph Matching Head LegLeg Arm Arm Body Head Head Body Body Arm ArmLeg Leg Leg Leg Model Feature detection results

15. Graph Matching Head LegLeg Arm Arm Body Constraints Next to, right of Next to, below

16. Graph Matching Head LegLeg Arm Arm Body Head Head Body Body Arm ArmLeg Leg Leg Leg Combinatorial search

17. Graph Matching Head LegLeg Arm Arm Body Head Head Body Body Arm ArmLeg Leg Leg Leg Combinatorial search OK

18. Graph Matching Head LegLeg Arm Arm Body Head Head Body Body Arm ArmLeg Leg Leg Leg Combinatorial search Already assigned

19. Graph Matching Head LegLeg Arm Arm Body Head Head Body Body Arm ArmLeg Leg Leg Leg Combinatorial search Violates constraint

20. Graph Matching Large search spaceLarge search space – Heuristics for pruning Missing featuresMissing features – Look for maximal consistent assignment Noise, spurious featuresNoise, spurious features Incomplete constraintsIncomplete constraints – Verification step at end