1. Venus

2. Classification

3. Faces – Different

4. Faces -- Same

6. Lighting affects appearance

8. Three-point alignment

9. Object Alignment Given three model points P 1, P 2, P 3, and three image points p 1, p 2, p 3, there is a unique transformation (rotation, translation, scale) that aligns the model with the image.  (SR + d)P i = p i

10. Alignment -- comments The projection is orthographic projection (combined with scaling). The 3 points are required to be non-collinear. The transformation is determined up to a reflection of the points about the image plane and translation in depth.

11. Proof of the 3-point Alignment: The 3 3-D points are P1, P2, P3. We can assume that they are initially in the image plane. In the 2-D image we get q1, q2, q3. The transformation P1 > q1, P2 > q2, P3 > q3, defines a unique linear transformation of the plane, L(x). We can easily recover this transformation. L is a 2*2 matrix. We fix the origin at P1 = q1. We have two more points that define 4 linear equations for the elements of L. We now choose two orthogonal vectors E1 and E2 in the original plane of P1, P2, P3. We can compute E1’ = L(E1), E2’ = L(E2). We seek a scaling S, Rotation R, so that the projection of SR(E1) = E1’ and SR(E2) = E2’. Let SR(E1) (without the projection) be V1 and SR(E2) = V2. V1 is E1’ plus a depth component, that is, V1 = E1’ + c1z, where z is a unit vector in the z direction. Similarly, V2 = E2’ + c2z. We wish to recover c1 and c2. This will give the transformation between the points (show that it is unique, and it will be possible to recover the transformation). We know that the scalar product (V1 V2) = 0. (E1’ + c1z) (E1’ + c1z) = 0 Therefore c1c2 = -(E’1 E’2). The magnitude -(E’1 E’2) is measurable in the image, call it C12, therefore c1c2 = c12. Also |V1| = |V2|. Therefore (E1’ + c1z) (E1’ + c1z) = (E1’ + c1z) (E1’ + c1z). This implies c1 2 - c2 2 = k12, where k12 is a measurable quantity in the image (it is |E’1 2 | - |E’2 2 |. The two equation of c1 c2 are: c1c2 = c12 c1 2 - c2 2 = k12 and they have a unique solution. One way of seeing this is by setting a complex number Z = c1 + ic2. Then Z 2 = k12 + ic12. Therefore, Z 2 is measurable. We take the square root and get Z, therefore c1, c2. There are exactly two roots giving the two mirror reflection solutions.

12. Car Recognition

13. Car Models

14. Alignment: Cars

15. Alignment: Unmatch

16. Face Alignment

17. Linear Combination of Views

18. O is a set of object points. I 1, I 2, I 3, are three images of O from different views. N is a novel view of O. Then O is the linear combination of I 1, I 2, I 3.

19. Car Recognition

20. VW – SAAB

21. LC – Car Images

22. Linear Combination: Faces

23. Classification

24. Structural descriptions

25. RBC

28. Structural Description G2 G4 G3 G1 G4 Above Right-of Left-of Touch

30. Fragment-based Representation

31. Mutual Information Mutual information Entropy Binary variable -H(C) = P(C=1)Log(P(C=1) + P(C=0)Log(P(C=0)

32. Mutual information H(C) when F=1H(C) when F=0 I(C;F) = H(C) – H(C/F) F=1 F=0 H(C)

33. Selecting Fragments

34. Fragments Selection For a set of training images: Generate candidate fragments –Measure p(F/C), p(F/NC) Compute mutual information Select optimal fragment After k fragments: Maximizing the minimal addition in mutual information with respect to each of the first k fragments

35. Optimal Face Fragments

36. Face Fragments by Type 1d. 1e. 1-st. Merit0.200.18 0.170.160.110.100.09 Weight6.55.56.455.453.522.9 2.86 2-nd 3-rd 4-th

37. Low-resolution Car Fragments Front – Middle - Back

38. Intermediate Complexity

39. Fragment ‘Weight’ Likelihood ratio: Weight of F:

40. Combining fragments w1w1 wkwk w2w2 D1D1 D2D2 DkDk

42. Non-optimal Fragments FragmentssizedetectionF/A Optimal11%950 Small3%9730 Large33%390 Same total area covered (8*object), on regular grid

43. Training & Test Images Frontal faces without distinctive features (K:496,W:385) Minimize background by cropping Training images for extraction: 32 for each class Training images for evaluation: 100 for each class Test images: 253 for Western and 364 for Korean

44. Training – Fragment Extraction

45. Western Fragment Score0.920.820.770.760.750.740.720.680.670.65 Weight3.422.401.992.231.902.116.584.144.126.47 Korean Fragment Score0.920.820.770.760.750.740.720.680.670.65 Weight3.422.401.992.231.902.116.584.144.126.47 Extracted Fragments

46. Classifying novel images Westerner Korean Unknown kFkF wFwF Detect Fragments Compare Summed Weights Decision

47. Effect of Number of Fragments 7 fragments: 95%, 80 fragments: 100% Inherent redundancy of the features Slight violation of independence assumption

48. Comparison with Humans Algorithm outperformed humans for low resolution images

49. Class examples

52. Distinctive Features