1. Object Recognizing

2. Object Classes

3. Individual Recognition

4. Object parts Headlight Window Door knob Back wheel Mirror Front wheel Headlight Window Bumper

5. ClassNon-class

7. Unsupervised Training Data

8. Features and Classifiers Same features with different classifiers Same classifier with different features

9. Generic Features Simple (wavelets)Complex (Geons)

10. Class-specific Features: Common Building Blocks

11. Mutual information H(C) when F=1H(C) when F=0 I(C;F) = H(C) – H(C/F) F=1 F=0 H(C)

12. Mutual Information I(C,F) Class:11010100 Feature:10011100 I(F,C) = H(C) – H(C|F)

13. Optimal classification features Theoretically: maximizing delivered information minimizes classification error In practice: informative object components can be identified in training images

14. Selecting Fragments

15. Adding a New Fragment (max-min selection) ? MIΔ MI = MI [ Δ ; class ] - MI [ ; class ] Select: Max i Min k ΔMI (Fi, Fk) (Min. over existing fragments, Max. over the entire pool)

16. Horse-class features Car-class features Pictorial features Learned from examples

17. Star model Detected fragments ‘vote’ for the center location Find location with maximal vote In variations, a popular state-of-the art scheme

18. Fragment-based Classification Fergus, Perona, Zisserman 2003 Agarwal, Roth 2002 Ullman, Sali 1999

19. Variability of Airplanes Detected

20. Recognition Features in the Brain

21. Class-fragments and Activation Malach et al 2008

22. EEG

23. ERP MI 1 — MI 2 — MI 3 — MI 4 — MI 5 — Harel, Ullman,Epshtein, Bentin Vis Res 2007

24. Bag of words

25. Bag of visual words A large collection of image patches –

26. Generate a dictionary using K-means clustering

32. Each class has its words historgram – – – Limited or no Geometry Simple and popular, no longer state-of-the art.

33. Classifiers

34. SVM – linear separation in feature space

35. Optimal Separation SVM Find a separating plane such that the closest points are as far as possible Advantages of SVM: Optimal separation Extensions to the non-separable case: Kernel SVM

36. Separating line:w ∙ x + b = 0 Far line:w ∙ x + b = +1 Their distance:w ∙ ∆x = +1 Separation:|∆x| = 1/|w| Margin:2/|w| 0 +1 The Margin

37. Max Margin Classification (Equivalently, usually used How to solve such constraint optimization? The examples are vectors x i The labels y i are +1 for class, -1 for non-class

38. Using Lagrange multipliers: Using Lagrange multipliers: Minimize L P = With α i > 0 the Lagrange multipliers

39. Minimizing the Lagrangian Minimize L p : Set all derivatives to 0: Also for the derivative w.r.t. α i Dual formulation: Maximize the Lagrangian w.r.t. the α i and the above two conditions.

40. Solved in ‘dual’ formulation Maximize w.r.t α i : With the conditions: Put into L p W will drop out of the expression

41. Dual formulation Mathematically equivalent formulation: Can maximize the Lagrangian with respect to the α i After manipulations – concise matrix form:

42. SVM: in simple matrix form We first find the α. From this we can find:w, b, and the support vectors. The matrix H is a simple ‘data matrix’: H ij = y i y j Final classification: w∙x + b ∑α i y i + b Because w = ∑α i y i x i Only with support vectors are used

43. DPM Felzenszwalb Felzenszwalb, McAllester, Ramanan CVPR 2008. A Discriminatively Trained, Multiscale, Deformable Part Model Many implementation details, will describe the main points.

44. HoG descriptor

45. HoG Descriptor Dallal, N & Triggs, B. Histograms of Oriented Gradients for Human Detection

46. Using patches with HoG descriptors and classification by SVM Person model: HoG

47. Object model using HoG A bicycle and its ‘root filter’ The root filter is a patch of HoG descriptor Image is partitioned into 8x8 pixel cells In each block we compute a histogram of gradient orientations

48. The filter is searched on a pyramid of HoG descriptors, to deal with unknown scale Dealing with scale: multi-scale analysis

49. A part Pi = (Fi, vi, si, ai, bi). Fi is filter for the i-th part, vi is the center for a box of possible positions for part i relative to the root position, si the size of this box ai and bi are two-dimensional vectors specifying coefficients of a quadratic function measuring a score for each possible placement of the i-th part. That is, a i and b i are two numbers each, and the penalty for deviation ∆x, ∆y from the expected location is a 1 ∆ x + a 2 ∆y + b 1 ∆x 2 + b 2 ∆y 2 Adding Parts

50. Bicycle model: root, parts, spatial map Person model

52. The full score of a potential match is: ∑ F i ∙ H i + ∑ a i1 x i + a i2 y i + b i1 x i 2 + b i2 y i 2 F i ∙ H i is the appearance part x i, y i, is the deviation of part p i from its expected location in the model. This is the spatial part. Match Score

53. search with gradient descent over the placement. This includes also the levels in the hierarchy. Start with the root filter, find places of high score for it. For these high-scoring locations, each for the optimal placement of the parts at a level with twice the resolution as the root-filter, using GD. Final decision β∙ψ > θ implies class Recognition Essentially maximize ∑ Fi Hi + ∑ ai1 xi + ai2 y + bi1x2 + bi2y2 Over placements (xi yi)

55. ‘Pascal Challenge’ Airplanes Obtaining human-level performance?

56. All images contain at least 1 bike

57. Bike Recognition