1. Object Recognizing

2. Object Classes

3. Individual Recognition

4. ClassNon-class

6. Features and Classifiers Same features with different classifiers Same classifier with different features

7. Generic Features Simple (wavelets)Complex (Geons)

8. Marr-Nishihara

9. Class-specific Features: Common Building Blocks

10. Optimal Class Components? Large features are too rare Small features are found everywhere Find features that carry the highest amount of information

11. Mutual Information I(C,F) Class:11010100 Feature:10011100 I(F,C) = H(C) – H(C|F)

12. Selecting Fragments

13. Horse-class features Car-class features Pictorial features Learned from examples

14. Fragments with positions On all detected fragments within their regions

15. Star model Detected fragments ‘vote’ for the center location Find location with maximal vote In variations, a popular state-of-the art scheme

16. Recognition Features in the Brain

17. fMRI Functional Magnetic Resonance Imaging

18. תמונות של פעילות המח

19. V1 early processing LO object recognition

20. Class-fragments and Activation Malach et al 2008

21. HoG Descriptor Dallal, N & Triggs, B. Histograms of Oriented Gradients for Human Detection

22. SVM – linear separation in feature space

23. Optimal Separation SVM Perceptron Find a separating plane such that the closest points are as far as possible Rosenblatt, Principles of Neurodynamics 1962. The Nature of Statistical Learning Theory, 1995

24. Optimal Separation SVM Find a separating plane such that the closest points are as far as possible Advantages of SVM: Optimal separation Extensions to the non-separable case: Kernel SVM

25. Separating line:w ∙ x + b = 0 Far line:w ∙ x + b = +1 Their distance:w ∙ ∆x = +1 Separation:|∆x| = 1/|w| Margin:2/|w| 0 +1 The Margin

26. DPM Felzenszwalb Felzenszwalb, McAllester, Ramanan CVPR 2008. A Discriminatively Trained, Multiscale, Deformable Part Model Many implementation details, will describe the main points.

27. HoG descriptor

28. Using patches with HoG descriptors and classification by SVM Person model: HoG

29. Object model using HoG A bicycle and its ‘root filter’ The root filter is a patch of HoG descriptor Image is partitioned into 8x8 pixel cells In each block we compute a histogram of gradient orientations

30. The filter is searched on a pyramid of HoG descriptors, to deal with unknown scale Dealing with scale: multi-scale analysis

31. A part Pi = (Fi, vi, si, ai, bi). Fi is filter for the i-th part, vi is the center for a box of possible positions for part i relative to the root position, si the size of this box ai and bi are two-dimensional vectors specifying coefficients of a quadratic function measuring a score for each possible placement of the i-th part. That is, a i and b i are two numbers each, and the penalty for deviation ∆x, ∆y from the expected location is a 1 ∆ x + a 2 ∆y + b 1 ∆x 2 + b 2 ∆y 2 Adding Parts

32. Bicycle model: root, parts, spatial map Person model

34. The full score of a potential match is: ∑ F i ∙ H i + ∑ a i1 x i + a i2 y i + b i1 x i 2 + b i2 y i 2 F i ∙ H i is the appearance part x i, y i, is the deviation of part p i from its expected location in the model. This is the spatial part. Match Score

35. search with gradient descent over the placement. This includes also the levels in the hierarchy. Start with the root filter, find places of high score for it. For these high-scoring locations, each for the optimal placement of the parts at a level with twice the resolution as the root-filter, using GD. Final decision β∙ψ > θ implies class Recognition Essentially maximize ∑ F i H i + ∑ a i1 x i + a i2 y i + b i1 x i 2 + b i2 y i 2 Over placements (xi yi)

36. The score of a match can be expressed as the dot-product of a vector β of coefficients, with the image: Score = β∙ψ Using the vectors ψ to train an SVM classifier: β∙ψ > 1 for class examples β∙ψ < 1 for class examples Using SVM: Z is the placement

38. Training -- positive examples with bounding boxes around the objects, and negative examples. Learn root filter using SVM Define fixed number of parts, at locations of high energy in the root filter HoG Use these to start the iterative learning

39. β∙ψ > 1 for class examples β∙ψ < 1 for class examples However, ψ depends on the placement z, that is, the values of ∆x i, ∆y i We need to take the best ψ over all placements. In their notation: Classification then uses β∙f > 1 We need to take the best ψ over all placements. In their notation: Classification then uses β∙f > 1

40. ‘Pascal Challenge’ Airplanes Obtaining human-level performance?

41. Deep Learning