Object Recognizing

Object Classes

Individual Recognition

ClassNon-class

Features and Classifiers Same features with different classifiers Same classifier with different features

Generic Features Simple (wavelets)Complex (Geons)

Marr-Nishihara

Class-specific Features: Common Building Blocks

Optimal Class Components? Large features are too rare Small features are found everywhere Find features that carry the highest amount of information

Mutual Information I(C,F) Class: Feature: I(F,C) = H(C) – H(C|F)

Selecting Fragments

Horse-class features Car-class features Pictorial features Learned from examples

Fragments with positions On all detected fragments within their regions

Star model Detected fragments ‘vote’ for the center location Find location with maximal vote In variations, a popular state-of-the art scheme

Recognition Features in the Brain

fMRI Functional Magnetic Resonance Imaging

תמונות של פעילות המח

V1 early processing LO object recognition

Class-fragments and Activation Malach et al 2008

HoG Descriptor Dallal, N & Triggs, B. Histograms of Oriented Gradients for Human Detection

SVM – linear separation in feature space

Optimal Separation SVM Perceptron Find a separating plane such that the closest points are as far as possible Rosenblatt, Principles of Neurodynamics The Nature of Statistical Learning Theory, 1995

Optimal Separation SVM Find a separating plane such that the closest points are as far as possible Advantages of SVM: Optimal separation Extensions to the non-separable case: Kernel SVM

Separating line:w ∙ x + b = 0 Far line:w ∙ x + b = +1 Their distance:w ∙ ∆x = +1 Separation:|∆x| = 1/|w| Margin:2/|w| 0 +1 The Margin

DPM Felzenszwalb Felzenszwalb, McAllester, Ramanan CVPR A Discriminatively Trained, Multiscale, Deformable Part Model Many implementation details, will describe the main points.

HoG descriptor

Using patches with HoG descriptors and classification by SVM Person model: HoG

Object model using HoG A bicycle and its ‘root filter’ The root filter is a patch of HoG descriptor Image is partitioned into 8x8 pixel cells In each block we compute a histogram of gradient orientations

The filter is searched on a pyramid of HoG descriptors, to deal with unknown scale Dealing with scale: multi-scale analysis

A part Pi = (Fi, vi, si, ai, bi). Fi is filter for the i-th part, vi is the center for a box of possible positions for part i relative to the root position, si the size of this box ai and bi are two-dimensional vectors specifying coefficients of a quadratic function measuring a score for each possible placement of the i-th part. That is, a i and b i are two numbers each, and the penalty for deviation ∆x, ∆y from the expected location is a 1 ∆ x + a 2 ∆y + b 1 ∆x 2 + b 2 ∆y 2 Adding Parts

Bicycle model: root, parts, spatial map Person model

The full score of a potential match is: ∑ F i ∙ H i + ∑ a i1 x i + a i2 y i + b i1 x i 2 + b i2 y i 2 F i ∙ H i is the appearance part x i, y i, is the deviation of part p i from its expected location in the model. This is the spatial part. Match Score

search with gradient descent over the placement. This includes also the levels in the hierarchy. Start with the root filter, find places of high score for it. For these high-scoring locations, each for the optimal placement of the parts at a level with twice the resolution as the root-filter, using GD. Final decision β∙ψ > θ implies class Recognition Essentially maximize ∑ F i H i + ∑ a i1 x i + a i2 y i + b i1 x i 2 + b i2 y i 2 Over placements (xi yi)

The score of a match can be expressed as the dot-product of a vector β of coefficients, with the image: Score = β∙ψ Using the vectors ψ to train an SVM classifier: β∙ψ > 1 for class examples β∙ψ < 1 for class examples Using SVM: Z is the placement

Training -- positive examples with bounding boxes around the objects, and negative examples. Learn root filter using SVM Define fixed number of parts, at locations of high energy in the root filter HoG Use these to start the iterative learning

β∙ψ > 1 for class examples β∙ψ < 1 for class examples However, ψ depends on the placement z, that is, the values of ∆x i, ∆y i We need to take the best ψ over all placements. In their notation: Classification then uses β∙f > 1 We need to take the best ψ over all placements. In their notation: Classification then uses β∙f > 1

‘Pascal Challenge’ Airplanes Obtaining human-level performance?

Deep Learning