Multiple Object Class Detection with a Generative Model K. Mikolajczyk, B. Leibe and B. Schiele Carolina Galleguillos

Goal  Simultaneous recognition and localization of multiple object classes using a generative model.  Recognition Codebook (features are shared among several object classes).  Detection Probabilistic model for various objects in the same image.

Introduction Single Object class detection is a mature problem Multiple Object class detection performance is far from single object. Their approach: Fast and dense sampling of scale invariant features. Effective object representation. Efficient and reliable training and recognition.

Introduction Other approaches:  Based on feature detectors: Local features & several detectors.  Based on appearance clusters: Visual vocab., codebook, keywords.  Represent object classes: Star shape, graphical model, etc.

Features - Appearance  We can compute them efficiently:  Scale space pyramid with a Gaussian kernel.  For each level Canny edge detection with Laplacian automatic scale (position, scale and dominant orientation).  For each edge point we identify a region of interest (in the gradient orientation). This region is described by SIFT descriptors (128 dimensional vector).  Use of PCA for dimensionality reduction (to 40 dimensions).

Features - Geometry  Rotation invariance: Convert position of features in polar coordinates. d: distance to object center. φ: angle. θ: dominant gradient orientation.

Hierarchical Codebook  Tree structure Hierarchical tree of clusters: Appearance clusters (formed by similar features at first level) Each cluster has several geometric distributions that correspond to object classes (info about geom. relations between object centers and local appearance). Node is a hyperball

Building Tree Efficiently  Apply K-means to divide space (top-down).  Use reciprocal nearest neighbor in each k-means partition with a similarity threshold.  Apply agglomerative clustering (bottom up).  Euclidean distance to group clusters.  Clustering trace is used to construct the tree.

Building Tree Efficiently

Tree - Advantages  Appearance clusters are shared within one image and among different classes (and object parts).  Compact representation.  Represent individual objects or all object classes.  Efficient search.

Recognition  Bayesian rule approach F: features. A: appearance clusters. G: geometric distribution. Each feature likelihood is modeled by a mixture of distributions from appearance clusters which match to a query feature. Decision:

Recognition  Problem: Similar objects in the model have probabilities comparables in shared clusters.  Condition: each feature can contribute only to one hypothesis.  Average confusion factor between pairs of objects.  If approaches to 1, we remove from both hypothesis all info that come from those clusters.

Learning  Joint probability distributions are separated in two terms  To estimate de model: Extract features F from labeled training examples. Build appearance clusters & match the features back to the cluster centers (threshold β). Each feature that matches to contributes to the prob. estimates for the appearance and to its geometric distrib. at the position.

Fast Matching  Match features to cluster centers using a ball tree.  Represent query and model as tree structures.  Match two trees computing Euclidean distance between centroids of top nodes. If distance is smaller than the sum of their radii, then the first node is compared with all the children of the intersecting node. Same precision to exhaustive search and 200 times faster.

Experimental results 5 object classes: pedestrian, cars, motorbikes, bicycles and RPG shooter.

Experimental results Motorbike test data Recall is higher and the number of appearance clusters grow sub-linearly with increasing number of object classes

Conclusions  Approach capable of detecting multiple object classes simultaneously in images using a single codebook.  Performance comparable with state of the art discriminative approaches.  Efficient method for building object class representation and recognition.