Efficient Region Search for Object Detection Sudheendra Vijayanarasimhan and Kristen Grauman Department of Computer Science, University of Texas at Austin Motivation Main Idea Efficient Region Search (ERS) 1. A rectangle is imprecise Results (code Background: Linear SVM with BoW 2. Extra features in a window can mislead the detector Goal: Identify the best-scoring region--- the subset of spatially contiguous subregions whose features will maximize a classifier’s score. Naïve approach would require exponential time. Our optimal solution leads to significantly more accurate results on this challenging dataset. ERS search times similar to ESS, and orders of magnitude faster than sliding windows. Unlike ESS, ERS permits pixel-level detections of any shape. Detection overlap accuracy on PASCAL 2008 compared to the global connectivity CRF [Nowozin et al. CVPR 2009] Contour strengths Given a test image, we construct a region-graph on an oversegmentation: Maximum-Weight Connected Subgraph (MWCS) Problem Region-graph Prize-collecting Steiner tree (PCST) problem: connected subgraph that maximizes sum of vertex weights minus (positive) edge costs Convert MWCS  PCST: subtract the smallest vertex weight from all vertex and edge weights. Point feature words: SURF within the superpixel Shape feature words: HoG on whole superpixel Branch-and-Cut Solution Branch-and-cut algorithm for PCST [Ljubic et al. ‘06] to obtain best scoring region: Optimal solutions Efficient in practice (100s of nodes) Efficient Region Search with Contours (ERS-C) A variant of ERS to help exclude background regions Training: Learning the Weights Vertex weights are obtained from SVM weights for: Our goal is to determine the arbitrarily shaped region within a novel image that maximizes the score: Region-graph Oversegmentation MWCS instance Point descriptors Shape descriptors Bag of features SVM PCST instance PCST instance Branch-and-cut solution Best-scoring region negative features, - positive features Our Approach 4 Main contribution: We show how to obtain the best-scoring region efficiently with a branch-and-cut solution. Applicable to classifiers whose total score is sum of localized feature scores (e.g., linear SVM, Naïve Bayes NN, boosting). Visual word histogram weights – linear SVM on segmented examples Bag-of-contours histogram weights – structured SVM Datasets Baselines Efficient Subwindow Search (ESS) [Lampert et al. 2008] Global connectivity CRF [Nowozin et al. 2009] Evaluation metrics Pixel-level AP, PASCAL bounding box metric, overlap scores ETHZ Shapes: 5 classes PASCAL 2008 seg: 20 classes PASCAL 2007: cat, dog PASCAL 2008 seg While windows over/underestimate object, ERS allows precise arbitrarily-shaped detections. Pixel-level precision recall curves on PASCAL 2007 (cat, dog) and ETHZ for our approach and ESS ERS more accurate than ESS, even under bounding box metric (19-70% better). Shape features excel on ETHZ; region detection crucial for “non-boxy” objects. Comparison with ESS Comparison with CRF Computation Time An efficient branch-and-cut method for region-based detection Demonstrated its advantages over both window-based detection and a CRF model In future work, we will examine the alternate classifiers accepted by our model. Conclusions Object detection via exhaustive search is too expensive. Branch-and-bound schemes can limit the search (Lampert et al. ’08, Lehmann et al. ’09, Yeh et al. ’09), but existing methods are restricted to rectangular or simple polygonal candidate windows. Problem: Divide image into superpixels and construct region-graph Weight each superpixel vertex by classifier output on its features Branch-and-cut to find best connected subgraph Maximum-weight connected subgraph → Prize- collecting Steiner tree problem As noted by Lampert et al. ‘08, for a linear SVM and bag-of- words, the classifier response for a region R can be written as sum of its N features’ word weights: Num occurrences of j-th word SVM weight for j-th word SVM weight for i-th feature’s word Identify the connected subgraph R* whose summed vertex weights are maximal. Edges set by adjacency, and to impose spatial layout. Class-specific edge weights via bag-of-contour strengths Example Detections PASCAL 2007 ETHZ (point/shape features) - neg features - pos features