Detection Evolution with Multi-Order Contextual Co-Occurrence Guang Chen (Missouri) Yuanyuan Ding (Epson) Jing Xiao (Epson) Tony Han (Missouri)

Object Detection Sliding Window Based Approach Context Helps Classifiers and features are typically inside the window. Context Helps Context outside the sliding window can be used to achieve better performances.

Context in Computer Vision High Level Context Semantic Context Geometric Context Low Level Context Pixel Context Shape Context Murphy et al, 2003 Hoiem et al, 2006 Avidan, 2006 Shotton et al, 2006 Rabinovich et al, 2007 Oliva & Torralba, 2007 Heitz & Koller, 2008 Desai et al, 2009 Divvala et al, 2009 Li, Socher & Fei-Fei, 2009 Marszalek et al, 2009 Bao & Savarese, 2010 Yao & Fei-Fei, 2010 Tu & Bai, 2010 Li, Parikh & Chen, 2011 Wolf & Bileschi, 2006 Belongie et al, 2000 [Rabinovich et al, 2007] [Yao & Fei-Fei, 2010] [Hoiem et al, 2006]

Classification Context for Segmentation Spatialboost and Auto-context Integrate classifier responses from nearby individual pixels for pixel level segmentation or labeling Auto-context [Tu & Bai, 2010] Spatial boost [Avidan 2006]

Classification Context for Object Detection Contextual Boost [Ding & Xiao, 2012] Directly uses the detector responses Adaboost Classification Based on Image Context Image Context + Adaboost Image Context Multi-scale HOG-LDP for Each Scan Window Classification Responses at Scale & Spatial Neighborhood Based on Augmented Context Contextual Boost

Co-Occurrence Context Can we further exploit co-occurrence information given only detectors for a single object type?

Co-Occurrence Context Co-Occurrence from Detector Response Map.

Our Contribution An Effective and Efficient Multi-Order Co-Occurrence Context Representation Using a Single Object Detector.

Our Contribution An Effective and Efficient Multi-Order Co-Occurrence Context Representation Using a Single Object Detector. Multi-Order Contextual Co-Occurrence (MOCO) 0th order: Classification Context 1st order: Randomized Binary Comparison High order: Co-Occurrence Descriptor

Constructing MOCO

0th Order Context Directly Using Classifier Responses Classifier response map (window width=100pixels) Classifier response map (window width=25pixels) Classifier response map (window width=50pixels)

0th Order Context Define Scale and Space Neighborhood P Spatial (x, y) Scale (l) P y x l

1st Order Context Comparison of Response Values P

1st Order Context Randomized Arrangement

High Order Context 1. Closeness Vector 2. Histogram

High Order Context 3. High Order Representation Tensor Product of Normalized Histogram

Detection Evolution Bootstrap training samples using detector responses from the previous iteration. Add MOCO context from previous iteration as additional features.

Baseline Detector Any Object Detection Algorithm Can be Used as Baseline Detector.

Baseline Detector Any Object Detection Algorithm Can be Used as Baseline Detector. Deformable-Parts-Model [Felzenszwalb et al, 2010] Inner Context: Parts Models Encodes Relationship between Parts. Outer Context: MOCO deals with Co-Occurrence among Scanning Windows

Experiments Datasets Deformable-Parts-Model PASCAL VOC 2007, 20 Object Categories Caltech Pedestrian Deformable-Parts-Model Default setting ( 3 components, each with 1 root and 8 part filters)

Experiment – 1st Order 1st Order & Context Neighbor Size

Experiment – 1st Order Pairwise Comparison: Arrangements

Experiment – High Order High Order Context Dimension

Experiment – Combinations Iterations

Comparison on Caltech Dataset

Comparison on PASCAL’07 Mean AP on 20 Categories

Conclusion An Efficient Context Representation Future Work Only Relying on Detectors for a Single Object Type Combining Deformable Parts Model to Model both inner and Outer Context around Detection Window Future Work Exploit Context With Detectors of Multiple Object Types?

Thanks for your attention! Questions? Thanks for your attention!