“Secret” of Object Detection Zheng Wu (Summer intern in MSRNE) Sep. 3, 2010 Joint work with Ce Liu (MSRNE) William T. Freeman (MIT) Adam Kalai (MSRNE)

Slides:



Advertisements
Similar presentations
Semantic Contours from Inverse Detectors Bharath Hariharan et.al. (ICCV-11)
Advertisements

Combining Detectors for Human Hand Detection Antonio Hernández, Petia Radeva and Sergio Escalera Computer Vision Center, Universitat Autònoma de Barcelona,
Jan-Michael Frahm, Enrique Dunn Spring 2013
EE462 MLCV Lecture 5-6 Object Detection – Boosting Tae-Kyun Kim.
EE462 MLCV Lecture 5-6 Object Detection – Boosting Tae-Kyun Kim.
Rapid Object Detection using a Boosted Cascade of Simple Features Paul Viola, Michael Jones Conference on Computer Vision and Pattern Recognition 2001.
Rapid Object Detection using a Boosted Cascade of Simple Features Paul Viola, Michael Jones Conference on Computer Vision and Pattern Recognition 2001.
Classification using intersection kernel SVMs is efficient Joint work with Subhransu Maji and Alex Berg Jitendra Malik UC Berkeley.
Ivan Laptev IRISA/INRIA, Rennes, France September 07, 2006 Boosted Histograms for Improved Object Detection.
Computer Vision for Human-Computer InteractionResearch Group, Universität Karlsruhe (TH) cv:hci Dr. Edgar Seemann 1 Computer Vision: Histograms of Oriented.
Lecture 31: Modern object recognition
Many slides based on P. FelzenszwalbP. Felzenszwalb General object detection with deformable part-based models.
Global spatial layout: spatial pyramid matching Spatial weighting the features Beyond bags of features: Adding spatial information.
Object-centric spatial pooling for image classification Olga Russakovsky, Yuanqing Lin, Kai Yu, Li Fei-Fei ECCV 2012.
Object Category Detection: Sliding Windows Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 04/10/12.
EE462 MLCV Lecture 5-6 Object Detection – Boosting Tae-Kyun Kim.
Enhancing Exemplar SVMs using Part Level Transfer Regularization 1.
Groups of Adjacent Contour Segments for Object Detection Vittorio Ferrari Loic Fevrier Frederic Jurie Cordelia Schmid.
Viola/Jones: features “Rectangle filters” Differences between sums of pixels in adjacent rectangles { y t (x) = +1 if h t (x) >  t -1 otherwise Unique.
Detecting Pedestrians by Learning Shapelet Features
Fast intersection kernel SVMs for Realtime Object Detection
More sliding window detection: Discriminative part-based models Many slides based on P. FelzenszwalbP. Felzenszwalb.
Student: Yao-Sheng Wang Advisor: Prof. Sheng-Jyh Wang ARTICULATED HUMAN DETECTION 1 Department of Electronics Engineering National Chiao Tung University.
1 Image Recognition - I. Global appearance patterns Slides by K. Grauman, B. Leibe.
Generic Object Detection using Feature Maps Oscar Danielsson Stefan Carlsson
Learning Spatial Context: Using stuff to find things Geremy Heitz Daphne Koller Stanford University October 13, 2008 ECCV 2008.
Object Detection using Histograms of Oriented Gradients
Object Recognition: Conceptual Issues Slides adapted from Fei-Fei Li, Rob Fergus, Antonio Torralba, and K. Grauman.
On the Object Proposal Presented by Yao Lu
What, Where & How Many? Combining Object Detectors and CRFs
Global and Efficient Self-Similarity for Object Classification and Detection CVPR 2010 Thomas Deselaers and Vittorio Ferrari.
Generic object detection with deformable part-based models
Face Detection using the Viola-Jones Method
EADS DS / SDC LTIS Page 1 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Image – 29/03/07 - Oberpfaffenhofen.
Object Recognizing. Recognition -- topics Features Classifiers Example ‘winning’ system.
Window-based models for generic object detection Mei-Chen Yeh 04/24/2012.
Visual Object Recognition
Object Detection with Discriminatively Trained Part Based Models
Lecture 31: Modern recognition CS4670 / 5670: Computer Vision Noah Snavely.
BING: Binarized Normed Gradients for Objectness Estimation at 300fps
Pedestrian Detection and Localization
Deformable Part Models (DPM) Felzenswalb, Girshick, McAllester & Ramanan (2010) Slides drawn from a tutorial By R. Girshick AP 12% 27% 36% 45% 49% 2005.
Project 3 Results.
Object detection, deep learning, and R-CNNs
Histograms of Oriented Gradients for Human Detection(HOG)
Methods for classification and image representation
Human Detection Method Combining HOG and Cumulative Sum based Binary Pattern Jong Gook Ko', Jin Woo Choi', So Hee Park', Jang Hee You', ' Electronics and.
CS 1699: Intro to Computer Vision Detection II: Deformable Part Models Prof. Adriana Kovashka University of Pittsburgh November 12, 2015.
Object Detection Overview Viola-Jones Dalal-Triggs Deformable models Deep learning.
Improved Object Detection
Object Recognition by Integrating Multiple Image Segmentations Caroline Pantofaru, Cordelia Schmid, Martial Hebert ECCV 2008 E.
Object Recognizing. Object Classes Individual Recognition.
Object Recognition by Discriminative Combinations of Line Segments and Ellipses Alex Chia ^˚ Susanto Rahardja ^ Deepu Rajan ˚ Maylor Leung ˚ ^ Institute.
More sliding window detection: Discriminative part-based models
Week 4: 6/6 – 6/10 Jeffrey Loppert. This week.. Coded a Histogram of Oriented Gradients (HOG) Feature Extractor Extracted features from positive and negative.
1 Bilinear Classifiers for Visual Recognition Computational Vision Lab. University of California Irvine To be presented in NIPS 2009 Hamed Pirsiavash Deva.
Recent developments in object detection
Object DetectionI Ali Taalimi 01/08/2013.
Cascade for Fast Detection
Object detection with deformable part-based models
Presented by Minh Hoai Nguyen Date: 28 March 2007
Lit part of blue dress and shadowed part of white dress are the same color
Object detection as supervised classification
R-CNN region By Ilia Iofedov 11/11/2018 BGU, DNN course 2016.
A Tutorial on HOG Human Detection
An HOG-LBP Human Detector with Partial Occlusion Handling
Object DetectionII Ali Taalimi 01/08/2013.
AHED Automatic Human Emotion Detection
Outline Background Motivation Proposed Model Experimental Results
RCNN, Fast-RCNN, Faster-RCNN
Presentation transcript:

“Secret” of Object Detection Zheng Wu (Summer intern in MSRNE) Sep. 3, 2010 Joint work with Ce Liu (MSRNE) William T. Freeman (MIT) Adam Kalai (MSRNE) Jianxiong Xiao (MIT)

Intro Sliding Windows Features Cascade Classifier PASCAL Con 2 motorbike person

Intro Sliding Windows Features Cascade Classifier PASCAL Con 3 Outline Introduction Sliding-Window-based Object Detection Window Generation Feature Extraction Cascade Classifier PASCAL Challenge Conclusion

Intro Sliding Windows Features Cascade Classifier PASCAL Con 4 Related Topic Image Matching Image Classification Object Detection

Intro Sliding Windows Features Cascade Classifier PASCAL Con Object Detection Single PatternMultiple Patterns 5 Viola & Jones Face Detector Dalal & Triggs Pedestrian Detector Felzenszwalb’s Part-based Detector

Intro Sliding Windows Features Cascade Classifier PASCAL Con 6 Outline Introduction Sliding-Window-based Object Detection Window Generation Feature Extraction Cascade Classifier PASCAL Challenge Conclusion

Intro Sliding Windows Features Cascade Classifier PASCAL Con 7 Sliding Windows Search over: Location Scale Aspect Ratio Millions of windows!

Intro Sliding Windows Features Cascade Classifier PASCAL Con 8 Sliding Windows Subsample on grid - have to set “optimal” step size manually Fix Aspect Ratio - assume single pattern detection Fix Scale - assume object’s resolution does not change much between training and test sets. Search with branch-and-bound method - have to use special scoring function

Intro Sliding Windows Features Cascade Classifier PASCAL Con 9 Sliding Windows We propose sliding windows from segmentation Superpixel Segmentation [Levinshtein et al, PAMI09] Region Segmentation [Felzenszwalb & Huttenlocher, IJCV03] < 100,000 sliding windows / image on PASCAL Dataset

Intro Sliding Windows Features Cascade Classifier PASCAL Con 10 Outline Introduction Sliding-Window-based Object Detection Window Generation Feature Extraction Cascade Classifier PASCAL Challenge Conclusion

Intro Sliding Windows Features Cascade Classifier PASCAL Con 11 Generic Feature “Objectness” features [Alexe et al, CVPR10]

Intro Sliding Windows Features Cascade Classifier PASCAL Con 12 Generic Feature Each type of generic feature is weak, but combination is stronger Low dimensional feature (=8) Not suitable for objects with “concave” shape, i.e. table, chair

Intro Sliding Windows Features Cascade Classifier PASCAL Con 13 Generic Feature

Intro Sliding Windows Features Cascade Classifier PASCAL Con 14 Class-specific Feature Histogram of Orientated Gradients

Intro Sliding Windows Features Cascade Classifier PASCAL Con 15 Class-specific Feature Dense grid (>10*10) (secret 1) Overlapping cells Histogram bin size High dimensional feature (>1000) - redundant or overfitting? Normalization No spatial relationship maintained

Intro Sliding Windows Features Cascade Classifier PASCAL Con 16 Outline Introduction Sliding-Window-based Object Detection Window Generation Feature Extraction Cascade Classifier PASCAL Challenge Conclusion

Intro Sliding Windows Features Cascade Classifier PASCAL Con 17 Cascade Classifier Same type of classifier with different features Viola & Jones Face Detector, IJCV01 Different types of classifier with same features Harzallah et al, ICCV09 (INRIA) Vedaldi et al, ICCV09 (Oxford)

Intro Sliding Windows Features Cascade Classifier PASCAL Con 18 Cascade Classifier Training SVM is slow… to train 20,000 examples with 4000 dimensions: >15min for Linear SVM >3 hours for Nonlinear SVM Training SVM requires a lot of memory… design matrix: 20,000*20,000 matrix Training with Imbalance data a few hundreds of positive examples billions of negative examples

Intro Sliding Windows Features Cascade Classifier PASCAL Con 19 Boosted SVM

Intro Sliding Windows Features Cascade Classifier PASCAL Con examples, half for training, half for testing Training error is 0.05 for all boosted classifiers Boosted SVM

Intro Sliding Windows Features Cascade Classifier PASCAL Con 21 Positive Training Set

Intro Sliding Windows Features Cascade Classifier PASCAL Con 22 Negative Training Set Random Samples SVM ver. 1 Training Sample Pool False Positives SVM ver. 2 … Secret 2

Intro Sliding Windows Features Cascade Classifier PASCAL Con 23 Outline Introduction Sliding-Window-based Object Detection Window Generation Feature Extraction Cascade Classifier PASCAL Challenge Conclusion

Intro Sliding Windows Features Cascade Classifier PASCAL Con 24 PASCAL Dataset 2009

Intro Sliding Windows Features Cascade Classifier PASCAL Con 25 PASCAL Dataset 2009

Intro Sliding Windows Features Cascade Classifier PASCAL Con 26 PASCAL Dataset 2009

Intro Sliding Windows Features Cascade Classifier PASCAL Con 27 PASCAL Dataset 2009

Intro Sliding Windows Features Cascade Classifier PASCAL Con 28 PASCAL Dataset 2009

Intro Sliding Windows Features Cascade Classifier PASCAL Con 29 PASCAL Dataset 2009

Intro Sliding Windows Features Cascade Classifier PASCAL Con 30 PASCAL Dataset 2009

Intro Sliding Windows Features Cascade Classifier PASCAL Con 31 PASCAL Dataset 2009

Intro Sliding Windows Features Cascade Classifier PASCAL Con 32 PASCAL 2009 (trainval + test)

Intro Sliding Windows Features Cascade Classifier PASCAL Con 33 PASCAL 2009 (train+val)

Intro Sliding Windows Features Cascade Classifier PASCAL Con 34 PASCAL 2009 (train+val,1/image)

Intro Sliding Windows Features Cascade Classifier PASCAL Con 35 PASCAL 2009 (train+val,5/image)

Intro Sliding Windows Features Cascade Classifier PASCAL Con 36 PASCAL 2009 (train+val,10/image)

Intro Sliding Windows Features Cascade Classifier PASCAL Con 37 True Positives - aeroplane

Intro Sliding Windows Features Cascade Classifier PASCAL Con 38 False Positives - aeroplane

Intro Sliding Windows Features Cascade Classifier PASCAL Con 39 True Positives - bicycle

Intro Sliding Windows Features Cascade Classifier PASCAL Con 40 False Positives - bicycle

Intro Sliding Windows Features Cascade Classifier PASCAL Con 41 True Positives - horse

Intro Sliding Windows Features Cascade Classifier PASCAL Con 42 False Positives - horse

Intro Sliding Windows Features Cascade Classifier PASCAL Con 43 Conclusion Proposing sliding windows without fixing scale or aspect ratio is possible. Simple feature (saliency, contrast, etc) is only useful for certain object classes. Histogram-based feature is not sufficient to represent real world object, no matter what learning algorithm is used. Boosting is helpful to speed up SVM-training and reduce the memory usage. Digging out “hard” negative examples. Throwing away “hard” positive examples.

Intro Sliding Windows Features Cascade Classifier PASCAL Con 44 Future Work It is time to go beyond the histogram-of-X feature - not every pixel within bounding box is informative - the appearance of object’s part is more robust Evolve the classifier - even PASCAL dataset is too small - the right decision boundary is still far away… - Active learning? Online learning?

Thank You ! 45