Video Surveillance E Senior/Feris/Tian 1 Emerging Topics in Video Surveillance Rogerio Feris IBM TJ Watson Research Center
Video Surveillance E Senior/Feris/Tian 2 Outline Video Surveillance in Crowded Scenarios Online Learning – Self-adaptation in Surveillance Other Recent Topics
Video Surveillance E Senior/Feris/Tian 3 Simple Scenarios Few Objects – Background Subtraction + Tracking + High-level Event/Alert Detection Current systems work well
Video Surveillance E Senior/Feris/Tian 4 Crowded Scenarios Many objects, occlusions, shadows, etc. Object Segmentation, Tracking and Event Analysis in crowded scenarios: Open Problem!
Video Surveillance E Senior/Feris/Tian 5 Parts-based Detectors [Pedro et al, A discriminatively trained, multiscale, deformable part model, CVPR08] Root filter (low-res) + Parts filters (high-res) Occlusion Handling
Video Surveillance E Senior/Feris/Tian 6 Parts-based Detectors Score of a window: score of root + score of parts Score of Parts: Appearance + Geometry Efficient localization of parts through Dynamic Programming SVM Classification (Structured prediction)
Video Surveillance E Senior/Feris/Tian 7 Detecting Pedestrians in Crowds [Leibe et al, Pedestrian Detection in Crowded Scenes, CVPR05] Combination of different models: bag of features, segmentation, and chamfer matching
Video Surveillance E Senior/Feris/Tian 8 Tracking in Crowds [Andriluka et al, People-tracking-by-detection and people-detection-by- tracking, CVPR08] Extends [Leibe et al, CVPR05] to temporal-domain and person articulation (parts) estimation Click for Video Demo
Video Surveillance E Senior/Feris/Tian 9 Crowd Segmentation [Dong et al, Fast Crowd Segmentation Using Shape Indexing, ICCV07]
Video Surveillance E Senior/Feris/Tian 10 Crowd Analysis [Ali & Shah, A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis, CVPR07]
Video Surveillance E Senior/Feris/Tian 11 Online Learning
Video Surveillance E Senior/Feris/Tian 12 Initialize sample weights For each cycle: Find a classifier/rectangle feature that performs well on the weighted samples Increase weights of misclassified examples Return a weighted combination of classifiers Adaboost ensembles many weak classifiers into one single strong classifier Offline Adaboost Learning
Video Surveillance E Senior/Feris/Tian 13 Offline Adaboost Learning Major Problems: Large number of examples required to train a robust classifier time consuming to label data slow training (may take several days) No Adaptation to particular surveillance scenarios
Video Surveillance E Senior/Feris/Tian 14 Learning from Small Sets Choice of Features (Levi & Weiss, CVPR04) Co-Training (Levin & Viola, ICCV2003) Online Adaptation: Online Boosting (Oza01, Javed05, Bischof06, Pham07)
Video Surveillance E Senior/Feris/Tian 15 Online Boosting [Oza,2001] Train a generic strong classifier (set of weak classifiers, # of weak classifiers fixed) on a small training set. Online Process: Given one single example with known label: Slide the example over each weak classifier When the weak classifier receives the example update the weak classifier online update the weight of the example and pass to the next weak classifier
Video Surveillance E Senior/Feris/Tian 16 Online Boosting [Oza,2001]
Video Surveillance E Senior/Feris/Tian 17 Online Boosting [Oza,2001]
Video Surveillance E Senior/Feris/Tian 18 Online Boosting Car and People Detection [Omar Javed, CVPR05] Train a generic strong classifier (set of weak classifiers, # of weak classifiers fixed) on a small training set. While running the classifier on unlabeled data, if an example is confidently predicted by a subset of weak classifiers use it for online learning Co-training framework BGS used for efficiency, for using more expensive features, and for balancing the number of positive and negative examples
Video Surveillance E Senior/Feris/Tian 19 Online Boosting Car and People Detection [Omar Javed, CVPR05]
Video Surveillance E Senior/Feris/Tian 20 More Recent Work [Bo Wu & Nevatia, Improving Part-based Object Detection by Unsupervised, Online Boosting, CVPR07] [Helmut & Hurst, Online Boosting and Vision, CVPR06] [Pham & Cham, Online Learning Asymmetric Boosted Classifiers for Object Detection, CVPR07] [Huang et al.,Incremental Learning of Boosted Face Detector, ICCV07] – Boosting Adaptation IEEE Online Learning for Classification Workshop (CVPR08)
Video Surveillance E Senior/Feris/Tian 21 Other Recent Topics
Video Surveillance E Senior/Feris/Tian 22 High-Resolution Imagery [Kopf et al, Capturing and Viewing Gigapixel Images, SIGGRAPH07] How can we make use of high-resolution in video analytics? Much more info – e.g., in face reco: skin texture, iris, etc.
Video Surveillance E Senior/Feris/Tian 23 Next Generation Neural Networks [Hinton, Reducing the dimensionality of data with neural networks, Science 2006] New algorithm for learning deep belief nets State-of-the art results in MNIST digit dataset (better than SVMs) Youtube talk at Google: Matlab Code:
Video Surveillance E Senior/Feris/Tian 24 Learning with lots of data How can we recognize thousands of products in a retail store for loss prevention? 80 Million Tiny Images ( Surveillance with Moving Cameras Cameras in vehicles, or even wearable cameras. New challenges: object detection, etc. [Leibe et al, Dynamic 3D Scene Analysis from a Moving Vehicle, CVPR 2007]
Video Surveillance E Senior/Feris/Tian 25 Many more recent topics: Check for papers in recent computer vision conferences (like CVPR, ICCV, and ECCV) and also specialized workshops/conferences such as AVSS and PETS