Domenico Bloisi, Luca Iocchi, Dorothy Monekosso, Paolo Remagnino

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 2 Video surveillance tasks A video surveillance system may accomplish a series of well-defined tasks: To detect objects of interest (we may want to detect all the moving cars in a street) [Yoneama et al. 2005, ] To track objects of interest (we may want to know the exact number of people standing in a room) [Khan and Shah 2006, ] To react to particular events (we may want to send an alarm if an unauthorized person enters a restricted area) [Leo et al. 2005, ] …

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 3 PB-KU Visual modeling of people behaviors and interactions for professional training (PB-KU) The method is applied to the training of nurses in the School of Nursing in the Faculty of CISM at Kingston University [ ]. The aim is to detect and track people in order to analyze their behavior

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 4 Summary Project Overview Segmentation Height Image Algorithm Examples Results

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 5 Features Background: – Dynamic background (indoor, crowded) Number of objects to track: – Up to 15 people in the scene Camera: – Two stereo cameras Evaluation method: – Evaluation on a on-site built data-set

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 6 General architecture

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 7 Hardware Videre Design STH-MDCS Intel Core 2 Duo 2,0 GHz CPU Mac mini Videre Design STH-MDCS Intel Core 2 Duo 2,0 GHz CPU Mac mini Wireless connection Firewire connection Firewire connection

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 8 Segmentation (Detecting objects of interest) Background Estimate Current Frame Background Model Foreground Extraction List of Detected Objects

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 9 Background Modeling Background Image computed from S (the image displays only the higher Gaussian values) Set S of n images from a camera Raw images Artificial image

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 10 Foreground Extraction (Background Subtraction Technique) THRESHOLD T (based on illumination conditions) blobs (Binary Large OBjectS) > T current frame foreground image background image

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 11 Background Subtraction Problems Background subtraction is a fast and effective technique, but it presents a series of problems:  How to compute a correct background? [Heikkilä and Silven 1999] [Stauffer and Grimson 1999]  How to manage gradual and sudden illumination changes? [Bloisi et al. 2007]  How to manage high-frequencies background objects (such as artificial light flickering, windows) [Bloisi and Iocchi 2008]

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 12 Proposed Solution Background Subtraction + Stereo Vision + Edge Detection + Height Image Algorithm Advantages: robust and efficient foreground extraction, shadow suppression, 3D information, non-moving object filtering, accurate multiple object segmentation.

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 13 System architecture PB-KU

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 14 Height Image Algorithm t: minimum area for a blob to be considered of interest A: set of found activity blobs F: final set of the segmented objects we are searching for H: the set of height images.

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 15 Height Image a)Active blobs b)Height image c)Segmented image

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 16 Example Segmented imageGround plane view

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 17 Example Crowd flow Analysis

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 18 Evaluation and Metrics Evaluation On-site dataset Metrics Scene accuracy A is the average accuracy Number of detected people Number of people in the scene

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 19 Segmentation Results Segmentation Accuracy on 100 randomly chosen images

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 20 Algorithm Speed

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 21 Conclusions Summary of results  Accurate segmentation even in case of 15 people in the scene  Real-time computation  Ground plane view projection for crowd flow analysis

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 22 Future Work (1) Add Radio Frequency Identifiers (RFID) to stereo for helping segmentation and dealing with occlusions RFID Identity LocalizationStereo

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 23 Future Work (2) Crowd flow analysis based on ground plane projection Example How many people are near a bed in event of an emergency?

25/06/2015A Novel Segmentation Method for Crowded Scenes – VISAPP 2009 Page 24 References - A. Yoneama C.H. Yeh, C.C.J. Kuo. Robust Vehicle and Traffic Information Extraction for Highway Surveillance, JASP(2005), No. 14, pp , M. Leo, T. D’Orazio, A. Caroppo, T. Martiriggiano P. Spagnolo. Automatic Monitoring of Forbidden Areas to Prevent Illegal Accesses ICAPR (2), pp , S. Khan and M. Shah. A multiview approach to tracking people in crowded scenes using a planar homography constraint. In ECCV (4), pp. 133–146, Y.T. Tsai, H.C. Shih and C.-L. Huang. Multiple human objects tracking in crowded scenes. In ICPR ’06, pp. 51–54, J. Heikkilä, O. Silven. A real-time system for monitoring of cyclists and pedestrians. Proc. 2° IEEE International Workshop on Visual Surveillance, pp , C. Stauffer, W. Grimson. Adaptive background mixture models for real-time tracking. (CVPR'99), pp , D. Bloisi, L. Iocchi, G.R. Leone, R. Pigliacampo, L. Tombolini, L. Novelli. A Distributed Vision System for Boat Traffic Monitoring in the Venice Grand Canal (VISAPP), pp , D. Bloisi and L. Iocchi. ARGOS - A Video Surveillance System for Boat Traffic Monitoring in Venice. IJPRAI, 2008.

Domenico Bloisi, Luca Iocchi, Dorothy Monekosso, Paolo Remagnino