1. U NIVERSITEIT VAN A MSTERDAM IAS INTELLIGENT AUTONOMOUS SYSTEMS 1 M. Hofmann Prof. Dr. D. M. Gavrila Intelligent Systems Laboratory Informatics Institute, Faculty of Science University of Amsterdam Web: www.gavrila.net Looking at People - Detecting People in Images by their Body Parts

2. U NIVERSITEIT VAN A MSTERDAM IAS INTELLIGENT AUTONOMOUS SYSTEMS 2 motion capture for animation and games surveillance (i.e. CASSANDRA system, see afternoon presentation) robotic pets motion analysis (sports, medical) pedestrian protection smart homes, elderly care Motivation for People Detection

3. U NIVERSITEIT VAN A MSTERDAM IAS INTELLIGENT AUTONOMOUS SYSTEMS 3 Project (Sub)Tasks Detect people in images by 1.identifying regions of interest (ROIs) 2.detecting individual body parts (faces, head-shoulders, upper bodies, lower bodies) 3.combining results of individual body part-detectors (This also is possible work-breakdown of 3 person DOAS team)

4. U NIVERSITEIT VAN A MSTERDAM IAS INTELLIGENT AUTONOMOUS SYSTEMS 4 1. Identifying ROIs: Background Modeling Source: P. Withagen (UvA) adjacent frame difference mean & threshold mean & covariance (single Gaussian) mixture of Gaussians Kalman filtering Pixel-based methods „Time of Day“: gradual illumination changes „Waving trees“: background can vacillate „Shadows“ „Camouflage“ „Initialisation“ Challenges

5. U NIVERSITEIT VAN A MSTERDAM IAS INTELLIGENT AUTONOMOUS SYSTEMS 5 2. Detecting Individual Body Parts Use of machine learning techniques Viola & Jones approach (ICCV’2003): use Haar wavelet features with AdaBoost cascade hypotheses classifier stage 2 classifier stage 1 classifier stage N accepted hypotheses (detections) hypotheses rejected hypotheses

6. U NIVERSITEIT VAN A MSTERDAM IAS INTELLIGENT AUTONOMOUS SYSTEMS 6 3. Combine Results of Individual Part-Detectors [Mohan2001, Wu2005]: fixed spatial layout, combination of contribution of individual part-detectors by weighted sum or by additional classifier [Mikolajczyk2004, Micilotta2005]: spatial distribution is learnt, estimation of joint probabilities

7. U NIVERSITEIT VAN A MSTERDAM IAS INTELLIGENT AUTONOMOUS SYSTEMS 7 Various Intel OpenCV Library, 2007 http://www.intel.com/technology/computing/opencv/index.htm for image filtering, individual body-part detectors, etc. http://www.intel.com/technology/computing/opencv/index.htm LibSVM, a library for Support Vector Machine classification http://www.csie.ntu.edu.tw/~cjlin/libsvm/ http://www.csie.ntu.edu.tw/~cjlin/libsvm/ Daimler Image Label Tool, ROC utilities Dataset Training: already pre-trained V&J cascade detectors: OpenCV, UvA any others from the web? Test: CASSANDRA dataset (about 5000 images, partially labeled, consider only fully visible people) System development under MS Visual Studio C++ environment. Use of following libraries / utilities: Software

8. U NIVERSITEIT VAN A MSTERDAM IAS INTELLIGENT AUTONOMOUS SYSTEMS 8 Bibliography [Gavrila1999] D. M. Gavrila. „The Visual Analysis of Human Movement: A Survey“, Computer Vision and Image Understanding, 73(1):82-98, 1999 [DOAS2007] S. Korzec, H. Visser and M. Goksun. “Detecting Humans by Combining Human Part-detectors in an Urban Setting”. DOAS Final Project 2007. [Viola2003] P. Viola, M.J. Jones and D. Snow. „Detecting Pedestrians using Patterns of Motion and Appearance“. Proc. of ICCV, pp.734-741, Nice, France, 2003. [Mohan2001] A. Mohan, C. Papageorgiou and T. Poggio „Example-Based Object Detection in Images by Components“, IEEE Transactions on PAMI, 23 (4), pp. 349-361, 2001. [Micilotta2005] A.S. Micilotta, E.J. Ong and R. Bowden. “Detection and Tracking of Humans by Probabilistic Body Part Assembly”. BMVC’05. [Wu2005a] B. Wu and R. Nevatia. “Detection of Multiple, Partially Occluded Humans in a Single Image by Bayesian Combination of Edgelet Part Detectors”, ICCV’05. [Mikolajczyk2004] K. Mikolajczyk, D. Schmid, A. Zisserman, “Human detection based on a probabilistic assembly of robust part detectors”, Proc. ECCV, Prague, Czech Republic, May 11–14, 2004.