Pedestrian Detection: introduction Approaches Holistic detection: use local search window that meets criterias Part-based detection: pedestrian as a collection of parts (to be found!) Patch-based detection: local features matched against a (learned) codebook, then voting for final detection For a survey on the approaches see: N.Dalal “Finding people in images and videos”, PhD thesis, July 2006.

Holistic approaches Some remarkable pedestrian detector Haar wavelets + SVM P. Papageorgiou and T. Poggio, “A trainable system for object detection,” Intl. J. of Computer Vision, vol. 38, no. 1, pp. 15–33, 2000. the popular face detector from Viola Jones (haar+adaboost face-detector) + motion cues P. Viola, M. Jones, and D. Snow, “Detecting pedestrians using patterns of motion and appearance,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition, New York, NY, volume 1, 2003, pp. 734–741. Histogram of oriented gradients (HOG) N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition, San Diego, CA, volume 1, 2005, pp. 886–893. Q. Zhu, S. Avidan, M. C. Yeh, and K. T. Cheng, “Fast human detection using a cascade of histograms of oriented gradients,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition, New York, NY, volume 2, 2006, pp. 1491 – 1498. Covariance Descriptors + Boosting O. Tuzel, F. Porikli, and P. Meer, “Pedestrian detection via classification on Riemannian manifolds,” To appear in IEEE Trans. Pattern Anal. Machine Intell., 2008.

Pedestrian detection using covariance descriptors and boosting rejection cascades approach Casc 1 Casc 2 Casc N Detection on negatives increases  Detection on positives decreases 

Pedestrian detection using covariance descriptors and boosting rejection cascades approach Detection on negatives increases  Detection on positives decreases 

Performance each black dot is an additional cascade Cov-Desc + Boosting HOG + Ker SVM HOG + Lin SVM HOG + Boosting each black dot is an additional cascade

Inside a cascade: the weak learners Add a weak learner to the cascade, that is Assign to it a (randomly extracted) sub-region Compute covariance-descriptor on this region for each Positive sample Negative sample Are positives and negatives “easily” separable? No? Go to 1 Yes? Done with this cascade

Covariance descriptors Define an image Region R Extract Pixel-wise Feature 1 Mean, var Covariance CR (NxN matrix, sym pos def) Extract Pixel-wise Feature 2 Mean, var Extract Pixel-wise Feature N Mean, var (e.g.: color components, luminance, gradients, …)

Covariance descriptors PROs Versatile and flexible (you can use the pixel-wise features most suitable for your goal) Computed very quickly using integral images Compact (N*(N+1)/2 independent values) CONs Euclidean distance is NOT appropriate over symmetric positive matrices: they lie over a Riemannian manifolds

Riemannian Manifolds In order to use traditional machine learning techiniques: Move back and forth from Riemannian manifold of sym pos def matrices to euclidean space of symmetric matrices using respectively Exponential of Matrix Logarithm of Matrix  Expansive computations! J. Jost, “Riemannian Geometry and Geometric Analysis”. Springer, fourth edition, 2005. X. Pennec, P. Fillard, and N. Ayache, “A Riemannian framework for tensor computing,” Intl. J. of Computer Vision, vol. 66, no. 1, pp. 41–66, 2006.

Using this detector in real-time surveillance applications Reduce the pedestrian search region where there is (or was) motion Build a motion history and focus the detection search over the most recent motion but keeping an eye on oldest motion regions Exploit the background when camera is static with implicit relevance feedback: In many surveillance scenarios it is possible to assume that the background image does not contain humans therefore, enrich the generic pedestrian classifier training some additional ad-hoc and view-dependent cascades that tackle the false-positives detected in the background The enriched classifier is to be used in the ped-detection over the given view Use the pedestrian detection to infer the scene perspective False detections are very limited, and most of them are out of perspective Therefore, having defined a perspective model, it is possible to estimate it, rejecting the outliers Once estimated, it can reject out-of-perspective detections @ICDSC09: Covariance Descriptors on Moving Regions for Human Detection in Very Complex Outdoor Scenes Giovanni Gualdi, Andrea Prati and Rita Cucchiara. Univ. of Modena and Reggio Emilia

Using this detector in real-time surveillance applications

Examples Detections using the generic classifier Detections using the + Perspective model Detections using the generic classifier + Perspective model Additional relevance feedback cascades

Videos