1. Jiu XU, Axel BEAUGENDRE and Satoshi GOTO Computer Sciences and Convergence Information Technology (ICCIT), 2011 6th International Conference on 1 Real-time Human Tracking by Detection based on HOG and Particle Filter

2.  Introduction  Proposed Method Human Detection Moving object feature, Hierarchical search algorithm Human Tracking Color-EdgeTexture histogram, Occlusion Handling Strategy  Experimental Results  Conclusion 2 Real-time Human Tracking by Detection based on HOG and Particle Filter

3.  The real-time foreground objects tracking and detection is the most critical and fundamental step in video surveillance systems.  Different from vehicles, it is much more difficult to locate and track the human body out of the background. 3 Real-time Human Tracking by Detection based on HOG and Particle Filter

4.  Pedestrian detection by Histogram of Oriented Gradient (HOG) put forward by Dalal [1] [2].  The main idea of this method is to use gradient direction histogram of small pieces to describe the image. 4 Real-time Human Tracking by Detection based on HOG and Particle Filter [1] N. Dalal, "Finding People in Images and Videos," PhD thesis, Institut National Polytechnique de Grenoble, 2006. [2] N. Dalal and B. Triggs, "Histogram of oriented gradient for human detection," in CVPR, 2005. Detection

5.  Feature extraction by adjusting the parameters, the HOG feature can effectively describe the body’s shape information.  Moreover, it also has the ability of invariance when the small local area occurs dithering and rotation. 5 Real-time Human Tracking by Detection based on HOG and Particle Filter

6.  However, the computational complexity of feature extraction makes the method very slow, and difficult to meet the needs of the practical application of the system. 6 Real-time Human Tracking by Detection based on HOG and Particle Filter

7.  Some common methods perform tracking by pre-initialized trackers based on Kalman filter [6] or particle filter [7]. 7 Real-time Human Tracking by Detection based on HOG and Particle Filter Tracking [6] Gutman, P., Velger, M. “Tracking Targets Using Adaptive Kalman Filtering”, IEEE Transactions on Aerospace and Electronic Systems Vol. 26, No. 5: pp. 691-699 1990. [7] B.Ristic, “Beyond the Kalman Filter: Particle Filters for Tracking Applications”. Arthech House, 2004.

8.  Particle filter is based on the Bayes principle and is a sequential Monte-Carlo simulation method indicated by probability density of particles. 8 Real-time Human Tracking by Detection based on HOG and Particle Filter

9.  one of the intractable problems is that the target is usually occluded by other objects frequently and instantaneously.  To select powerful features to avoid hijacking problems when tracking similar objects.  If several humans walk together as a group, we cannot separate them individually. 9 Real-time Human Tracking by Detection based on HOG and Particle Filter

10.  The target of this paper is to build a powerful tracking system for real-time surveillance system.  Combining human detection and tracking and do tracking by detection in order to achieve high accuracy and low time consumption together with occlusion solutions and group segmentations. 10 Real-time Human Tracking by Detection based on HOG and Particle Filter

11.  Introduction  Proposed Method Human Detection Moving object feature, Hierarchical search algorithm Human Tracking Color-EdgeTexture histogram, Occlusion Handling Strategy  Experimental Results  Conclusion 11 Real-time Human Tracking by Detection based on HOG and Particle Filter

12. 12 Real-time Human Tracking by Detection based on HOG and Particle Filter a Tracking by Detection System using human detection together with human tracking

13.  Histogram of Oriented Gradient (HOG) is quite a popular method of detecting people in static image. 13 Real-time Human Tracking by Detection based on HOG and Particle Filter

14.  The image is first divided into blocks while these blocks overlap with each other. Each block contains four cells. 14 Real-time Human Tracking by Detection based on HOG and Particle Filter

15.  For each pixel I(x,y), the orientation Θ(x,y) and the magnitude m (x, y) of the gradient 15 Real-time Human Tracking by Detection based on HOG and Particle Filter

16.  A histogram is calculated for each cell, and the volume of each bin is the sum of magnitude of the pixels whose orientations are in the corresponding angle interval. 16 Real-time Human Tracking by Detection based on HOG and Particle Filter

17.  However, HOG can’t be used into real-time system due to its high time consumption. Improvements  Moving object feature  Hierarchical search algorithm 17 Real-time Human Tracking by Detection based on HOG and Particle Filter

18.  Moving object feature  Kim [10] presented a new kind of non- parametric algorithm for background subtraction.  For each pixel, it builds a codebook consisting of one or more codewords. 18 Real-time Human Tracking by Detection based on HOG and Particle Filter [10] K. Kim, T. H. Chalidabhonse, D. Harwood, and L. Davis. “Real-time foreground-background segmentation using codebook model”. Elsevier Real-Time Imaging, vol. 11, no.3, 167–256, June 2005.

19.  In our previous work [13], a block-based codebook method has been proposed to make it more adoptable for human tracking.  The block feature heighten the pixel-based method to block level, thus we could take advantage of the relationship between neighboring pixels. 19 Real-time Human Tracking by Detection based on HOG and Particle Filter [13] Jiu Xu, Ning Jiang, Satoshi Goto, “Block-based Codebook Model with Oriented-Gradient Feature for Real-time Foreground Detection”, IEEE 13th International Workshop on Multimedia Signal Processing(MMSP), 2011.

20. 20 Real-time Human Tracking by Detection based on HOG and Particle Filter morphological operation (opening, closing)

21. Real-time Human Tracking by Detection based on HOG and Particle Filter 21  Hierarchical search algorithm  The expected target might appear at any position and the size would also keep changing

22. Real-time Human Tracking by Detection based on HOG and Particle Filter 22  From our experiments, we find that the curve of the number of detected pedestrians under various scaling satisfied the curve of normal distribution probability density function.  We could estimate the minimum interval [a, b].  Image scale levels only from a to b should be detected, reduce the searching levels and the amount of computation as well.

23. Real-time Human Tracking by Detection based on HOG and Particle Filter 23

24. Real-time Human Tracking by Detection based on HOG and Particle Filter 24  Particle tracker initialization

25. Real-time Human Tracking by Detection based on HOG and Particle Filter 25

26. Real-time Human Tracking by Detection based on HOG and Particle Filter 26  Color-EdgeTexture histogram  We proposed a Color-EdgeTexture Histogram to generate the weight for the observation models.  We choose the HSV color. The brightness variations while in HSV color space we could better separate the brightness with others. HSV = Hue, Saturation, Value

27. Real-time Human Tracking by Detection based on HOG and Particle Filter 27  Moreover, we add the edge local binary pattern to describe the shape texture of the moving objects.  Local binary pattern (LBP) [14] is an effective texture description operator, which could be used to measure and extract texture information from the local neighborhood in a gray image. [14] T. Ojala, M. Pietikainen, and D. Harwood, “A Comparative Study of Texture Measures with Classification Based on Feature Distributions”, Pattern Recognition, vol. 29, pp. 51-59. 1996

28. Real-time Human Tracking by Detection based on HOG and Particle Filter 28

29. Real-time Human Tracking by Detection based on HOG and Particle Filter  LBP value together with H-S color information improve the performance  Two problems : 1. time consumption. Since LBP is a pixel-wise coding, if we calculate all the LBP value inside the whole regions of predicted position of the particles, the calculation is really quite huge, thus it will extremely decrease the real-time ability.

30. Real-time Human Tracking by Detection based on HOG and Particle Filter 2. the LBP value for background part of the regions within the trackers are also calculated, it will greatly reduce the tracking rate when the size of the object is changing, and the portion of the background becomes larger and larger since the weight of the background is increased.

31. Real-time Human Tracking by Detection based on HOG and Particle Filter  To solve these problems, we use a concept of edge LBP and the edge LBP only focuses on the edge points of the foreground objects. canny edge detector

32. Real-time Human Tracking by Detection based on HOG and Particle Filter  we use a kind of H-S-ForegroundEdgeLBP histogram in which the size is 8x8x8 for each component. H-S-LBP histogram

33. Real-time Human Tracking by Detection based on HOG and Particle Filter  Particles are weighted according to the similarity between the target histogram distribution q(u) and the histogram distributions p(u) given by particles.

34. Real-time Human Tracking by Detection based on HOG and Particle Filter  the weight of the i-th particle is defined as

35. Real-time Human Tracking by Detection based on HOG and Particle Filter 35  Occlusion Handling Strategy  In traditional particle filter tracking, if the object meets some partial or total occlusions, the observation model will turn to the occluder and will not track the previous objects any longer.

36. Real-time Human Tracking by Detection based on HOG and Particle Filter 36  After color-edgetextue histogram, we define a threshold to this weight.  If the tracker moves out of the margin of the frame ‚ If the tracker is still inside the frame and the max weight is great than the threshold ƒ If the tracker is still inside the frame and the max weight is less than the threshold Delete Update Keep increase the number of the particles, searching range

37.  Introduction  Proposed Method Human Detection Moving object feature, Hierarchical search algorithm Human Tracking Color-EdgeTexture histogram, Occlusion Handling Strategy  Experimental Results  Conclusion 37 Real-time Human Tracking by Detection based on HOG and Particle Filter

38. 38 Other method

39. Real-time Human Tracking by Detection based on HOG and Particle Filter 39 Other method

40. Real-time Human Tracking by Detection based on HOG and Particle Filter 40 Other method

41. Real-time Human Tracking by Detection based on HOG and Particle Filter 41 Separate group

42. Real-time Human Tracking by Detection based on HOG and Particle Filter 42 The time consumption of our method is much less than [8][9][11]. (400 frames long)

43. [8] L.M.Fuentes and S.A.Velastin, “People tracking in surveillance applications”, Image and Vision Computing, pp.1165-1171, 2006 [9] Tao Yang, Quan Pan, Jing Li and Li, S.Z. “Real-time Multiple Objects Tracking with Occlusion Handling in Dynamic Scene” In CVPR, Vol.1. pp. 970-975, 2005 [11] R. Hess and A. Fern, “Discriminatively Trained Particle Filters for Complex Multi-Object Tracking”. In CVPR, 2009. Real-time Human Tracking by Detection based on HOG and Particle Filter 43

44.  Uses a combination strategy of HOG human detection method and particle filter human tracking algorithm in surveillance system.  Original HOG feature is not be so efficient in real- time system  In order to enhance the performance in color- based particle tracking  The proposed method has a good robustness in all kinds of situations together with low time consumption. Real-time Human Tracking by Detection based on HOG and Particle Filter 44

45. Real-time Human Tracking by Detection based on HOG and Particle Filter 45