Jiu XU, Axel BEAUGENDRE and Satoshi GOTO Computer Sciences and Convergence Information Technology (ICCIT), th International Conference on 1 Real-time Human Tracking by Detection based on HOG and Particle Filter
Introduction Proposed Method Human Detection Moving object feature, Hierarchical search algorithm Human Tracking Color-EdgeTexture histogram, Occlusion Handling Strategy Experimental Results Conclusion 2 Real-time Human Tracking by Detection based on HOG and Particle Filter
The real-time foreground objects tracking and detection is the most critical and fundamental step in video surveillance systems. Different from vehicles, it is much more difficult to locate and track the human body out of the background. 3 Real-time Human Tracking by Detection based on HOG and Particle Filter
Pedestrian detection by Histogram of Oriented Gradient (HOG) put forward by Dalal [1] [2]. The main idea of this method is to use gradient direction histogram of small pieces to describe the image. 4 Real-time Human Tracking by Detection based on HOG and Particle Filter [1] N. Dalal, "Finding People in Images and Videos," PhD thesis, Institut National Polytechnique de Grenoble, [2] N. Dalal and B. Triggs, "Histogram of oriented gradient for human detection," in CVPR, Detection
Feature extraction by adjusting the parameters, the HOG feature can effectively describe the body’s shape information. Moreover, it also has the ability of invariance when the small local area occurs dithering and rotation. 5 Real-time Human Tracking by Detection based on HOG and Particle Filter
However, the computational complexity of feature extraction makes the method very slow, and difficult to meet the needs of the practical application of the system. 6 Real-time Human Tracking by Detection based on HOG and Particle Filter
Some common methods perform tracking by pre-initialized trackers based on Kalman filter [6] or particle filter [7]. 7 Real-time Human Tracking by Detection based on HOG and Particle Filter Tracking [6] Gutman, P., Velger, M. “Tracking Targets Using Adaptive Kalman Filtering”, IEEE Transactions on Aerospace and Electronic Systems Vol. 26, No. 5: pp [7] B.Ristic, “Beyond the Kalman Filter: Particle Filters for Tracking Applications”. Arthech House, 2004.
Particle filter is based on the Bayes principle and is a sequential Monte-Carlo simulation method indicated by probability density of particles. 8 Real-time Human Tracking by Detection based on HOG and Particle Filter
one of the intractable problems is that the target is usually occluded by other objects frequently and instantaneously. To select powerful features to avoid hijacking problems when tracking similar objects. If several humans walk together as a group, we cannot separate them individually. 9 Real-time Human Tracking by Detection based on HOG and Particle Filter
The target of this paper is to build a powerful tracking system for real-time surveillance system. Combining human detection and tracking and do tracking by detection in order to achieve high accuracy and low time consumption together with occlusion solutions and group segmentations. 10 Real-time Human Tracking by Detection based on HOG and Particle Filter
Introduction Proposed Method Human Detection Moving object feature, Hierarchical search algorithm Human Tracking Color-EdgeTexture histogram, Occlusion Handling Strategy Experimental Results Conclusion 11 Real-time Human Tracking by Detection based on HOG and Particle Filter
12 Real-time Human Tracking by Detection based on HOG and Particle Filter a Tracking by Detection System using human detection together with human tracking
Histogram of Oriented Gradient (HOG) is quite a popular method of detecting people in static image. 13 Real-time Human Tracking by Detection based on HOG and Particle Filter
The image is first divided into blocks while these blocks overlap with each other. Each block contains four cells. 14 Real-time Human Tracking by Detection based on HOG and Particle Filter
For each pixel I(x,y), the orientation Θ(x,y) and the magnitude m (x, y) of the gradient 15 Real-time Human Tracking by Detection based on HOG and Particle Filter
A histogram is calculated for each cell, and the volume of each bin is the sum of magnitude of the pixels whose orientations are in the corresponding angle interval. 16 Real-time Human Tracking by Detection based on HOG and Particle Filter
However, HOG can’t be used into real-time system due to its high time consumption. Improvements Moving object feature Hierarchical search algorithm 17 Real-time Human Tracking by Detection based on HOG and Particle Filter
Moving object feature Kim [10] presented a new kind of non- parametric algorithm for background subtraction. For each pixel, it builds a codebook consisting of one or more codewords. 18 Real-time Human Tracking by Detection based on HOG and Particle Filter [10] K. Kim, T. H. Chalidabhonse, D. Harwood, and L. Davis. “Real-time foreground-background segmentation using codebook model”. Elsevier Real-Time Imaging, vol. 11, no.3, 167–256, June 2005.
In our previous work [13], a block-based codebook method has been proposed to make it more adoptable for human tracking. The block feature heighten the pixel-based method to block level, thus we could take advantage of the relationship between neighboring pixels. 19 Real-time Human Tracking by Detection based on HOG and Particle Filter [13] Jiu Xu, Ning Jiang, Satoshi Goto, “Block-based Codebook Model with Oriented-Gradient Feature for Real-time Foreground Detection”, IEEE 13th International Workshop on Multimedia Signal Processing(MMSP), 2011.
20 Real-time Human Tracking by Detection based on HOG and Particle Filter morphological operation (opening, closing)
Real-time Human Tracking by Detection based on HOG and Particle Filter 21 Hierarchical search algorithm The expected target might appear at any position and the size would also keep changing
Real-time Human Tracking by Detection based on HOG and Particle Filter 22 From our experiments, we find that the curve of the number of detected pedestrians under various scaling satisfied the curve of normal distribution probability density function. We could estimate the minimum interval [a, b]. Image scale levels only from a to b should be detected, reduce the searching levels and the amount of computation as well.
Real-time Human Tracking by Detection based on HOG and Particle Filter 23
Real-time Human Tracking by Detection based on HOG and Particle Filter 24 Particle tracker initialization
Real-time Human Tracking by Detection based on HOG and Particle Filter 25
Real-time Human Tracking by Detection based on HOG and Particle Filter 26 Color-EdgeTexture histogram We proposed a Color-EdgeTexture Histogram to generate the weight for the observation models. We choose the HSV color. The brightness variations while in HSV color space we could better separate the brightness with others. HSV = Hue, Saturation, Value
Real-time Human Tracking by Detection based on HOG and Particle Filter 27 Moreover, we add the edge local binary pattern to describe the shape texture of the moving objects. Local binary pattern (LBP) [14] is an effective texture description operator, which could be used to measure and extract texture information from the local neighborhood in a gray image. [14] T. Ojala, M. Pietikainen, and D. Harwood, “A Comparative Study of Texture Measures with Classification Based on Feature Distributions”, Pattern Recognition, vol. 29, pp
Real-time Human Tracking by Detection based on HOG and Particle Filter 28
Real-time Human Tracking by Detection based on HOG and Particle Filter LBP value together with H-S color information improve the performance Two problems : 1. time consumption. Since LBP is a pixel-wise coding, if we calculate all the LBP value inside the whole regions of predicted position of the particles, the calculation is really quite huge, thus it will extremely decrease the real-time ability.
Real-time Human Tracking by Detection based on HOG and Particle Filter 2. the LBP value for background part of the regions within the trackers are also calculated, it will greatly reduce the tracking rate when the size of the object is changing, and the portion of the background becomes larger and larger since the weight of the background is increased.
Real-time Human Tracking by Detection based on HOG and Particle Filter To solve these problems, we use a concept of edge LBP and the edge LBP only focuses on the edge points of the foreground objects. canny edge detector
Real-time Human Tracking by Detection based on HOG and Particle Filter we use a kind of H-S-ForegroundEdgeLBP histogram in which the size is 8x8x8 for each component. H-S-LBP histogram
Real-time Human Tracking by Detection based on HOG and Particle Filter Particles are weighted according to the similarity between the target histogram distribution q(u) and the histogram distributions p(u) given by particles.
Real-time Human Tracking by Detection based on HOG and Particle Filter the weight of the i-th particle is defined as
Real-time Human Tracking by Detection based on HOG and Particle Filter 35 Occlusion Handling Strategy In traditional particle filter tracking, if the object meets some partial or total occlusions, the observation model will turn to the occluder and will not track the previous objects any longer.
Real-time Human Tracking by Detection based on HOG and Particle Filter 36 After color-edgetextue histogram, we define a threshold to this weight. If the tracker moves out of the margin of the frame If the tracker is still inside the frame and the max weight is great than the threshold If the tracker is still inside the frame and the max weight is less than the threshold Delete Update Keep increase the number of the particles, searching range
Introduction Proposed Method Human Detection Moving object feature, Hierarchical search algorithm Human Tracking Color-EdgeTexture histogram, Occlusion Handling Strategy Experimental Results Conclusion 37 Real-time Human Tracking by Detection based on HOG and Particle Filter
38 Other method
Real-time Human Tracking by Detection based on HOG and Particle Filter 39 Other method
Real-time Human Tracking by Detection based on HOG and Particle Filter 40 Other method
Real-time Human Tracking by Detection based on HOG and Particle Filter 41 Separate group
Real-time Human Tracking by Detection based on HOG and Particle Filter 42 The time consumption of our method is much less than [8][9][11]. (400 frames long)
[8] L.M.Fuentes and S.A.Velastin, “People tracking in surveillance applications”, Image and Vision Computing, pp , 2006 [9] Tao Yang, Quan Pan, Jing Li and Li, S.Z. “Real-time Multiple Objects Tracking with Occlusion Handling in Dynamic Scene” In CVPR, Vol.1. pp , 2005 [11] R. Hess and A. Fern, “Discriminatively Trained Particle Filters for Complex Multi-Object Tracking”. In CVPR, Real-time Human Tracking by Detection based on HOG and Particle Filter 43
Uses a combination strategy of HOG human detection method and particle filter human tracking algorithm in surveillance system. Original HOG feature is not be so efficient in real- time system In order to enhance the performance in color- based particle tracking The proposed method has a good robustness in all kinds of situations together with low time consumption. Real-time Human Tracking by Detection based on HOG and Particle Filter 44
Real-time Human Tracking by Detection based on HOG and Particle Filter 45