Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern Recognition (ICPR ’ 06)

Outline  Introduction  Initialization and Pixel Classification  Single Object Tracking  Tracking Occluded Objects  Experimental Results  Conclusion

Outline  Introduction  Initialization and Pixel Classification  Single Object Tracking  Tracking Occluded Objects  Experimental Results  Conclusion

Introduction  human objects tracking systems Pfinder  Utilize stochastic, region-based feature McKenna et al.  Adaptive Gaussian mixture to model color distribution M2Tracker  Combine presence probability with color model to classify each pixel Tsutsui et al.  Exchange the optical flow information within multiple cameras

Multiple human objects tracking system  System consist of Model-based object segmentation Remove noise of segmented region Optical flow-based position estimation Occlusion detection Object separation from occlusion  Contribution Track occlusion, separate object and track it individually afterwards

Outline  Introduction  Initialization and Pixel Classification  Single Object Tracking  Tracking Occluded Objects  Experimental Results  Conclusion

Gaussian Mixture Color Model  Condition probability for pixel i belong to object O is  Parameters: mean, and covariance matrix  Expectation-maximization (EM) algorithm To determine the maximum likelihood parameters of a mixture of m Gaussian

Color Model  Use HIS space to reduce ambient illumination change Each pixel i has 2-D feature vector where h i is the hue, s i is the saturation  Likelihood pixel i belonging to torso (n=0) or the bottom (n=1) of a person O is

Color similarity  The color of the torso of object 1 is similar to the color of the bottom of object 2  (b) is the result of applying the torso color model of object 2 for all pixels

Initialize Presence Map  Presence map The set of presence probabilities of the pixels inside the object  Head line Scan the torso projection profile H 0 (y i ) y HL =arg min yi H 0 (y i )  Torso line  Central vertical axis Probabilities of the pixels will be larger

Bayesian Classification  Only consider pixels in the neighborhood of an object P posteriori (O k |i) : posterior prob. of pixel i belong to object O k P(i|O k ) : probability defined by torso or bottom model P priori (O k ) : presence probability of O k  Relative coordinate Defined by the head line and central axis  Color model selection for torso or bottom  If P posteriori (O k |i) >=0.05, then i classified to O k

Outline  Introduction  Initialization and Pixel Classification  Single Object Tracking  Tracking Occluded Objects  Experimental Results  Conclusion

Single Object Tracking  Flow chart of single object tracking  Newcomer detection By using background subtraction

Tracking process  Calculate angles and magnitudes of the flow vectors in the neighborhood of window  Quantize the direction into 12 bins (30 degree/bin) and determine which bin object belong to  Find the most significant bin and calculate average flow  Shift object window by average flow

Update presence map  Size and shape of a moving object change over time Need to update the presence map  If pixel at (x r, y r ) classified correctly, increase the corresponding priori prob. for every 10 frames

Outline  Introduction  Initialization and Pixel Classification  Single Object Tracking  Tracking Occluded Objects  Experimental Results  Conclusion

Tracking Occluded Objects  Optical flow and presence probability are unreliable  Only use color models to estimate object ’ s central vertical axis  Use distance between central axes to determine object becomes separable

Occlusion detection  Each individual object has five attributes based on its activity Two object windows touch and form an occlusion window Two object windows overlap and form an occlusion window A single object joins an occlusion and form a new occlusion window

Occlusion group separation  Compute distance between every two objects in an occlusion group as  If Two extreme objects O i and O i+1 If |d i | > threshold, then determine O i or O i+1 can be separate from the original occlusion

Object separation example separate

Resume tracking  One an object separate from occlusion, we need to update: Object window location, head line, and torso line Central vertical axis  Left and right boundary Scan the vertical projection profile of O k From the central vertical axis leftward and then rightward  Head line and torso line Analyze the horizontal projection profile

Outline  Introduction  Initialization and Pixel Classification  Single Object Tracking  Tracking Occluded Objects  Experimental Results  Conclusion

Tracking example 1  Format: Image frame is 160x120x24 bits, 15 frames/sec Occlusion 1 Occlusion 2 Object 2 separate and join occlusion 1 single object 4 occlusion 1

Tracking example 2  Two occlusion groups merge as one and then separate to another two occlusion groups

System evaluation and error analysis  Three separation events: 2-object, 3-object, and 4-object separation event  Define separation occurs ’ accuracy based on: A single object leaves an occlusion and track him correctly afterward If an occlusion splits into two, system identify the correct objects in the two pairs.  More 2-object separation events

Outline  Introduction  Initialization and Pixel Classification  Single Object Tracking  Tracking Occluded Objects  Experimental Results  Conclusion

Conclusion  Object tracking consists of: Gaussian mixture model Presence probability Optical flow  Objects under occlusion Use color model to distinguish each object and locate central vertical axes  Object separation Determine by distances between the central vertical axes of objects