Real-Time Tracking with Mean Shift Presented by: Qiuhua Liu May 6, 2005

Outline Color model for the target Introduction to mean-shift Tracking algorithm with mean shift Compassion with Particle Filter algorithm with the similar color model

Color Model for the Target The target is represented by an ellipsoidal region in the image, normalized to a unit circle. Let be the normalized pixel locations in the region centered at 0. The probability of the feature(color) of the target was modeled by the its histogram with kernel : The kernel has a convex and monotonic decreasing kernel profile, assigning small weights to pixels farther away from the center.

The profile of kernel is defined as a function such that Let be the normalized pixel locations of the target candidates, centered at y in the current frame. The target candidate is modeled as: Target Candidate

Similarity Function The similarity function is defined as the metric distance between the candidate and the target model: Choose as the Bhattacharyya coefficients (it is a divergence type measure) Minimizing the distance is equivalent to maximizing.

Maximization with Mean Shift Assume the target candidate histogram does not change drastically, using Taylor expansion around the values at location : Only need to maximize the second term, which is the density estimate with kernel profile k(x) at y in the current frame, with the data being weighted by w i. where

Mean Shift First Introduced by Fukunaga and Hostetler in 1975 [1], Mean shift is a non-parametric, iterative procedure to find the mode of a density function represented by a set of samples and a Kernel K : d: dimension of data; h: band width. With the definition of the Profile of a kernel:

With mean shift method, the kernel is recursively moved from the current location to the new location until converge with: For a kernel with a convex and monotonic decreasing kernel profile, it is guaranteed to converge (to local maxima) where Mean Shift

The Epanechnikov kernel has a profile: Then where c d is the volume of the unit d -dimensional sphere. One Normally Used Kernel (*)

Tracking Algorithm with Mean Shift Very Simple: Given the target model and its location in the previous frame. 1. Initialize the location at the current frame with. 2. Compute the next location according to (*). 3. Iterate 1 and 2 until converge.

Tracked Result:

Mean Shift Maximization:

Summary and Comparison to Particle Filter Method Advantage: Good color histogram model and distance measure. Deterministic method: the mean shift usually converged at 2 to 3 iterations – Fast. Disadvantage: Sometimes get stuck at local minimum. Difficult to handle abrupt motion: Due to use of the kernels, the center of the target in the current frame has to be covered by the target model in the previous frame. Otherwise, the local maximum of the Bhattacharyya coefficient would not be a reliable indicator.

Connection to Particle Filter Tracking Adopting the same distance measure, Jaco Vermaak [4][5] proposed the following observation likelihood function for probabilistic tracking with particle filters and VB inference : The histogram does not necessarily need a kernel.

Comparison Top: Deterministic with Mean-shift Bottom: Probabilistic with particle filters

References [1] Fukunaga et al, “ The Estimation of the Gradient of a Density Function, with Applications in Pattern Recognition ”, IEEE Trans. on Information Theory, 1975 [2] Dorin Comaniciu et al, “ Real-time Tracking of Non-Rigid Objects Using Mean Shift ”, CVPR [3] Dorin Comaniciu et al, “ Kernel-Based Object Tracking ”, IEEE Trans. On Pattern Analysis and Machine Learning, May [4] Jaco Vermaak et al [5] Jaco Vermaak et al, “ Variational Inference for Visual Tracking ”, CVPR, 2003