1 Mean shift and feature selection ECE 738 course project Zhaozheng Yin Spring 2005 Note: Figures and ideas are copyrighted by original authors

2 Outline Detecting, tracking and recognizing moving objects in video sequences is a hot topic. Mean shift algorithm---target localization Feature selection---target representation Experiment demos Conclusion and future work

3 Mean shift: (1)history Y. Cheng. “Mean Shift, Mode Seeking, and Clustering” PAMI 1995 D. Comaniciu, P Meer. “Robust analysis of feature spaces: color image segmentation” CVPR 1997 GR Bradski. “Computer vision face tracking for use in a perceptual user interface”. Intel Technology Jounal D Comaniciu, V Ramesh, P Meer. “Real-time tracking of non- rigid objects using mean shift”. CVPR 2000 Best paper award and patent filed.

4 Mean shift: (2)Idea

5 Mean shift algorithm climbs the gradient of a probability distribution to find the nearest domain mode Collins CVPR PAMI 2003

6 Mean shift: (3)Comments Instead of exhaustive search in the window, the gradient information provided by the mean shift is used to reduce the time cost Much better than moving the search window pixel by pixel and scanning row by row

7 Mean shift: (4)feature selection comes up… Given a likelihood image, locate the optimal location of the tracked object The likelihood image is generated by computing, at each pixel, the probability that the pixel belongs to the object based on the distribution of the feature

8 Feature selection: (1)Introduction Feature description approaches –Statistical descriptor –Structure descriptor –Spectral descriptor e.g. intensity, color, texture, appearance, shape, motion, depth and so Intel Technology Journal 98’

9 Feature selection: (2) histogram Color histogram is widely used as object Intel Technology Journal 98’ Red: 1D cross section of an subsampled color probability distribution of a image White: search window Blue: mean shift point

10 Feature selection: (3) comments Foreground and background appearance changes every frame, so the object features need to update Color tracking affected by colored lighting, dim illumination or too much illumination More object information should be used to increase the tracker Intel Technology Journal 98’

11 Experiment demos Original frame Probability distribution color tracking

12 Experiment demos Non-rigid shape changelighting change on object Object with non-identical color similar color with background

13 Conclusion and future work Mean shift algorithm can use view-point insensitive appearance model (like color histogram) to track non-rigid object on changing background Tracking success and failure will depend on the likelihood image which relates to the feature selection