1. CVPR 2006 New York City Granularity and Elasticity Adaptation in Visual Tracking Ming Yang, Ying Wu NEC Laboratories America Cupertino, CA 95014 myang@sv.nec-labs.com EECS Dept. Northwestern Univ. Evanston, IL 60208 yingwu@ece.northwestern.edu

2. CVPR 2008 Anchorage, Alaska2 Motivation General targets exhibit enormous variability and unpredictable changes. –rotation and scale changes –different degrees of deformations –partial occlusions Most observation models tend to focus on certain characteristics of targets. Adaptation of more aspects of target observation models is preferable.

3. CVPR 2008 Anchorage, Alaska3 Appearance-based visual tracking Two key aspects in designing appearance based observation models: –the abstraction level of features, –how to take into account the geometrical structures of targets.

4. CVPR 2008 Anchorage, Alaska4 Granularity vs. Elasticity Feature Granularity: the abstraction level of features. –e.g. features describe attributes of a pixel, a blob region or a whole object. Model Elasticity: the ability that the model can tolerate geometrical changes among components.

5. CVPR 2008 Anchorage, Alaska5 Comparisons Comparisons of different tracking approaches in terms of their relative granularity and elasticity.

6. CVPR 2008 Anchorage, Alaska6 The paradigm We propose a general tracking paradigm. The target is represented by a MRF of interest regions. Adaptation of the feature granularity and model elasticity to maximize the likelihood of the MRF.

7. CVPR 2008 Anchorage, Alaska7 Target observation model An MRF model of interest regions –X={x i }: the initial interest regions –Y={y i }: the detected interest regions in every frame Substantialize to different models –Every pixel is an interest region => Template –The target is one interest region => Meanshift

8. CVPR 2008 Anchorage, Alaska8 Target model construction Harris-Laplace i nterest region detection –Represented by the location, characteristic scale, and shape matrix MRF model: pair-wise potential among overlapped interest regions.

9. CVPR 2008 Anchorage, Alaska9 Model the granularity and elasticity The pair-site potential is defined based on the relative angles. –The parameter models the elasticity. The likelihood of individual interest region is defined using the Bahattachaya coefficient of feature histograms –The scale ratio r regulates the image region to extract features so as to models the granularity.

10. CVPR 2008 Anchorage, Alaska10 Motion estimation Coarse motion estimation –The motion parameters are estimated independently based on the detected pair-wise cliques. Motion parameters refinement –Jointly sample the motion parameters and evaluate the posteriors of the hypotheses

11. CVPR 2008 Anchorage, Alaska11 Feature granularity adaptation Update the scale ratio by searching r t until a local maximum of Rigid and stable targets => large ratio r can yield good matching Partial occlusion or deformation happens => small ratio r may be appropriate.

12. CVPR 2008 Anchorage, Alaska12 Model elasticity adaptation Update the parameter in the pair-site potential function by maximizing the likelihood of the current tracking result: The optimal is the variance of the observed angle differences.

13. CVPR 2008 Anchorage, Alaska13 Experiment settings Up to 12 integration scales used in Harris- Laplace interest region detection. Features for the interest regions are 2D histograms in Normalized-RG space with 24*24 bins. Interest regions matching: Runs at 2-10 fps on a Pentium 3GHz desktop.

14. CVPR 2008 Anchorage, Alaska14 Illustration

15. CVPR 2008 Anchorage, Alaska15 More tracking results

16. CVPR 2008 Anchorage, Alaska16 Conclusion A novel perspective of adapting target observation models. –able to automatically tune the observation model’s focus on target’s appearances and structures. –flexible to incorporate different interest region detection and features extraction.