1. Online Updating Appearance Generative Mixture Model for Meanshift Tracking Jilin Tu, Thomas Huang Elec. and Comp. Engr. Dept. Univ. of Illinois at Urbana and Champaign Urbana, IL 61801 {jilintu, huang}@ifp.uiuc.edu Hai Tao Elec. Engr. Dept. Univ. of Calif. at Santa Cruz Santa Cruz, CA 12345 tao@soe.ucsc.edu

2. Content Problem Statement Overview of tracking object with appearance variations Overview of Meanshift Tracking Meanshift tracking with online appearance updating Generative model for histogram appearance updating Experiments Summary

3. Problem Statement Our Goal: –To track object of vast (but typical) appearance variations by online appearance updating. –To avoid drifting problem which is typical in most tracking algorithm with online model updating mechanism. –To Infer object status by modeling the appearance variations.

4. Overview: tracking of object with appearance variations Subspace Model –Black1998 Eigentracking: Model the object appearances by PCA. Unable to incorporate novelties. –Ross2004 Eigentracking with incremental PCA. Handles appearance novelties better. –Ho2004 Tracking with L-Infinity norm subspace. Handles appearance novelties the best, but no theoretical gurranttee for avoiding drifting problem. Mixture model –Jepson2003 Model the appearance variations as mixture of stable appearance, outliers and two frame variations caused by optical flow. Drifting problem is not solved.

5. Overview of Meanshift Tracking –Meanshift(Chen1995): An iterative procedure for finding the data density modes (or cluster centers) of an ensemble of data samples. By shifting toward the mean of the samples in the vicinity.

6. Overview of Meanshift Tracking Meanshift tracking –Setting: The likelihood of the object appearance is represented by ensemble of likelihood pixel locations in a likelihood image. –Na ï ve Intuition (CAMSHIFT): Follow the cluster center using meanshift while the cluster center of the likelihood image moves. –Approach(Comaniciu2000): Maximizing the similarity of the histogram in the tracking window and the static histogram of the object appearance.

7. Overview of Meanshift Tracking Meanshift tracking in a nut shell

8. Overview of Meanshift Tracking Meanshift tracking properties –Pros: Realtime Histogram appearance model tolarates minor appearance variations. –Cons: One single static histogram does not capture the appearance variations of the object.

9. Meanshift Tracking with online appearance updating The framework –The idea: updating the histogram with the constraint of key appearances.

10. Generative model for histogram updating The generative model

11. EM rules for histogram updating Expectation Maximization

12. Online EM rules for histogram updating Expectation of Log-likelihood taking into the consideration of past observations where Online Maximization Rule

13. Histogram Updating Rule Given the generative model we have

14. Experiments Application: Human head tracking with coarse head pose inference. –Head appearance changes substantially when human turns his/her head. –Usually only near frontal head pose provides more information about the person ’ s identity, facial expression, eye gaze, lip movement, etc. –It is important to track the head and detect near frontal head pose simultaneously.

15. Experiments Tracking with L-Infinity Subspace Updating

16. Experiments Tracking by Meanshift Tracker

17. Experiments Meanshift Tracking with online updating

18. Experiments Meanshift Tracking with online updating: Another example Pose estimation accuracy 77%

19. Summary Our tracker with online appearance updating –More robust comparing to the standard Meanshift tracker. –Better tracking accuracy. –Avoid drifting problem. –Can infer the object state in relation to the appearance variation. –Acquisition of more than one key appearance of the object sounds annoying, but is practical in most online/offline tracking tasks.

20. Questions