1. Visual Tracking Decomposition Junseok Kwon* and Kyoung Mu lee Computer Vision Lab. Dept. of EECS Seoul National University, Korea Homepage: http://cv.snu.ac.krhttp://cv.snu.ac.kr

2. Goal of Visual Tracking  Robustly tracks target in real-world scenarios Frame #1 Frame #60

3. Real-World Scenarios Occlusions Illumination changes Abrupt motions Pose variations Mixed

4. Previous Works  In the real-world scenarios, conventional tracking methods frequently fail. OAL Tracker [2] MIL Tracker [1]Our method [2] Ross et. al. Incremental learning for robust visual tracking. IJCV 2007. [1] Babenko et. al. Visual tracking with online multiple instance learning. CVPR 2009.

5. Bayesian Tracking Approach  Maximum a Posteriori (MAP) estimate Position, scale color edge

6. Bayesian Tracking Approach  Observation model  Motion model  Update rule

7. Compound Model Compound Observation Model Pose variation Occlusion Illumination change Clutters Smooth Abrupt Compound Motion Model  Need for real-world scenarios  But difficult to design

8. Our Approach  Observation Model Decomposition + ++ Basic Observation Model 1 Basic Observation Model r Basic Observation Model 2 Compound Observation Model

9. Our Approach + ++ Basic Motion Model 2 Basic Motion Model s Basic Motion Model 1  Motion Model Decomposition Compound Motion Model

10. Our Approach Basic Tracker 1 Basic Tracker 2 Basic Tracker rs  Tracker Decomposition Basic Observation Model 1 Basic Motion Model 1 Basic Motion Model 2 Basic Motion Model s Basic Observation Model r Basic Observation Model 2 Basic Observation Model 1 Basic Motion Model 1 Basic Observation Model 1 Basic Motion Model 2 Basic Observation Model r Basic Motion Model s

11. Our Approach Basic Tracker 1 Basic Tracker 2 Basic Tracker rs  Tracker Decomposition  Each tracker takes charge of a certain change in the object.

12. Our Approach  Sampling based Tracker  Markov Chain Monte Carlo (MCMC) Basic Tracker Sampling… Basic Observation Model i Basic Motion Model j Basic Observation Model i Basic Motion Model j

13. Remaining Tasks  How to determine the basic models ?  How to estimate weights of the models ? Basic Observation Model 1 Basic Observation Model r Basic Motion Model 1 Basic Motion Model s Basic Observation Model 1 Basic Observation Model r Basic Motion Model 1 Basic Motion Model s

14. Remaining Tasks  How to determine the basic models ?  Sparse PCA [1]  How to estimate weights of the models ?  Interactive MCMC [2] [2] J. Corander et. al. Parallell interacting MCMC for learning of topologies of graphical models. Data Min. Knowl. Discov., 2005. [1] A. d’Aspremont et. al., A direct formulation for sparse PCA using semidefinite programming. SIAM Review, 49(3), 2007.

15. Design of Basic Observation Models Template set Hue SaturationValueEdge 4 recent frames 1 initial frame Object models A subset of the template set Basic observation models Diffusion distance

16. Object Model  Three conditions  Representativeness  The model has to cover most appearance changes in an object over time.  Compactness  The formation of it should be as compact as possible.  Complementary relation  The relations between models should be complementary.

17. Object Model  Sparse Principal Component Analysis (SPCA) PCASparseness : Gram matrix of the template set : Principal component

18. Object Model Template set

19. Object Model Sparse PC 1 0 00 0000 0 Object model 1 Representativeness Sparse PC 2 Object model 2 0 00 0000 0 0 0 Compactness Sparse PC r Object model r 0 0 0 00 0 0 0 0 Complementary relation

20. Basic Observation Model  Diffusion distance [3] [3] H. Ling and K. Okada. Diffusion distance for histogram comparison. CVPR, 2006. Edge Object model Diffusion distance [3] Hue Saturation Value Edge

21.  Two conditions  Exploitation ( for smooth motions )  Further simulating the seemingly good moves near the local minima  Exploration ( for abrupt motions )  Further simulating moves that have not been explored much ExplorationExploitation Design of Basic Motion Models

22. Weights of Basic Models Parallel ModeInteraction Mode Basic Tracker 1 Basic Tracker 2 Basic Tracker rs Basic Observation Model 1 Basic Motion Model 2 Basic Observation Model 1 Basic Motion Model 1 Basic Observation Model r Basic Motion Model s State

23. Experimental Results  The number of models  Basic observation models : #4  Basic motion models : #2  Basic tracker models: #8(=4X2)  Settings for comparison  Standard MCMC (MC) : 800 samples  Mean Shift (MS)  On-line Appearance Learning (OAL) : 800 samples  Multiple Instance Learning (MIL) OAL : Ross et. al. Incremental learning for robust visual tracking. IJCV 2007. MIL : Babenko et. al. Visual tracking with online multiple instance learning. CVPR 2009.

24. Experimental Results

25. Abrupt Motions and Illumination Changes

26. Illumination Changes and Pose Variations

27. Occlusions and Pose Variations

28. Background Clutters

29. Quantitative Results MCMSOALMILVTD tiger12793651513 david49884237 face194519277 shaking9724195385 soccer47971514121 animal32207233011 skating1111141174857 - Average center location errors in pixels OAL : Ross et. al. Incremental learning for robust visual tracking. IJCV 2007. MIL : Babenko et. al. Visual tracking with online multiple instance learning. CVPR 2009. MS : Comaniciu et. al. Real-time tracking of nonrigid objects using mean shift. CVPR 2000.

30. Summary  Visual tracking decomposition (VTD)  Our method successfully tracks an object whose motion and appearance change at the same time  Since VTD is easy to extend by adding new features or trackers, our method can be more improved.

31. http://cv.snu.ac.kr/paradiso