1. Dept. of ECE 1 Feature-based Object Tracking Dr. Dapeng Oliver Wu Joint Work with Bing Han, William Roberts, and Jian Li Department of Electrical and Computer Engineering University of Florida

2. Dept. of ECE 2 Outline  What is object tracking?  How to track an object?  KLT feature-based tracking algorithm  Our feature-based tracking algorithm  Experimental results  Conclusions

3. Dept. of ECE 3 What is Object Tracking?  Object tracking: Track an object over a sequence of images

4. Dept. of ECE 4 Why Object/Vehicle Tracking is Challenging?  Occlusion or partial occlusion  Re-appearance  A vehicle makes a left/right turn  A vehicle passes another vehicle in the same direction  Tracking of a small imaged object (consisting of 4 pixels)  Shadow effect: use local histogram-based equalization  Cloud effect  Tracking of many vehicles (e.g., thousands of vehicles in a city)  Clutters  Parallax problems (caused by high-rise buildings)  Multi-camera based tracking (e.g., urban surveillance networks)

5. Dept. of ECE 5 How to Track an Object?  Kanade-Lucas-Tomasi (KLT) feature-based tracking algorithm  Han, Roberts, Wu, Li (HRWL) feature-based tracking algorithm  ……

6. Dept. of ECE 6 KLT Feature-based Tracker 1. Identification of feature points 2. Determination of an optimization criterion for feature correspondence 3. Computational method to solve the optimization problem Web: http://www.ces.clemson.edu/~stb/klt/ References [1] Bruce D. Lucas and Takeo Kanade. An Iterative Image Registration Technique with an Application to Stereo Vision. International Joint Conference on Artificial Intelligence, pages 674-679, 1981. [2] Carlo Tomasi and Takeo Kanade. Detection and Tracking of Point Features. Carnegie Mellon University Technical Report CMU-CS-91-132, April 1991. [3] Jianbo Shi and Carlo Tomasi. Good Features to Track. IEEE Conference on Computer Vision and Pattern Recognition, pages 593-600, 1994. [4] Stan Birchfield. Derivation of Kanade-Lucas-Tomasi Tracking Equation. Unpublished.

7. Dept. of ECE 7 Identification of Feature Points  Harris criterion:  For low threshold For high threshold

8. Dept. of ECE 8 Feature Correspondence  Determination of an optimization criterion for feature correspondence (find a similarity measure) l lCorrelation lInformation distance: e.g., mutual information

9. Dept. of ECE 9 How to Solve the Optimization problem?  Key difficulties: lDiscrete optimization problem lGradient methods not directly applicable  Exhaustive search: incurs exponential complexity  Lucas-Kanade computational method: lNewton-Raphson type search

10. Dept. of ECE 10 More about KLT Tracker  Hierarchical search  Work for any dimensional vector  Can address affine motion (translation, rotation, scaling)  Can address intensity adjustment

11. Dept. of ECE 11 Hierarchical Search

12. Dept. of ECE 12 Limitation of KLT Tracker  Limitation: KLT tracker does not guarantee the corresponding point in the next frame is a feature point.  Why? lBecause KLT tracker only uses the Harris criterion for the first frame but not for other frames.  What problem it may cause? lKLT may not handle occlusion well.  How to improve it? lEvaluate the quality of the corresponding point using the Harris criterion lRefer to: B. Han, W. Roberts, D. Wu, J. Li, ``Robust Feature-based Object Tracking,'' SPIE Defense & Security Symposium 2007, Orlando, FL, USA, April 9–13, 2007.

13. Dept. of ECE 13 Our Tracking approach  Feature tracking: SSD criteria applied to find the feature point whose window minimizes the following energy function:  Feature quality: If the quality of a tracked feature point decreases below a chosen threshold, that point is removed from consideration. To compensate, new features are identified in the same window. The feature point with the minimum SSD criteria is retained if its SSD is below a certain threshold.

14. Dept. of ECE 14 Tracking Diagram Feature point in current frame, new frame Find point that min. SSD criteria Meets threshold? Retain feature Identify new feature points Select feature that min. SSD Meets threshold? Feature lost Retain new feature N N Y Y

15. Dept. of ECE 15 Experimental Results  Biker image sequence l50 frame sequence of bikers. lVideo contains global and local motions. lObjects undergo scaling, translation, and rotation.

16. Dept. of ECE 16 Experimental Results (2)  Biker image sequence (Cont’d) l1 st frame. l258 features selected. l50 th frame. l241 features tracked.

17. Dept. of ECE 17 Experimental Results (3)  Biker image sequence (Cont’d) lBlue: our approach. lRed: KLT algorithm.

18. Dept. of ECE 18 Conclusions  Number of successfully tracked features increased by over 10% versus Tomasi-Kanade’s approach.  Computationally inexpensive and robust against various types of object motion.  For pure translational object motion, the combined algorithm does not offer improved performance to Tomasi-Kanade’s method.

19. Dept. of ECE 19 Future Research Direction  Design tracking schemes, which are proven to converge. lKLT algorithm does not guarantee convergence; no proof of convergence lIt is important to design convergence-guaranteed tracking algorithms. lWe will systematically study this problem.

20. Dept. of ECE 20 THANK YOU!