Real-time foreground object tracking with moving camera P Martin Chang

Previous work Thompson, W.B. and Pong, T.C. Detecting moving objects. International Journal of Computer Vision, 4(1): (January 1990). Stationary Camera K Daniilidis, C Krauss, M Hansen, G Sommer. Real-Time Tracking of Moving Objects with an Active Camera. Real-Time Imaging, Two degrees of freedom of a camera platform E Hayman, JO Eklundh - Procs. Statistical Background Subtraction for a Mobile Observer. IEEE Intl. Conf. on Computer Vision. Moving foreground object static background Mobile observer

Step & idea 1. Find good feature to track 2. Track features 3. Classify foreground and background features 4. Use foreground features to detect foreground object region

Step 1: Find good feature to track Finding good feature to track Shi and Tomasi ‘ s method

Step 2: Track features Optical flow Disadvantage: time consuming Dynamic adjust the parameter of Optical flow

Step 3: Classify foreground and background features Classify feature points by Optical flow Direction of optical flow Length of optical flow More attributes Temporal information Texture of neighbor image patch MTF of neighbor image patch

How to identify foreground features? Background Object Case 1: The camera rotates The background image moves more

How to identify foreground features? Case 2: The background moves The background image moves more Background object

How to identify foreground features? Background object Case 3: The object moves The foreground image moves more

Simple demo Only using optical flow Direction of optical flow Length of optical flow Identify foreground features by Calculate the variance of each group The result is not good Without Temporal information Neighbor image patch information

Use foreground features to detect object region Plenty of reliable feature points cause good object segmentation

Expected Result An effective foreground/background feature point selection algorithm Effective foreground/background segmentation algorithm Preprocess of object recognition system