COMPUTER VISION Tam Lam

Tracking: General idea Initialize model in the first frame Given model estimate for frame t-1: Predict for frame t Use dynamics model of how the model changes Correct for frame t Use observations from the image predict correct

Motion Tracking Deciding on the structure of the model points curves pictorial structures

Tracking issues Deciding on the structure of the model Initialization Specifying the dynamics model Specifying the observation model Data association problem: which measurements tell us about the object(s) being tracked?

Data association Simple strategy: only pay attention to the measurement that is “closest” to the prediction

Data association Simple strategy: only pay attention to the measurement that is “closest” to the prediction Doesn’t always work…

Data association Simple strategy: only pay attention to the measurement that is “closest” to the prediction More sophisticated strategy: keep track of multiple state/observation hypotheses This is a general problem in computer vision, there is no easy solution

Tracking issues Deciding on the structure of the model Initialization Specifying the dynamics model Specifying the observation model Data association problem Prediction vs. correction If the dynamics model is too strong, will end up ignoring the data If the observation model is too strong, tracking is reduced to repeated detection Drift

DRIFT the model is trying to predict, change over time in unforeseen ways the predictions become less accurate as time passes  problem

Drift

Tracking with person-specific appearance models Tracker pictorial structure

Tracking with person-specific appearance models Structure and dynamics are generic, appearance is person-specific Trying to acquire an appearance model “on the fly” can lead to drift Instead, can use the whole sequence to initialize the appearance model and then keep it fixed while tracking Given strong structure and appearance models, tracking can essentially be done by repeated detection (with some smoothing)

Bottom-up initialization: Clustering

Top-down initialization: Exploit “easy” poses

Tracking by model detection

QUESTIONS????