Feature tracking

Identify features and track them over video –Small difference between frames –potential large difference overall Standard approach: KLT (Kanade-Lukas-Tomasi)

Brightness constancy assumption Intermezzo: optical flow (small motion) 1D example possibility for iterative refinement

Brightness constancy assumption Intermezzo: optical flow (small motion) 2D example (2 unknowns) (1 constraint) ? isophote I(t)=I isophote I(t+1)=I the “aperture” problem

Intermezzo: optical flow How to deal with aperture problem? Assume neighbors have same displacement (3 constraints if color gradients are different)

Lucas-Kanade Assume neighbors have same displacement least-squares:

Compute translation assuming it is small Alternative derivation differentiate: Affine is also possible, but a bit harder (6x6 in stead of 2x2)