Feature and object tracking algorithms for video tracking Student: Oren Shevach Instructor: Arie nakhmani

Overview Given a video sequence, the purpose is to track the objects in the video and overcome occlusions. Camera can be stationary or moving. Movement between frames: ▫Translations: movement in the x-y axis. ▫Affine transformations: Rotations, scaling.

Overview Project goal: study and understand 2 algorithms for tracking ▫KLT- Feature Tracking ▫GLOMO-Object Learning

Kanade Lucas Tomasi-KLT Basic feature tracking algorithm. Good feature: Consider small rectangular windows all over the image, Good feature is a window that can be tracked easily in a sequence of images. Feature movement :

Kanade Lucas Tomasi-KLT Tracking Goal: Find and parameters that Minimize the dissimilarity between current and next image in the sequence. To Find minimum, we set first derivative of dissimilarity to zero. Taylor extension for next Image, assuming small movements: -Gradients vector

Kanade Lucas Tomasi-KLT We receive the equation to solve: Consists of gradient and pixels location Movement parameters Error vector

Kanade Lucas Tomasi-KLT Solution is iterative: Initialization: Iteration step: continue until convergence Calculate T and a matrixes Receive parameters from z Update

KLT-Results

GLOMO- Greedy Unsupervised Learning Of Multiple Objects Tracking all objects in the sequence and the background Algorithm output (for one object): The object Object mask Background transformationObject Transformation Background Object Variance Background Variance

Given a sequence of frames, in each frame object goes through transformations and might be noisy Assuming there are J possible transformations. Tracking the object parameters using EM,an iterative procedure to compute the Maximum Likelihood estimate in the presence of missing or hidden data Object density: Tracking objects from images

EM Example:(for one object with static background) ▫Expectation: given current parameters find Giving weight to each transformation possible ▫Maximization: Update the parameters Tracking objects from images

Background,objects and all parameters are found together in EM iterations. For more than one object and moving background the complexity is too high. The probability model doesn’t handle object occlusion Tracking objects from images- problems A more efficient approach is needed!

First finding the background and then every object separately. New probability density model: Each pixel is part of the object/background or uniform for other. Tracking will be in relevant pixels only for speeding up the tracking. The Algorithm-GLOMO

Algorithm Steps: ▫User defines the number of objects to find and number of EM iterations. ▫Find the background and its transformations assuming all masks are zero. ▫Define vector Z which contains relevant pixels and initialize: ▫For each object, find object parameters and transformations by applying EM for the object. The Algorithm-GLOMO

▫After tracking the object, Update Vector Z with the object pixels. The Algorithm-GLOMO

GLOMO-Results Example: Reconstruction: Original OrderingNew Ordering

GLOMO-Results Example : moving background

GLOMO-Results Example :change number of frames 20 frames 60 frames

GLOMO-Results Example :change number of frames 110 frames

GLOMO-Results Example :change number of EM iterations for 20 frames 70 iterations 300 Iterations

Conclusions Both algorithms worked well on high quality pictures with large and defined objects. On low quality pictures with less defined objects, GLOMO didn’t recognize the objects very well and KLT lost all the features very quickly. Both Algorithms handled well with moving camera and changing background. KLT doesn’t recover from occlusions, while GLOMO handles them very well.

Conclusions GLOMO doesn’t work well on 20 frames but works well on more than 100 frames can’t work on real time systems. while KLT works on real time systems.

Kanade Lucas Tomasi-KLT Finding good features: For all the possible windows in the current image compute the Eigen values for the gradients matrix Z: Find the maximum and minimum Eigen values and Set threshold as the middle point between them For every window check that Z Eigen values maintains:

EM Example:(for one object with static background) ▫Expectation: find ▫Maximization: Update the parameters Tracking objects from images -The pixel is part of the object