1. Rotem Kupfer & Alon Gat Under supervision of Dr. Ohad Ben Shahar & Mr. Yair Adato Department of Computer Science Ben-Gurion University Undergraduate Project Presentation Day May 2011

2. Given: Two or more frames of an image sequence. Wanted: Displacement field between two consecutive frames  optical flow

3. Smoothness term Data term brightness constancy Data term - Penalizes deviations from constancy assumptions. Smoothness term - Penalizes dev. from smoothness of the solution. Regularization parameter α - Determines the degree of smoothness. Output – the optical flow!

7.  Minimizing using Euler-Lagrange doesn’t guarantee global minimum, so a Pyramid Method is enforced, which causes loss of vital information in some type of problems.  Main Goal: trying to find a method that wont need Pyramids.

8.  Instead of solving E-L equations (which need to be linearized) we wrote the Functional as a set of n*m equations and minimized it with regular minimization methods (Gradient decent, Quasi Newton, and others).

9. Where P are the Image derivatives, and u, v is the optical flow components in the vertical and horizontal directions. Mathematica Symbolic Toolbox for MATLAB--Version 2.0 (http://library.wolfram.com/infocenter/MathSource/5344/)http://library.wolfram.com/infocenter/MathSource/5344/

11.  Quasi Newton Iteration Problem: Calculation time of the inverse of non sparse matrix is long. O(n^3) -> resulting ((n*m)^3)  Gradient Decent. Advantage: Doesn’t require calculation of the Hessian matrix.

12.  Translating the symbolic expression to numeric scheme: for i=2:imageSizeN-1 for j=2:imageSizeM-1 gradC(i,j)=gradC(i,j)+alpha*(2*(-c(i-1,j)+c(i,j))+2*(-c(i,j-1)+c(i,j))-2*alpha*(-c(i,j)+c(i,j+1)) - 2*alpha*(- c(i,j)+c(i+1,j))+4*exp((-1+c(i,j)^2+s(i,j)^2)^2)*c(i,j)*(-1+c(i,j)^2+s(i,j)^2) +2*(Ix(i,j)*m(i,j)+Ixz(i,j)*m(i,j)+2*Ixx(i,j)*c(i,j)*m(i,j)^2+Ixy(i,j)*m(i,j)^2*s(i,j))*(Iz(i,j)+Izz(i,j)+Ix(i,j)*c(i,j)*m (i,j)+Ixz(i,j)*c(i,j)*m(i,j)+Ixx(i,j)*c(i,j)^2*m(i,j)^2+Iy(i,j)*m(i,j)*s(i,j) +Iyz(i,j)*m(i,j)*s(i,j)+Ixy(i,j)*c(i,j)*m(i,j)^2*s(i,j)+Iyy(i,j)*m(i,j)^2*s(i,j)^2)); gradS(i,j)=gradS(i,j)+4*exp((-1+c(i,j)^2+s(i,j)^2)^2)*s(i,j)*(- 1+c(i,j)^2+s(i,j)^2)+2*(Iy(i,j)*m(i,j)+Iyz(i,j)*m(i,j)+Ixy(i,j)*c(i,j)*m(i,j)^2+2*Iyy(i,j)*m(i,j)^2*s(i,j))*(Iz(i,j)+Izz( i,j) +Ix(i,j)*c(i,j)*m(i,j)+Ixz(i,j)*c(i,j)*m(i,j)+Ixx(i,j)*c(i,j)^2*m(i,j)^2+Iy(i,j)*m(i,j)*s(i,j)+Iyz(i,j)*m(i,j)*s(i,j)+Ixy(i, j)*c(i,j)*m(i,j)^2*s(i,j)+Iyy(i,j)*m(i,j)^2*s(i,j)^2) +alpha*(2*(-s(i-1,j)+s(i,j))+2*(-s(i,j-1)+s(i,j))-2*alpha*(- s(i,j)+s(i,j+1))-2*alpha*(-s(i,j)+s(i+1,j))); gradM(i,j)=gradM(i,j)+beta*(2*(-m(i-1,j)+m(i,j))+2*(-m(i,j-1)+m(i,j)))-2*beta*(-m(i,j)+m(i,j+1))-2*beta*(- m(i,j)+m(i+1,j))+2*(Ix(i,j)*c(i,j)+Ixz(i,j)*c(i,j) +2*(Ixx(i,j)*c(i,j)^2*m(i,j)+Iy(i,j)*s(i,j)+Iyz(i,j)*s(i,j)+2*Ixy(i,j)*c(i,j)*m(i,j)*s(i,j)+2*Iyy(i,j)*m(i,j)*s(i,j)^2)*(Iz (i,j)+Izz(i,j)+Ix(i,j)*c(i,j)*m(i,j)+Ixz(i,j)*c(i,j)*m(i,j) +Ixx(i,j)*c(i,j)^2*m(i,j)^2+Iy(i,j)*m(i,j)*s(i,j)+Iyz(i,j)*m(i,j)*s(i,j)+Ixy(i,j)*c(i,j)*m(i,j)^2*s(i,j)+Iyy(i,j)*m(i,j)^2* s(i,j)^2)); end

13.  Calculation time of 60-360 minutes! While high end algorithms run at around 20 secs. (CPU time complexity)  There are couple of parameters (alpha, number of iteration, with or without filters (texture decomposition and median filter)) that are needed to be set each run, and they are problem specific. (OUR time complexity) Resulting Big HEADACHE !

14.  So why use only one core when we have two or more?  The OpenMP Application Program Interface supports multi- platform shared-memory parallel programming in C/C++ on all architectures, OpenMP is a portable, scalable model that gives shared-memory parallel programmers a simple and flexible interface for developing parallel applications for platforms ranging from the desktop to the supercomputer. http://openmp.org/ MATLAB MEX FILES (In C++) OPENMP

15.  Condor is a specialized workload management system for compute-intensive jobs.  So instead of specifically trying to find the ‘best’ parameters, we just ‘tried them all’.  We used condor in order run multiple (100 and more) experiments in parallel. http://www.cs.wisc.edu/condor/ It’s installed only on EE Labs…  It’s easy to use, very practical, it’s FREE and CONDOR VIEW

16. Starting from boundaries

18. 6 pyramids, 1000 iterations 840secs NO pyramids, 1000 iterations 283 sec ~ 3mins!

19.  We have successfully ran experiments without the need for Pyramids (in “fair” play).  Gradient Decent uses normalized steps - very small step size, resulting |u| |v| that are small.  Using problem specific tools (like Condor and OPENMP) is HIGHLY recommended. (saves time, and time is money)

20. The Ground Truth.