2. Optical Flow

3. Brightness Constancy The Aperture problem Regularization Lucas-Kanade Coarse-to-fine Parametric motion models Direct depth SSD tracking Robust flow

4. Optical Flow: Where do pixels move to?

5. Motion is a basic cue Motion can be the only cue for segmentation

6. Motion is a basic cue Motion can be the only cue for segmentation

7. Motion is a basic cue Even impoverished motion data can elicit a strong percept

8. Applications tracking structure from motion motion segmentation stabilization compression mosaicing …

9. Optical Flow Brightness Constancy The Aperture problem Regularization Lucas-Kanade Coarse-to-fine Parametric motion models Direct depth SSD tracking Robust flow

10. Definition of optical flow OPTICAL FLOW = apparent motion of brightness patterns Ideally, the optical flow is the projection of the three-dimensional velocity vectors on the image 

11. Caution required ! Two examples : 1. Uniform, rotating sphere  O.F. = 0 2. No motion, but changing lighting  O.F.  0 

12. Mathematical formulation I (x,y,t) = brightness at (x,y) at time t Optical flow constraint equation :  Brightness constancy assumption:

13. Optical Flow Brightness Constancy The Aperture problem Regularization Lucas-Kanade Coarse-to-fine Parametric motion models Direct depth SSD tracking Robust flow

14. The aperture problem 1 equation in 2 unknowns 

15. Aperture Problem: Animation

16. The Aperture Problem 0

18. What is Optical Flow, anyway? Estimate of observed projected motion field Not always well defined! Compare: Motion Field (or Scene Flow) projection of 3-D motion field Normal Flow observed tangent motion Optical Flow apparent motion of the brightness pattern (hopefully equal to motion field) Consider Barber pole illusion

20. Optical Flow Brightness Constancy The Aperture problem Regularization Lucas-Kanade Coarse-to-fine Parametric motion models Direct depth SSD tracking Robust flow

21. Horn & Schunck algorithm Additional smoothness constraint : besides OF constraint equation term minimize e s + e c 

22. The calculus of variations look for functions that extremize functionals and 

23. Calculus of variations Suppose 1. f(x) is a solution 2.  (x) is a test function with  (x 1 )= 0 and  (x 2 ) = 0 for the optimum : 

24. Calculus of variations Using integration by parts : Therefore regardless of  (x), then Euler-Lagrange equation 

25. Calculus of variations Generalizations 1. Simultaneous Euler-Lagrange equations : 2. 2 independent variables x and y 

26. Hence Now by Gauss’s integral theorem, so that = 0 Calculus of variations 

27. Given the boundary conditions, Consequently, for all test functions . is the Euler-Lagrange equation 

28. Horn & Schunck The Euler-Lagrange equations : In our case, so the Euler-Lagrange equations are is the Laplacian operator 

29. Horn & Schunck Remarks : 1. Coupled PDEs solved using iterative methods and finite differences 2. More than two frames allow a better estimation of I t 3. Information spreads from corner-type patterns 

30. 

31. Horn & Schunck, remarks 1. Errors at boundaries 2. Example of regularization (selection principle for the solution of ill-posed problems) 

32. Results of an enhanced system

33. Optical Flow Brightness Constancy The Aperture problem Regularization Lucas-Kanade Coarse-to-fine Parametric motion models Direct depth SSD tracking Robust flow

37. Optical Flow Brightness Constancy The Aperture problem Regularization Lucas-Kanade Coarse-to-fine Parametric motion models Direct depth SSD tracking Robust flow

41. Optical Flow Brightness Constancy The Aperture problem Regularization Lucas-Kanade Coarse-to-fine Parametric motion models Direct depth SSD tracking Robust flow

70. Optical Flow Brightness Constancy The Aperture problem Regularization Lucas-Kanade Coarse-to-fine Parametric motion models Direct depth SSD tracking Robust flow

74. Optical Flow Brightness Constancy The Aperture problem Regularization Lucas-Kanade Coarse-to-fine Parametric motion models Direct depth SSD tracking Robust flow

80. Optical Flow Brightness Constancy The Aperture problem Regularization Lucas-Kanade Coarse-to-fine Parametric motion models Direct depth SSD tracking Robust flow

87. Textured Motion http://www.cs.ucla.edu/~wangyz/Research/w yzphdresearch.htm http://www.cs.ucla.edu/~wangyz/Research/w yzphdresearch.htm Natural scenes contain rich stochastic motion patterns which are characterized by the movement of a large amount of particle and wave elements, such as falling snow, water waves, dancing grass, etc. We call these motion patterns "textured motion".