1. Effective Optical Flow Estimation
Jan Kamenický

2. Motivation

3. Usage Motion detection Object segmentation
Video encoding (compression)
Stereo disparity measurement

4. Optical flow equation Color constancy Taylor series

5. Estimating optical flow
Basic equation
Local methods
Lucas & Kanade
flow field is locally constant (or affine)
least squares minimization
cannot handle interior parts of objects

6. Estimating optical flow
Basic equation
Global methods
Horn & Schunk
more sensitive to noise
data term
smoothing term

7. Problems Data term Smoothness term
non-homogenous surface (shading, reflections)
non-flat scene / non-uniform lighting
spatial discontinuities
Smoothness term
discontinuities (moving objects boundaries)

8. Estimating partial derivatives
Discrete approximation by differences
forward, backward – not exact
use 2x2x2 cube in (x,y,t) space
compute the difference as an average of 4 adjacent first order differences
use larger support
e.g. [1, -8, 0, 8, -1]/12
j+1 j
k+1 k i
i+1

9. Data term
L2 norm
L1 norm
Many modifications
generalized Charbonnier

10. Regularization term Enforces smooth flow field
Similar norms can be used
L2, L1 (total variation), …
Other possibilities
Laplacian instead of gradient

11. Dealing with larger displacements
Smoothing (blurring)
usually Gaussian kernel
decreases flow field accuracy
Pyramidal approach
compute flow on down-sampled images
up-sample the flow to next level
compute the warping (using the optical flow)
repeat

12. More optimizations Graduated non-convexity Median filtering (5x5)
iteratively move from convex energy function to the more robust non-convex form
Median filtering (5x5)
weighted modification
More warping steps on one pyramid level

13. OF methods comparison Optical flow estimation benchmark
Average end-point error

14. References Main described method
D. Sun, S. Roth, M. J. Black: Secrets of Optical Flow Estimation and Their Principles, CVPR 2010