1. Optical flow , A tutorial of the paper:
G. Farneback , “Two-frame Motion Estimation based on Polynomial Expansion”, 13th Scandinavian Conference, SCIA 2003 Halmstad, Sweden, June 29 – July 2, 2003.
KH Wong
Optical Flow v.5a (beta)

2. Introduction
Optical flow : Based on two frames sampled at different slight different time, find the change of position of each pixel in the first image to the second image.
Reference:
G. Farneback , “Two-frame Motion Estimation based on Polynomial Expansion”, 13th Scandinavian Conference, SCIA 2003 Halmstad, Sweden, June 29 – July 2, 2003.
Optical Flow v.5a

3. Problem definition and method
Input: Two images
Output : motion vectors (v) and xx( c )? of each pixel in the first image.
The Farneback method [1] is fast and linear.
Method:
Step1: Approximate the image using a polynomial
Step2: Use the model of polynomial to approximate the motion for each pixel.
Optical Flow v.5a

4. Example
Image1
Image2
Optical flow result , vectors showing the direction and intensity of flow of each pixel.
Plotting all vectors is too messy, only plot one vector in a window of 10x10 pixels.
Optical Flow v.5a

5. Ideal and method x=[x,y]’ x1 (just show one dimension of x)
We use a polynomial (with parameters A,b,c) to model the intensity change of an image (small window). Assume x is dimension2 here, but it can be larger.
In image f1 (x) is taken at t , and f2 (x) at t+dt
f1 (x)=xTA1x+b1x+c1
x is 2x1, A1 is 2x2, b1 is 2x1, c1 is 1x1
Assume f2 is f1 with a global displacement by d=[d1,d2]’ (shape no change, just displacement)
f2 (x)=xTA2x+b2x+c2
By comparing f1 and f2, we get
A2=A1, b2=b1-2A1d
C2=dTA1d-bT1d+c1
We observe that (when A1 is non-singular , i.e. invertible)
2A1d=-(b2-b1)
d=-(1/2)(A1)-1(b2-b1)
So if we can measure f1 and f2, can may find d=[d1,d2]’. d is the 2-D displacement and hence the flow of the image pixel.
Intensity of image
f1(x))
f2(x) d x
Image 1, at t
Image 2, at t=dt
Use a polynomial (with parameters A,b,c) to model the intensity change of an image (assume a small window).
Optical Flow v.5a

6. Step: 1a Model an image using a polynomial
For a polynomial with parameters A,b,c :
x= is 2x1
A is 2x2
b is 2x1
C is 1x1
d is 2x1
Optical Flow v.5a

7. Modeling images using polynomials
Read the Ph.D thesis
Orientation and Velocity, Estimation , Ph.D (page22 the example)
and also the code make_Abc_fast.m in Code,
Optical Flow v.5a

8. step1b
Assume all pixels in a small window I() behave the same. If we can measure A1, b1, c1 from f1 for each pixel in I() ; and A2, A2 c2 from f2 we can calculate d.
Pointwise is too noisy
Sum over an window of I()
Optical Flow v.5a

9. Step2 : find displacement d=[dx,dy]’ (=optical flow)
We assume d is not a constant over the window I(), but a function according to its position x,y based on affine transformation.
Affine model of d
S is 2x8
ST is 8x2
p is 8x1
d is 2x1
wi= is 1x1 0
A , p ,b can be measured.
Put them in (19) and minimize (19), or set it to [0,0]’. We can solve for p using (20), if p is found d can be found by (15)
Optical Flow v.5a

10. Some modification
Optical Flow v.5a

11. Result Image: yos2 Image: yos8
Optical flow (averaged) using a sequence of 15 pictures. Plotting all vectors is too messy, only plot one vector in a window of 10x10 pixels. Using the code
Optical Flow v.5a
Image: yos16

12. Summary
Studied a linear method to find the optical flow based on two frames of image.
Optical Flow v.5a

13. References
G. Farneback , “Two-frame Motion Estimation based on Polynomial Expansion”, 13th Scandinavian Conference, SCIA 2003 Halmstad, Sweden, June 29 – July 2, 2003.
Publications of G. Farneback ,
G. Farneback , Spatial Domain Methods for
Orientation and Velocity, Estimation , Ph.D
Code,
Optical Flow v.5a