2. CS 6825: Motion Part 2 – Optical Flow

3. Recall: Optical Flow is an approximation of the 2D motion field. is an approximation of the 2D motion field. Motion in the world usually occurs in 3D, but, we have a 2D image sensor. So, we see the results as movement across the 2D image plane. Motion in the world usually occurs in 3D, but, we have a 2D image sensor. So, we see the results as movement across the 2D image plane. Hence we are seeing the projection of the 3D moving points onto the image plane. Hence we are seeing the projection of the 3D moving points onto the image plane.

4. Motion Field and Optical Flow Field Motion field: projection of 3D motion vectors on image plane Motion field: projection of 3D motion vectors on image plane Optical flow field: apparent motion of brightness patterns Optical flow field: apparent motion of brightness patterns We equate motion field with optical flow field We equate motion field with optical flow field

5. Brightness Constancy Equation Let P be a moving point in 3D: Let P be a moving point in 3D: At time t, P has coords (X(t),Y(t),Z(t))At time t, P has coords (X(t),Y(t),Z(t)) Let p=(x(t),y(t)) be the coords. of its image at time t.Let p=(x(t),y(t)) be the coords. of its image at time t. Let E(x(t),y(t),t) be the brightness at p at time t.Let E(x(t),y(t),t) be the brightness at p at time t. Brightness Constancy Assumption: Brightness Constancy Assumption: As P moves over time, E(x(t),y(t),t) remains constant.As P moves over time, E(x(t),y(t),t) remains constant.

6. Brightness Constancy Equation Taking derivative wrt time:

7. Brightness Constancy Equation Let (Frame spatial gradient) (optical flow) and (derivative across frames)

8. Brightness Constancy Equation Becomes: vxvxvxvx vyvyvyvy r Er Er Er E The Optical Flow is CONSTRAINED to be on a line ! -E t /| r E| Can calculate these: Different techniques to figure these out. E is the spatial change in brightness in image i E t is the difference in the brightness at (x,y) between image i and image i+1 We want to calculate v = [dx/dt dy/dt]