1-1 Measuring image motion velocity field “local” motion detectors only measure component of motion perpendicular to moving edge “aperture problem” 2D velocity field not determined uniquely from the changing image need additional constraint to compute a unique velocity field

1-2 Assume pure translation or constant velocity VyVy VxVx o Error in initial motion measurements o Velocities not constant locally o Image features with small range of orientations In practice…

1-3 Measuring motion in one dimension VxVx x I(x) V x = velocity in x direction  rightward movement: V x > 0  leftward movement: V x < 0  speed: |V x |  pixels/time step  I/  x  I/  t  I/  x V x = -

1-4 Measurement of motion components in 2-D +x +y (1) gradient of image intensity  I = (  I/  x,  I/  y) (2) time derivative  I/  t (3) velocity along gradient: v   movement in direction of gradient: v  > 0  movement opposite direction of gradient: v  < 0 true motion motion component v  = -  I/  t [(  I/  x) 2 + (  I/  y) 2 ] 1/2  I/  t |  I| = -

1-5 2-D velocities (V x,V y ) consistent with v  (V x, V y ) VxVx VyVy vv All (V x, V y ) such that the component of (V x, V y ) in the direction of the gradient is v  (u x, u y ): unit vector in direction of gradient Use the dot product: (V x, V y )  (u x, u y ) = v  V x u x + V y u y = v 

1-6 In practice… VyVy VxVx V x u x + V y u y = v  Find (V x, V y ) that best fits all motion components together Previously… Find (Vx, Vy) that minimizes: Σ (V x u x + V y u y - v  ) 2 New strategy:

1-7 Smoothness assumption: Compute a velocity field that: (1)is consistent with local measurements of image motion (perpendicular components) (2)has the least amount of variation possible

1-8 Computing the smoothest velocity field (V x i-1, V y i-1 ) (V x i, V y i ) (V x i+1, V y i+1 ) i i+1 i-1 motion components: V x i u x i + V y i u y i = v  i change in velocity: (V x i+1 -V x i, V y i+1 -V y i ) Find (V x i, V y i ) that minimize: Σ (V x i u x i + V y i u y i - v  i ) 2 + [(V x i+1 -V x i ) 2 + (V y i+1 -V y i ) 2 ]

1-9 Ambiguity of 3D recovery birds’ eye views We need additional constraint to recover 3D structure uniquely “rigidity constraint”

1-10 Image projections Z X perspective projection image plane Z X orthographic projection image plane (X, Y, Z)  (X, Y) (X, Y, Z)  (X/Z, Y/Z)  only relative depth  requires object rotation  only scaled depth  requires translation of observer relative to scene

1-11 Incremental Rigidity Scheme x y image (x 1 y 1 z 1 ) (x 2 y 2 z 2 ) (x 3 y 3 z 3 ) (x 1 ' y 1 ' ??) (x 2 ' y 2 ' ??) (x 3 ' y 3 ' ??) depth: Z initially, Z=0 at all points Find new 3D model that maximizes rigidity Compute new Z values that minimize change in 3D structure

1-12 Bird’s eye view: depth x image current model new image Find new Z i that minimize Σ (L ij – l ij ) 2 /L ij 3 L ij l ij Z

1-13 Observer motion problem, revisited From image motion, compute:  Observer translation (T x T y T z )  Observer rotation (R x R y R z )  Depth at every location Z(x,y) Observer undergoes both translation + rotation

1-14 Equations of observer motion

1-15 Longuet-Higgins & Prazdny  Along a depth discontinuity, velocity differences depend only on observer translation  Velocity differences point to the focus of expansion