1. Motion from normal flow

2. Optical flow difficulties The aperture problemDepth discontinuities

3. Translational Normal Flow In the case of translation each normal flow vector constrains the location of the FOE to a half-plane. Intersection of half-planes provides FOE.

4. Egoestimation from normal flow Idea: choose particular directions: patterns defined on the sign of normal flow along particular orientation fields positive depth constraint 2 classes of orientation fields: copoint vectors and coaxis vectors

5. Optical flow and normal flow

7. Coaxis vectors with respect to axis (A,B,C) P(Af/C, Bf/C

8. Coaxis vectors with respect to axis (A,B,C)

9. Coaxis vectors

10. Translational coaxis vectors

11. h passes through FOE and (Af/C, Bf/C), defined by 2 parameters

12. Rotational coaxis vectors

14. g passes through AOR and (Af/C, Bf/C), defined by 1 parameter

15. Combine translation and rotation Positive + positive  positive Negative + negative  negative Positive + negative  don’t know (depends on structure)

16. Coaxis pattern translational rotational combined

17.  -vectors: Translation

18.  -vectors: Rotation

19.  -vectors: Translation and Rotation

20. alphabetagamma Three coaxis vector fields

21. Copoint vectors

22. copoint vectors

23. Copoint vectors defined by point (r,s)

24. Translational copoint vectors

26. k passes through FOE and (r,s) defined by 1 parameter

27. Rotational copoint vectors

28. l passes through AOR and (r,s), is defined by 2 parameters

29. translational componentrotational component

30. (a)(b)(c) Three coaxis vector fields

31. a,b,c : positive and negative  vectors c,d,e: Fitting of  patterns g: Separation of (  coaxis pattern h: Separation of (x 0, y 0 ) copoint pattern

34. Optical illusion

36. What is the Problem? Flow can be accurately estimated in an image patch corresponding to a smooth scene patch, But erroneous flow estimates are obtained for image patches corresponding to scene patches containing discontinuities Image Flow 3D Motion Scene structure Discontinuities

37. Depth variability constraint Errors in motion estimates lead to distortion of the scene estimates. The distortion is such that the correct motion gives the “smoothest” (least varying) scene structure.

38. Scene depth can be estimated from normal flow measurements: Depth estimation

39. Visual Space Distortion Wrong 3D motion gives rise to a rugged (unsmooth) depth function (surface). The correct 3D motion leads to the “smoothest” estimated depth.

40. Inverse depth estimates correct motion incorrect motion

41. The error function A normal flow measurement: For an estimate The error function to be minimized:

42. The error function Estimated normal flow The error function to be minimized: Global parameters: Local parameter: locally planar patches:

43. Error function evaluation Given a translation candidate, each local depth can be computed as a linear function of the rotation. We obtain a second order function of the rotation; its minimization provides both the rotation and the value of the error function.

44. Is derived from image gradients only Brightness consistency: Flow: Planar patch:

45. Handling depth discontinuities Given a candidate motion, the scene depth can be estimated and further processed to find depth discontinuities. Split a region if it corresponds to two depth values separated in space.

46. The algorithm Compute spatio-temporal image derivatives and normal flow. Find the direction of translation that minimizes the depth-variability criterion. –Hierarchical search of the 2D space. –Iterative minimization. –Utilize continuity of the solution in time; usually the motion changes slowly over time.

47. Divide image into small patches Search in the 2D space of translations For each candidate 3D motion, using normal flow measurements in each patch, compute depth of the scene. For each image patch NO Depth variation small? Distinguish between two cases wrong 3D motion existence of a discontinuity at the patch Use the error Split the patch and repeat the process Use the error YES The Algorithm

49. 3D reconstruction Comparison of the original sequence and the re-projection of the 3D reconstruction.

50. 3D model construction