2. Lecture 11 Stereo Reconstruction I Lecture 11 Stereo Reconstruction I Mata kuliah: T0283 - Computer Vision Tahun: 2010

3. January 20, 2010T0283 - Computer Vision3 Learning Objectives After carefullylistening this lecture, students will be able to do the following : After carefully listening this lecture, students will be able to do the following : demonstrate 3D Stereo Computation by solving point- correspondence problems and fundamental matrix demonstrate 3D Stereo Computation by solving point- correspondence problems and fundamental matrix Calculate object-depth information using disparity and triangulation techniques. Calculate object-depth information using disparity and triangulation techniques.

4. January 20, 2010T0283 - Computer Vision4 Stereo Reconstruction knowncameraviewpoints Shape (3D) from two (or more) images

5. January 20, 2010T0283 - Computer Vision5 Examples images shape surface reflectance

6. January 20, 2010T0283 - Computer Vision6 Scenarios The two images can arise from A stereo rig consisting of two cameras A stereo rig consisting of two cameras the two images are acquired simultaneously OR A single moving camera (static scene) A single moving camera (static scene) the two images are acquired sequentially The two scenarios are geometrically equivalent

7. January 20, 2010T0283 - Computer Vision7 Stereo head Camera on a mobile vehicle

8. January 20, 2010T0283 - Computer Vision8 Imaging geometry central projection central projection camera centre, image point and scene point are collinear camera centre, image point and scene point are collinear an image point back projects to a ray in 3-space an image point back projects to a ray in 3-space depth of the scene point is unknown depth of the scene point is unknown camera centre image plane image point scene point ?

9. January 20, 2010T0283 - Computer Vision9 The objective Given two images of a scene acquired by known cameras, compute the 3D position of the scene (structure recovery) Basic principle: triangulate from corresponding image points Determine 3D point at intersection of two back-projected rays

10. January 20, 2010T0283 - Computer Vision10 Corresponding points are images of the same scene point Triangulation C C / The back-projected points generate rays which intersect at the 3D scene point

11. January 20, 2010T0283 - Computer Vision11 An algorithm for stereo reconstruction 1.For each point in the first image determine the corresponding point in the second image (this is a search problem) 2.For each pair of matched points determine the 3D point by triangulation (this is an estimation problem)

12. January 20, 2010T0283 - Computer Vision12 The correspondence problem Given a point x in one image, find the corresponding point in the other image This appears to be a 2D search problem, but it is reduced to a 1D search by the epipolar constraint

13. January 20, 2010T0283 - Computer Vision13 Notation x x / X C C / The two cameras are P and P /, and a 3D point X is imaged as for equations involving homogeneous quantities ‘=’ means ‘equal up to scale’ P P/P/ Warning

14. January 20, 2010T0283 - Computer Vision14 Epipolar geometry

15. January 20, 2010T0283 - Computer Vision15 Epipolar geometry Given an image point in one view, where is the corresponding point in the other view? A point in one view “generates” an epipolar line in the other view The corresponding point lies on this line epipolar line ? baseline epipole C / C

16. January 20, 2010T0283 - Computer Vision16 Epipolar line Epipolar constraint Reduces correspondence problem to 1D search along an epipolar line

17. January 20, 2010T0283 - Computer Vision17 Epipolar geometry continued Epipolar geometry is a consequence of the coplanarity of the camera centers and scene point x x / X C C / The camera centers, corresponding points and scene point lie in a single plane, known as the epipolar plane

18. January 20, 2010T0283 - Computer Vision18 Nomenclature The epipolar line l / is the image of the ray through x The epipole e is the point of intersection of the line joining the camera centres with the image plane this line is the baseline for a stereo rig, and the translation vector for a moving camera The epipole is the image of the centre of the other camera: e = P C /, e / = P / C x x / X C C / e left epipolar line right epipolar line e / l/l/

19. January 20, 2010T0283 - Computer Vision19 The epipolar pencil e e / baseline X As the position of the 3D point X varies, the epipolar planes “rotate” about the baseline. This family of planes is known as an epipolar pencil. All epipolar lines intersect at the epipole. (a pencil is a one parameter family)

20. January 20, 2010T0283 - Computer Vision20 The epipolar pencil e e / baseline X As the position of the 3D point X varies, the epipolar planes “rotate” about the baseline. This family of planes is known as an epipolar pencil. All epipolar lines intersect at the epipole. (a pencil is a one parameter family)

21. January 20, 2010T0283 - Computer Vision21 Epipolar geometry ex. 1: parallel cameras Epipolar geometry depends only on the relative pose (position and orientation) and internal parameters of the two cameras, i.e. the position of the camera centres and image planes. It does not depend on the scene structure (3D points external to the camera).

22. January 20, 2010T0283 - Computer Vision22 Epipolar geometry ex. 2: converging cameras Note, epipolar lines are in general not parallel e e /

23. January 20, 2010T0283 - Computer Vision23 Homogeneous notation for lines

24. January 20, 2010T0283 - Computer Vision24 The line l through the two points p and q is l = p x q Example: compute the point of intersection of the two lines l and m in the figure below Proof y x 1 2 The intersection of two lines l and m is the point x = l x m l m which is the point (2,1)

25. January 20, 2010T0283 - Computer Vision25 Matrix representation of the vector cross product

26. January 20, 2010T0283 - Computer Vision26 Example: compute the cross product of l and m

27. January 20, 2010T0283 - Computer Vision27 Algebraic representation of epipolar geometry We know that the epipolar geometry defines a mapping x l / point in first image epipolar line in second image

28. January 20, 2010T0283 - Computer Vision28 Derivation of the algebraic expression Outline Step 1: for a point x in the first image back project a ray with camera P Step 2: choose two points on the ray and project into the second image with camera P / Step 3: compute the line through the two image points using the relation l / = p x q P P/P/

29. January 20, 2010T0283 - Computer Vision29 choose camera matrices internal calibration rotationtranslation from world to camera coordinate frame first camera world coordinate frame aligned with first camera second camera

30. January 20, 2010T0283 - Computer Vision30 Step 1: for a point x in the first image back project a ray with camera P A point x back projects to a ray where Z is the point’s depth, since satisfies

31. January 20, 2010T0283 - Computer Vision31 Step 2: choose two points on the ray and project into the second image with camera P / P/P/ Consider two points on the ray Z = 0 is the camera centre Z = is the point at infinity Project these two points into the second view

32. January 20, 2010T0283 - Computer Vision32 Using the identity Compute the line through the points F F is the fundamental matrix Step 3: compute the line through the two image points using the relation l / = p x q

33. January 20, 2010T0283 - Computer Vision33 Example 1: compute the fundamental matrix for a parallel camera reduces to y = y /, i.e. raster correspondence (horizontal scan-lines) f f X Y Z

34. January 20, 2010T0283 - Computer Vision34 f f X Y Z

35. January 20, 2010T0283 - Computer Vision35 Example 2: compute F for a forward translating camera f f X Y Z

36. January 20, 2010T0283 - Computer Vision36 first image second image f f X Y Z

37. January 20, 2010T0283 - Computer Vision37

38. January 20, 2010T0283 - Computer Vision38

39. January 20, 2010T0283 - Computer Vision39 Summary: Properties of the Fundamental matrix