Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford Stereo

Sebastian Thrun Stanford University CS223B Computer Vision Stereo Vision: Illustration

Sebastian Thrun Stanford University CS223B Computer Vision Stereo Vision: Outline Basic Equations Epipolar Geometry Image Rectification Reconstruction Correspondence Dense and Layered Stereo (Active Range Imaging Techniques)

Sebastian Thrun Stanford University CS223B Computer Vision Pinhole Camera Model Image plane Focal length f Center of projection

Sebastian Thrun Stanford University CS223B Computer Vision Pinhole Camera Model Image plane

Sebastian Thrun Stanford University CS223B Computer Vision Pinhole Camera Model Image plane

Sebastian Thrun Stanford University CS223B Computer Vision Basic Stereo Derivations

Sebastian Thrun Stanford University CS223B Computer Vision Basic Stereo Derivations

Sebastian Thrun Stanford University CS223B Computer Vision What If…?

Sebastian Thrun Stanford University CS223B Computer Vision Epipolar Geometry p l p r P OlOl OrOr XlXl XrXr PlPl PrPr flfl frfr ZlZl YlYl ZrZr YrYr

Sebastian Thrun Stanford University CS223B Computer Vision Epipolar Geometry p l p r P OlOl OrOr elel erer PlPl PrPr Epipolar Plane Epipolar Lines Epipoles

Sebastian Thrun Stanford University CS223B Computer Vision Epipolar Geometry Epipolar plane: plane going through point P and the centers of projection (COPs) of the two cameras Epipoles: The image in one camera of the COP of the other Epipolar Constraint: Corresponding points must lie on epipolar lines

Sebastian Thrun Stanford University CS223B Computer Vision Essential Matrix p l p r P OlOl OrOr elel erer PlPl PrPr Orthogonality T, P l, P l  T : Coordinate Transformation: Resolves to Essential Matrix

Sebastian Thrun Stanford University CS223B Computer Vision Essential Matrix p l p r P OlOl OrOr elel erer PlPl PrPr Projective Line:

Sebastian Thrun Stanford University CS223B Computer Vision Fundamental Matrix Same as Essential Matrix in Camera Pixel Coordinates Pixel coordinates Intrinsic parameters

Sebastian Thrun Stanford University CS223B Computer Vision Computing F: The Eight-Point Algorithm Input: n point correspondences ( n >= 8) –Construct homogeneous system Ax= 0 from x = (f 11,f 12,,f 13, f 21,f 22,f 23 f 31,f 32, f 33 ) : entries in F Each correspondence give one equation A is a nx9 matrix –Obtain estimate F^ by SVD of A: x (up to a scale) is column of V corresponding to the least singular value –Enforce singularity constraint: since Rank (F) = 2 Compute SVD of F: Set the smallest singular value to 0: D -> D’ Correct estimate of F : Output: the estimate of the fundamental matrix F’ Similarly we can compute E given intrinsic parameters

Sebastian Thrun Stanford University CS223B Computer Vision Recitification Idea: Align Epipolar Lines with Scan Lines. Question: What type transformation?

Sebastian Thrun Stanford University CS223B Computer Vision Locating the Epipoles p l p r P OlOl OrOr elel erer PlPl PrPr Input: Fundamental Matrix F –Find the SVD of F –The epipole e l is the column of V corresponding to the null singular value (as shown above) –The epipole e r is the column of U corresponding to the null singular value (similar treatment as for e l ) Output: Epipole e l and e r e l lies on all the epipolar lines of the left image

Sebastian Thrun Stanford University CS223B Computer Vision Stereo Rectification (see Trucco) Stereo System with Parallel Optical Axes n Epipoles are at infinity n Horizontal epipolar lines p l p r P OlOl OrOr XlXl XrXr PlPl PrPr ZlZl YlYl ZrZr YrYr T

Sebastian Thrun Stanford University CS223B Computer Vision p l p r P OlOl OrOr PlPl PrPr Reconstruction (3-D): Idealized

Sebastian Thrun Stanford University CS223B Computer Vision p l p r P OlOl OrOr PlPl PrPr Reconstruction (3-D): Real See Trucco/Verri, pages

Sebastian Thrun Stanford University CS223B Computer Vision Summary Stereo Vision (Class 1) Epipolar Geometry: Corresponding points lie on epipolar line Essential/Fundamental matrix: Defines this line Eight-Point Algorithm: Recovers Fundamental matrix Rectification: Epipolar lines parallel to scanlines Reconstruction: Minimize quadratic distance