2. Lecture 04 22/11/2011 Shai Avidan הבהרה : החומר המחייב הוא החומר הנלמד בכיתה ולא זה המופיע / לא מופיע במצגת.

3. Epipolar Geometry Essential Matrix (Longuet-Higgins 1981) Fundamental Matrix (Faugeras 1992) F and Homographies

4. ESSENTIAL MATRIX

5. Stereo correspondence constraints Geometry of two views allows us to constrain where the corresponding pixel for some image point in the first view must occur in the second view. Epipolar constraint: Why is this useful? Reduces correspondence problem to 1D search along conjugate epipolar lines epipolar plane epipolar line Adapted from Steve Seitz

6. Epipolar geometry: terms Baseline: line joining the camera centers Epipole: point of intersection of baseline with the image plane Epipolar plane: plane containing baseline and world point Epipolar line: intersection of epipolar plane with the image plane All epipolar lines intersect at the epipole An epipolar plane intersects the left and right image planes in epipolar lines

7. Potential matches for p have to lie on the corresponding epipolar line l’. Potential matches for p’ have to lie on the corresponding epipolar line l. Source: M. Pollefeys http://www.ai.sri.com/~luong/research/Meta3DViewer/EpipolarGeo.html Epipolar constraint

8. Example

9. Example: converging cameras Figure from Hartley & Zisserman As position of 3d point varies, epipolar lines “rotate” about the baseline

10. Example: motion parallel with image plane Figure from Hartley & Zisserman

11. Example: forward motion Figure from Hartley & Zisserman e e’ Epipole has same coordinates in both images. Points move along lines radiating from e: “Focus of expansion”

12. Stereo geometry, with calibrated cameras If the rig is calibrated, we know : how to rotate and translate camera reference frame 1 to get to camera reference frame 2. Rotation: 3 x 3 matrix; translation: 3 vector.

13. 3d rigid transformation ‘ ‘ ‘

14. Stereo geometry, with calibrated cameras Camera-centered coordinate systems are related by known rotation R and translation T:

15. Cross product Vector cross product takes two vectors and returns a third vector that’s perpendicular to both inputs. So here, c is perpendicular to both a and b, which means the dot product = 0.

16. From geometry to algebra Normal to the plane

17. Matrix form of cross product Can be expressed as a matrix multiplication.

18. From geometry to algebra Normal to the plane

19. Essential matrix E is called the essential matrix, which relates corresponding image points [Longuet-Higgins 1981] Let This holds for the rays p and p’ that are parallel to the camera-centered position vectors X and X’, so we have:

20. Essential matrix and epipolar lines is the coordinate vector representing the epipolar line associated with point p’ is the coordinate vector representing the epipolar line associated with point p Epipolar constraint: if we observe point p in one image, then its position p’ in second image must satisfy this equation.

21. Essential matrix: properties Relates image of corresponding points in both cameras, given rotation and translation Assuming intrinsic parameters are known E is a rank 2 matrix with two equal eigvenvalues Given E, the camera projection matrix can be computed using SVD factorization. There are four possible solutions

22. From E to Motion

23. FUNDAMENTAL MATRIX

24. Uncalibrated case Camera coordinates Internal calibration matrices, one per camera Image pixel coordinates So, for two cameras (left and right): For a given camera: Camera coordinates

25. Uncalibrated case: fundamental matrix From before, the essential matrix E. Fundamental matrix

26. Relates pixel coordinates in the two views More general form than essential matrix: we remove need to know intrinsic parameters If we estimate fundamental matrix from correspondences in pixel coordinates, can reconstruct epipolar geometry without intrinsic or extrinsic parameters

27. Computing F from correspondences Cameras are uncalibrated: we don’t know E or left or right K matrices Estimate F from 8+ point correspondences.

28. Computing F from correspondences Each point correspondence generates one constraint on F Collect n of these constraints Solve for f, vector of parameters.

29. F AND HOMOGRAPHIES

30. F decomposition Given F, we want to decompose to left and right projection matrices. Because reconstruction is up to global projective transformation we can set the left camera projection matrix to [I;0]. Since p’^tFp=0 for every pair of points, there is one point e’ s.t. e’^tFp=0 for every p -> e’ is in the null space of F. e’ is the epipole and all epipolar lines go through it. From the epipolar constraint (shown for the essential matrix) we have that So, we have M_1=[I;0] and M_2 = [H,e’] Where H is a homography

31. The rank-4 of homographies

32. The canonical homography Choose some line l’ going through p’ then: l’^T p’ = 0 And from the epipolar constraint we have: p’^TFp = 0 Combining the two we have that p’ is on the intersection of the two lines: P’ =~ [l’]_x Fp And we have a homography equation: p’ =~ [l’]_x Fp =Hp In particular, lets choose l’ =~ e’, which is guaranteed not to coincide with the epipolar lines (e’Te’ !=0) and have that H = [e’]_xF is a homography that is called the canonical homography matrix

33. Relationship between F and H What is the relationship between homography and fundamental matrix? Let’s choose some homography, then we have: (Hp)^TFp = 0 and this is true for every point p. So we have that Fp =~ [e’]_x Hp, because they both describe the epipolar line s.t. p’^TFp and p’^T [e’]_x Hp=0 Geometric interpretation: The points Hp, p’ and e’ are all on the same line (the epipolar line) And this can be written as: p’^T[e]_xHp=0