1. Multiview Geometry and Stereopsis

2. Inputs: two images of a scene (taken from 2 viewpoints). Output: Depth map. Inputs: multiple images of a scene. Output: 3D model.

3. Stereopsis Parallax Wikipedia

4. Compute Correspondences

6. Correspondences Geometric Constraints (given by epipolar geometry) make this 1D search problem instead of 2D.

7. Reconstruction / Triangulation

8. Reconstruction Linear Method: find P such that Non-Linear Method: find Q minimizing

9. Geometry of Two-View Epipolar geometry –Essential matrix –Fundamental matrix

10. Stereopsis Define a (linear) function F(p 1, p 2 ) such that it is zero if p 1 and p 2 are corresponding points in two images.

11. Epipolar plane defined by P, O, O’, p and p’ Epipoles e, e’ Epipolar lines l, l’ Baseline OO’ Epipolar geometry p’ lies on l’ where the epipolar plane intersects with image plane π’ l’ is epipolar line associated with p and intersects baseline OO’ on e’ e’ is the projection of O observed from O’

12. Potential matches for p have to lie on the corresponding epipolar line l’ Potential matches for p’ have to lie on the corresponding epipolar line l Epipolar constraint

13. Epipolar Constraint: Calibrated Case Essential Matrix (Longuet-Higgins, 1981) 3 ×3 skew-symmetric matrix: rank=2 O’P’ = t + R p’

14. E is defined by 5 parameters (scaling not relevant) E p’ is the epipolar line associated with p’ E T p is the epipolar line associated with p Can write as l.p = 0 The point p lies on the epipolar line associated with the vector E p’ Properties of essential matrix

15. E is defined by 5 parameters (scaling not relevant) E e’=0 and E T e=0 ( E e’=-R T [t x ]e=0 ) E is singular E has two equal non-zero singular values

16. Epipolar Constraint: Uncalibrated Case Fundamental Matrix (Faugeras and Luong, 1992) are normalized image coordinate

17. F has rank 2 and is defined by 7 parameters F p’ is the epipolar line associated with p’ in the 1 st image F T p is the epipolar line associated with p in the 2 nd image F e’=0 and F T e=0 F is singular Properties of fundamental matrix

18. Weak calibration In theory: –E can be estimated with 5 point correspondences –F can be estimated with 7 point correspondences –Some methods estimate E and F matrices from a minimal number of parameters Estimating epipolar geometry from a redundant set of point correspondences with unknown intrinsic parameters

19. The Eight-Point Algorithm (Longuet-Higgins, 1981) | F | =1. Minimize: under the constraint 2 Homogenous system, set F 33 =1

20. Enforcing Rank 2 Constraint Singular Value Decomposition of F (svd in MATLAB) Sigma is diagonal (its (non-negative) values are called singular values of F). U, V are orthogonal. Set the smallest singular value to zero.

21. The Normalized Eight-Point Algorithm (Hartley, 1995) Estimation of transformation parameters suffer from poor numerical condition problem Center the image data at the origin, and scale it so the mean squared distance between the origin and the data points is 2 pixels: q = T p, q’ = T’ p’ Use the eight-point algorithm to compute F from the points q and q’ Enforce the rank-2 constraint (use singular value decomposition) Output T F T’ T

22. Weak calibration experiment a)Linear Least Squares b)Hartley