Multiview Geometry and Stereopsis

Inputs: two images of a scene (taken from 2 viewpoints). Output: Depth map. Inputs: multiple images of a scene. Output: 3D model.

Stereopsis Parallax Wikipedia

Compute Correspondences

Correspondences Geometric Constraints (given by epipolar geometry) make this 1D search problem instead of 2D.

Reconstruction / Triangulation

Reconstruction Linear Method: find P such that Non-Linear Method: find Q minimizing

Geometry of Two-View Epipolar geometry –Essential matrix –Fundamental matrix

Stereopsis Define a (linear) function F(p 1, p 2 ) such that it is zero if p 1 and p 2 are corresponding points in two images.

Epipolar plane defined by P, O, O’, p and p’ Epipoles e, e’ Epipolar lines l, l’ Baseline OO’ Epipolar geometry p’ lies on l’ where the epipolar plane intersects with image plane π’ l’ is epipolar line associated with p and intersects baseline OO’ on e’ e’ is the projection of O observed from O’

Potential matches for p have to lie on the corresponding epipolar line l’ Potential matches for p’ have to lie on the corresponding epipolar line l Epipolar constraint

Epipolar Constraint: Calibrated Case Essential Matrix (Longuet-Higgins, 1981) 3 ×3 skew-symmetric matrix: rank=2 O’P’ = t + R p’

E is defined by 5 parameters (scaling not relevant) E p’ is the epipolar line associated with p’ E T p is the epipolar line associated with p Can write as l.p = 0 The point p lies on the epipolar line associated with the vector E p’ Properties of essential matrix

E is defined by 5 parameters (scaling not relevant) E e’=0 and E T e=0 ( E e’=-R T [t x ]e=0 ) E is singular E has two equal non-zero singular values

Epipolar Constraint: Uncalibrated Case Fundamental Matrix (Faugeras and Luong, 1992) are normalized image coordinate

F has rank 2 and is defined by 7 parameters F p’ is the epipolar line associated with p’ in the 1 st image F T p is the epipolar line associated with p in the 2 nd image F e’=0 and F T e=0 F is singular Properties of fundamental matrix

Weak calibration In theory: –E can be estimated with 5 point correspondences –F can be estimated with 7 point correspondences –Some methods estimate E and F matrices from a minimal number of parameters Estimating epipolar geometry from a redundant set of point correspondences with unknown intrinsic parameters

The Eight-Point Algorithm (Longuet-Higgins, 1981) | F | =1. Minimize: under the constraint 2 Homogenous system, set F 33 =1

Enforcing Rank 2 Constraint Singular Value Decomposition of F (svd in MATLAB) Sigma is diagonal (its (non-negative) values are called singular values of F). U, V are orthogonal. Set the smallest singular value to zero.

The Normalized Eight-Point Algorithm (Hartley, 1995) Estimation of transformation parameters suffer from poor numerical condition problem Center the image data at the origin, and scale it so the mean squared distance between the origin and the data points is 2 pixels: q = T p, q’ = T’ p’ Use the eight-point algorithm to compute F from the points q and q’ Enforce the rank-2 constraint (use singular value decomposition) Output T F T’ T

Weak calibration experiment a)Linear Least Squares b)Hartley