Estimating 2-view relationships CMPUT 499/615

Fundamental matrix Algebraic representation of epipolar geometry [Faugeras ’92, Hartley ’92 ] Algebraic representation of epipolar geometry Step 1: X on a plane  Step 2: epipolar line l’ F 3x3, Rank 2, det(F)=0 Linear sol. – 8 corr. Points (unique) Nonlinear sol. – 7 corr. points (3sol.) Very sensitive to noise & outliers Epipolar lines: Epipoles: Projection matrices:

The projective reconstruction theorem If a set of point correspondences in two views determine the fundamental matrix uniquely, then the scene and cameras may be reconstructed from these correspondences alone, and any two such reconstructions from these correspondences are projectively equivalent allows reconstruction from pair of uncalibrated images!

Upcoming: 2view Reconstuction Objective Given two uncalibrated images compute (PM,P‘M,{XMi}) (i.e. within similarity of original scene and cameras) Algorithm Compute projective reconstruction (P,P‘,{Xi}) Compute F from xi↔x‘i Compute P,P‘ from F Triangulate Xi from xi↔x‘i Rectify reconstruction from projective to metric Direct method: compute H from control points Stratified method: Affine reconstruction: compute p∞ Metric reconstruction: compute IAC w

separate known from unknown Epipolar geometry: basic equation separate known from unknown (data) (unknowns) (linear)

8-point algorithm Solve for nontrivial solution using SVD: Var subst: Now Min Hence x = last vector in V

! the NOT normalized 8-point algorithm Orders of magnitude difference ~10000 ~100 1 ! Orders of magnitude difference Between column of data matrix  least-squares yields poor results

Transform image to ~[-1,1]x[-1,1] the normalized 8-point algorithm Transform image to ~[-1,1]x[-1,1] (0,0) (700,500) (700,0) (0,500) (1,-1) (0,0) (1,1) (-1,1) (-1,-1) normalized least squares yields good results (Hartley, PAMI´97) 8

the singularity constraint Non-singular F Singular F

the singularity constraint SVD from linearly computed F matrix (rank 3) Compute closest rank-2 approximation

Geometric distance: Gold standard Maximum Likelihood Estimation (= least-squares for Gaussian noise) Initialize: normalized 8-point, (P,P‘) from F, reconstruct Xi Parameterize: (overparametrized) Minimize cost using Levenberg-Marquardt (preferably sparse LM, see book)

Some experiments:

Some experiments:

Some experiments:

Automatic computation of F Recommendations: Do not use unnormalized algorithms Quick and easy to implement: 8-point normalized Better: enforce rank-2 constraint during minimization Best: Maximum Likelihood Estimation (minimal parameterization, sparse implementation) Automatic computation of F Interest points Putative correspondences RANSAC (iv) Non-linear re-estimation of F Guided matching (repeat (iv) and (v) until stable)

The envelope of epipolar lines What happens to an epipolar line if there is noise? Monte Carlo n=10 n=15 n=25 n=50

Other entities? Lines give no constraint for two view geometry (but will for three and more views) Curves and surfaces yield some constraints related to tangency

Structure from images: 3D Point reconstruction

Solve with SVD for nullvector linear triangulation Solve with SVD for nullvector homogeneous inhomogeneous

Linear triangulation Alternative way of linear intersection: Formulate a set of linear equations explicitly solving for l’s See our VR2003 tutorial p. 26

geometric error possibility to compute using LM (for 2 or more points) or directly (for 2 points)

Geometric error Reconstruct matches in projective frame by minimizing the reprojection error Non-iterative optimal solution (see Hartley&Sturm,CVIU´97)

Optimal 3D point in epipolar plane Given an epipolar plane, find best 3D point for (x1,x2) x1 x2 l1 l2 l1 x1 x2 l2 x1´ x2´ Select closest points (x1´,x2´) on epipolar lines Obtain 3D point through exact triangulation Guarantees minimal reprojection error (given this epipolar plane)

Optimal epipolar plane Reconstruct matches in projective frame by minimizing the reprojection error Non-iterative method Determine the epipolar plane for reconstruction Reconstruct optimal point from selected epipolar plane 3DOF (Hartley and Sturm, CVIU´97) 1DOF m1 m2 l1(a) l2(a) (polynomial of degree 6 check all minima, incl ∞)

consider angle between rays Reconstruction uncertainty consider angle between rays

doesn‘t work for epipolar plane Line reconstruction doesn‘t work for epipolar plane