3D Photography: Epipolar geometry

Slides:



Advertisements
Similar presentations
Epipolar Geometry.
Advertisements

The fundamental matrix F
Lecture 11: Two-view geometry
3D reconstruction.
MASKS © 2004 Invitation to 3D vision Lecture 7 Step-by-Step Model Buidling.
Two-View Geometry CS Sastry and Yang
Jan-Michael Frahm, Enrique Dunn Spring 2012
Two-view geometry.
Lecture 8: Stereo.
Epipolar Geometry class 11 Multiple View Geometry Comp Marc Pollefeys.
Camera calibration and epipolar geometry
Structure from motion.
Robot Vision SS 2008 Matthias Rüther 1 ROBOT VISION Lesson 6: Shape from Stereo Matthias Rüther Slides partial courtesy of Marc Pollefeys Department of.
Epipolar Geometry Class 7 Read notes Feature tracking run iterative L-K warp & upsample Tracking Good features Multi-scale Transl. Affine.
Computing F and rectification class 14 Multiple View Geometry Comp Marc Pollefeys.
Computing F Class 8 Read notes Section 4.2. C1C1 C2C2 l2l2  l1l1 e1e1 e2e2 Fundamental matrix (3x3 rank 2 matrix) 1.Computable from corresponding points.
Epipolar geometry. (i)Correspondence geometry: Given an image point x in the first view, how does this constrain the position of the corresponding point.
Uncalibrated Geometry & Stratification Sastry and Yang
Epipolar Geometry and the Fundamental Matrix F
Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai.
Multiple View Geometry
Multiple View Geometry Comp Marc Pollefeys
Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai.
Two-view geometry Epipolar geometry F-matrix comp. 3D reconstruction Structure comp.
Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai.
Triangulation and Multi-View Geometry Class 9 Read notes Section 3.3, , 5.1 (if interested, read Triggs’s paper on MVG using tensor notation, see.
Assignment 2 Compute F automatically from image pair (putative matches, 8-point, 7-point, iterative, RANSAC, guided matching) (due by Wednesday 19/03/03)
Previously Two view geometry: epipolar geometry Stereo vision: 3D reconstruction epipolar lines Baseline O O’ epipolar plane.
Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai.
May 2004Stereo1 Introduction to Computer Vision CS / ECE 181B Tuesday, May 11, 2004  Multiple view geometry and stereo  Handout #6 available (check with.
Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai.
Epipolar geometry Class 5
Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai.
3-D Scene u u’u’ Study the mathematical relations between corresponding image points. “Corresponding” means originated from the same 3D point. Objective.
Epipolar geometry Class 5. Geometric Computer Vision course schedule (tentative) LectureExercise Sept 16Introduction- Sept 23Geometry & Camera modelCamera.
Multi-view geometry. Multi-view geometry problems Structure: Given projections of the same 3D point in two or more images, compute the 3D coordinates.
Automatic Camera Calibration
Epipolar Geometry and Stereo Vision Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 03/05/15 Many slides adapted from Lana Lazebnik,
Computer vision: models, learning and inference
Lecture 11 Stereo Reconstruction I Lecture 11 Stereo Reconstruction I Mata kuliah: T Computer Vision Tahun: 2010.
Multi-view geometry.
Lecture 12 Stereo Reconstruction II Lecture 12 Stereo Reconstruction II Mata kuliah: T Computer Vision Tahun: 2010.
Epipolar geometry The fundamental matrix and the tensor
Lecture 04 22/11/2011 Shai Avidan הבהרה : החומר המחייב הוא החומר הנלמד בכיתה ולא זה המופיע / לא מופיע במצגת.
Metrology 1.Perspective distortion. 2.Depth is lost.
Stereo Course web page: vision.cis.udel.edu/~cv April 11, 2003  Lecture 21.
Robot Vision SS 2007 Matthias Rüther 1 ROBOT VISION Lesson 6a: Shape from Stereo, short summary Matthias Rüther Slides partial courtesy of Marc Pollefeys.
1 Formation et Analyse d’Images Session 7 Daniela Hall 25 November 2004.
Two-view geometry Epipolar geometry F-matrix comp. 3D reconstruction
Computing F. Content Background: Projective geometry (2D, 3D), Parameter estimation, Algorithm evaluation. Single View: Camera model, Calibration, Single.
Two-view geometry. Epipolar Plane – plane containing baseline (1D family) Epipoles = intersections of baseline with image planes = projections of the.
Feature Matching. Feature Space Outlier Rejection.
Uncalibrated reconstruction Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration.
Multiview geometry ECE 847: Digital Image Processing Stan Birchfield Clemson University.
Multi-view geometry. Multi-view geometry problems Structure: Given projections of the same 3D point in two or more images, compute the 3D coordinates.
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
Epipolar Geometry and Stereo Vision
Two-view geometry Computer Vision Spring 2018, Lecture 10
Homography From Wikipedia In the field of computer vision, any
Epipolar geometry.
Epipolar Geometry class 11
Multiple View Geometry Comp Marc Pollefeys
The Brightness Constraint
Estimating 2-view relationships
3D reconstruction class 11
Uncalibrated Geometry & Stratification
Two-view geometry Epipolar geometry F-matrix comp. 3D reconstruction
Two-view geometry.
Two-view geometry.
Multi-view geometry.
Presentation transcript:

3D Photography: Epipolar geometry Kevin Köser, Marc Pollefeys Sprint 2011 http://www.cvg.ethz.ch/teaching/2011spring/3dphoto/

Two-view geometry Three questions: Correspondence geometry: Given an image point x in the first image, how does this constrain the position of the corresponding point x’ in the second image? (ii) Camera geometry (motion): Given a set of corresponding image points {xi ↔x’i}, i=1,…,n, what are the cameras P and P’ for the two views? (iii) Scene geometry (structure): Given corresponding image points xi ↔x’i and cameras P, P’, what is the position of (their pre-image) X in space?

The epipolar geometry C,C’,x,x’ and X are coplanar

The epipolar geometry What if only C,C’,x are known?

The epipolar geometry All points on p project on l and l’

The epipolar geometry Family of planes p and lines l and l’ Intersection in e and e’

The epipolar geometry epipoles e,e’ = intersection of baseline with image plane = projection of projection center in other image = vanishing point of camera motion direction an epipolar plane = plane containing baseline (1-D family) an epipolar line = intersection of epipolar plane with image (always come in corresponding pairs)

Example: converging cameras

(simple for stereo  rectification) Example: motion parallel with image plane (simple for stereo  rectification)

Example: forward motion

The fundamental matrix F algebraic representation of epipolar geometry we will see that mapping is (singular) correlation (i.e. projective mapping from points to lines) represented by the fundamental matrix F

The fundamental matrix F geometric derivation mapping from 2-D to 1-D family (rank 2)

The fundamental matrix F algebraic derivation (note: doesn’t work for C=C’  F=0)

The fundamental matrix F correspondence condition The fundamental matrix satisfies the condition that for any pair of corresponding points x↔x’ in the two images

The fundamental matrix F F is the unique 3x3 rank 2 matrix that satisfies x’TFx=0 for all x↔x’ Transpose: if F is fundamental matrix for (P,P’), then FT is fundamental matrix for (P’,P) Epipolar lines: l’=Fx & l=FTx’ Epipoles: on all epipolar lines, thus e’TFx=0, x e’TF=0, similarly Fe=0 F has 7 d.o.f. , i.e. 3x3-1(homogeneous)-1(rank2) F is a correlation, projective mapping from a point x to a line l’=Fx (not a proper correlation, i.e. not invertible)

Fundamental matrix for pure translation

Fundamental matrix for pure translation

Fundamental matrix for pure translation General motion Pure translation for pure translation F only has 2 degrees of freedom

The fundamental matrix F relation to homographies valid for all plane homographies

The fundamental matrix F relation to homographies requires

Projective transformation and invariance Derivation based purely on projective concepts F invariant to transformations of projective 3-space unique not unique canonical form

Projective ambiguity of cameras given F previous slide: at least projective ambiguity this slide: not more! Show that if F is same for (P,P’) and (P,P’), there exists a projective transformation H so that P=HP and P’=HP’ ~ ~ ~ lemma: (22-15=7, ok)

The projective reconstruction theorem If a set of point correspondences in two views determine the fundamental matrix uniquely, then the scene and cameras may be reconstructed from these correspondences alone, and any two such reconstructions from these correspondences are projectively equivalent allows reconstruction from pair of uncalibrated images!

Fundamental matrix (3x3 rank 2 matrix) Epipolar geometry l2 C1 m1 L1 m2 L2 M C2 C1 C2 l2 p l1 e1 e2 m1 L1 m2 L2 M C1 C2 l2 p l1 e1 e2 Underlying structure in set of matches for rigid scenes m1 m2 lT1 l2 Fundamental matrix (3x3 rank 2 matrix) Computable from corresponding points Simplifies matching Allows to detect wrong matches Related to calibration Canonical representation:

Epipolar geometry? courtesy Frank Dellaert

Other entities besides points? Lines give no constraint for two view geometry (but will for three and more views) Curves and surfaces yield some constraints related to tangency (e.g. Sinha et al. CVPR’04)

Computation of F (and E) Linear (8-point) Minimal (7-point) Robust (RANSAC) Non-linear refinement (MLE, …) Practical approach Calibrated 5-point Calibrated + know vertical 3-point

separate known from unknown Epipolar geometry: basic equation separate known from unknown (data) (unknowns) (linear)

! the NOT normalized 8-point algorithm Orders of magnitude difference ~10000 ~100 1 ! Orders of magnitude difference between column of data matrix  least-squares yields poor results

Transform image to ~[-1,1]x[-1,1] the normalized 8-point algorithm Transform image to ~[-1,1]x[-1,1] (0,0) (700,500) (700,0) (0,500) (1,-1) (0,0) (1,1) (-1,1) (-1,-1) normalized least squares yields good results (Hartley, PAMI´97)

the singularity constraint SVD from linearly computed F matrix (rank 3) Compute closest rank-2 approximation

the minimum case – 7 point correspondences one parameter family of solutions but F1+lF2 not automatically rank 2

the minimum case – impose rank 2 F1 F2 F 3 F7pts (obtain 1 or 3 solutions) (cubic equation) Compute possible l as eigenvalues of (only real solutions are potential solutions)

Automatic computation of F Step 1. Extract features Step 2. Compute a set of potential matches Step 3. do Step 3.1 select minimal sample (i.e. 7 matches) Step 3.2 compute solution(s) for F Step 3.3 determine inliers until (#inliers,#samples)<95% (generate hypothesis) (verify hypothesis) RANSAC Step 4. Compute F based on all inliers Step 5. Look for additional matches Step 6. Refine F based on all correct matches #inliers 90% 80% 70% 60% 50% #samples 5 13 35 106 382

restrict search range to neighborhood of epipolar line Finding more matches restrict search range to neighborhood of epipolar line (e.g. 1.5 pixels) relax disparity restriction (along epipolar line)

(Mostly) planar scene (see next slide) Issues: (Mostly) planar scene (see next slide) Absence of sufficient features (no texture) Repeated structure ambiguity Robust matcher also finds support for wrong hypothesis solution: detect repetition (Schaffalitzky and Zisserman, BMVC‘98)

Computing F for quasi-planar scenes QDEGSAC 337 matches on plane, 11 off plane #inliers %inclusion of out-of-plane inliers 17% success for RANSAC 100% for QDEGSAC data rank

5-point relative motion (Nister, CVPR03) Linear equations for 5 points Linear solution space Non-linear constraints scale does not matter, choose 10 cubic polynomials

5-point relative motion (Nister, CVPR03) Perform Gauss-Jordan elimination on polynomials represents polynomial of degree n in z -z -z -z

Minimal relative pose with know vertical Fraundorfer, Tanskanen and Pollefeys, ECCV2010 Vertical direction can often be estimated inertial sensor vanishing point -g 5 linear unknowns  linear 5 point algorithm 3 unknowns  quartic 3 point algorithm

geometric relations between two views is fully two-view geometry geometric relations between two views is fully described by recovered 3x3 matrix F