Parameter estimation class 5

Slides:



Advertisements
Similar presentations
Projective 3D geometry class 4
Advertisements

Epipolar Geometry.
Lecture 11: Two-view geometry
The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.
Computing 3-view Geometry Class 18
Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer.
Two-View Geometry CS Sastry and Yang
Multiple View Reconstruction Class 24 Multiple View Geometry Comp Marc Pollefeys.
N-view factorization and bundle adjustment CMPUT 613.
Epipolar Geometry class 11 Multiple View Geometry Comp Marc Pollefeys.
Camera calibration and epipolar geometry
Structure from motion.
Scene Planes and Homographies class 16 Multiple View Geometry Comp Marc Pollefeys.
Projective structure from motion
Multiple View Geometry
More on single-view geometry class 10 Multiple View Geometry Comp Marc Pollefeys.
3D reconstruction class 11
Projective 2D geometry (cont’) course 3
Parameter estimation class 5 Multiple View Geometry Comp Marc Pollefeys.
Epipolar geometry. (i)Correspondence geometry: Given an image point x in the first view, how does this constrain the position of the corresponding point.
Structure from motion. Multiple-view geometry questions Scene geometry (structure): Given 2D point matches in two or more images, where are the corresponding.
Uncalibrated Geometry & Stratification Sastry and Yang
Parameter estimation class 6 Multiple View Geometry Comp Marc Pollefeys.
Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai.
Multiple-view Reconstruction from Points and Lines
Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai.
Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai.
Multiple View Geometry in Computer Vision
Single View Metrology Class 3. 3D photography course schedule (tentative) LectureExercise Sept 26Introduction- Oct. 3Geometry & Camera modelCamera calibration.
CMPUT 412 3D Computer Vision Presented by Azad Shademan Feb , 2007.
Camera Models class 8 Multiple View Geometry Comp Marc Pollefeys.
 -Linearities and Multiple View Tensors Class 19 Multiple View Geometry Comp Marc Pollefeys.
Singular Value Decomposition. Homogeneous least-squares Span and null-space Closest rank r approximation Pseudo inverse.
More on single-view geometry class 10 Multiple View Geometry Comp Marc Pollefeys.
Multiple View Reconstruction Class 23 Multiple View Geometry Comp Marc Pollefeys.
Algorithm Evaluation and Error Analysis class 7 Multiple View Geometry Comp Marc Pollefeys.
Camera Calibration class 9 Multiple View Geometry Comp Marc Pollefeys.
Projective 2D geometry course 2 Multiple View Geometry Comp Marc Pollefeys.
The Trifocal Tensor Class 17 Multiple View Geometry Comp Marc Pollefeys.
Structure Computation. How to compute the position of a point in 3- space given its image in two views and the camera matrices of those two views Use.
Projective 2D geometry course 2 Multiple View Geometry Comp Marc Pollefeys.
Computer vision: models, learning and inference
Projective geometry of 2-space DLT alg HZ 4.1 Rectification HZ 2.7 Hierarchy of maps Invariants HZ 2.4 Projective transform HZ 2.3 Behaviour at infinity.
Computing the Fundamental matrix Peter Praženica FMFI UK May 5, 2008.
Robot Vision SS 2008 Matthias Rüther 1 ROBOT VISION Lesson 2: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer.
Geometry and Algebra of Multiple Views
Epipolar geometry The fundamental matrix and the tensor
1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained.
Camera Calibration class 9 Multiple View Geometry Comp Marc Pollefeys.
Multiview Geometry and Stereopsis. Inputs: two images of a scene (taken from 2 viewpoints). Output: Depth map. Inputs: multiple images of a scene. Output:
Parameter estimation. 2D homography Given a set of (x i,x i ’), compute H (x i ’=Hx i ) 3D to 2D camera projection Given a set of (X i,x i ), compute.
Day x 5-10 minute quiz SVD demo review of metric rectification algorithm 1.build a 2x3 matrix A from orthog line pairs A = [L1M1, L1M2+L2M1, L2M2; L1’M1’,
A Flexible New Technique for Camera Calibration Zhengyou Zhang Sung Huh CSPS 643 Individual Presentation 1 February 25,
Parameter estimation class 5 Multiple View Geometry CPSC 689 Slides modified from Marc Pollefeys’ Comp
Camera Calibration Course web page: vision.cis.udel.edu/cv March 24, 2003  Lecture 17.
Projective 2D geometry course 2 Multiple View Geometry Comp Marc Pollefeys.
Homogeneous Coordinates (Projective Space)
Two-view geometry Computer Vision Spring 2018, Lecture 10
Epipolar geometry.
Epipolar Geometry class 11
3D Photography: Epipolar geometry
Multiple View Geometry Comp Marc Pollefeys
More on single-view geometry class 10
Estimating 2-view relationships
Multiple View Geometry Comp Marc Pollefeys
3D reconstruction class 11
Camera Calibration class 9
Uncalibrated Geometry & Stratification
Back to equations of geometric transformations
Parameter estimation class 6
Presentation transcript:

Parameter estimation class 5 Multiple View Geometry Comp 290-089 Marc Pollefeys

Content Background: Projective geometry (2D, 3D), Parameter estimation, Algorithm evaluation. Single View: Camera model, Calibration, Single View Geometry. Two Views: Epipolar Geometry, 3D reconstruction, Computing F, Computing structure, Plane and homographies. Three Views: Trifocal Tensor, Computing T. More Views: N-Linearities, Multiple view reconstruction, Bundle adjustment, auto-calibration, Dynamic SfM, Cheirality, Duality

Multiple View Geometry course schedule (subject to change) Jan. 7, 9 Intro & motivation Projective 2D Geometry Jan. 14, 16 (no class) Jan. 21, 23 Projective 3D Geometry Jan. 28, 30 Parameter Estimation Feb. 4, 6 Algorithm Evaluation Camera Models Feb. 11, 13 Camera Calibration Single View Geometry Feb. 18, 20 Epipolar Geometry 3D reconstruction Feb. 25, 27 Fund. Matrix Comp. Structure Comp. Mar. 4, 6 Planes & Homographies Trifocal Tensor Mar. 18, 20 Three View Reconstruction Multiple View Geometry Mar. 25, 27 MultipleView Reconstruction Bundle adjustment Apr. 1, 3 Auto-Calibration Papers Apr. 8, 10 Dynamic SfM Apr. 15, 17 Cheirality Apr. 22, 24 Duality Project Demos

Projective 3D Geometry Points, lines, planes and quadrics Transformations П∞, ω∞ and Ω ∞

Singular Value Decomposition Homogeneous least-squares

Parameter estimation 2D homography 3D to 2D camera projection Given a set of (xi,xi’), compute H (xi’=Hxi) 3D to 2D camera projection Given a set of (Xi,xi), compute P (xi=PXi) Fundamental matrix Given a set of (xi,xi’), compute F (xi’TFxi=0) Trifocal tensor Given a set of (xi,xi’,xi”), compute T

Number of measurements required At least as many independent equations as degrees of freedom required Example: 2 independent equations / point 8 degrees of freedom 4x2≥8

Approximate solutions Minimal solution 4 points yield an exact solution for H More points No exact solution, because measurements are inexact (“noise”) Search for “best” according to some cost function Algebraic or geometric/statistical cost Exact solutions are important for robust approaches

Gold Standard algorithm Cost function that is optimal for some assumptions Computational algorithm that minimizes it is called “Gold Standard” algorithm Other algorithms can then be compared to it Other algorithms can have other advantages: faster, more reliable, etc.

Direct Linear Transformation (DLT)

Direct Linear Transformation (DLT) Equations are linear in h Only 2 out of 3 are linearly independent (indeed, 2 eq/pt) row 3 = x.(row 1)+y.(row 2) (only drop third row if wi’≠0) Holds for any homogeneous representation, e.g. (xi’,yi’,1)

Direct Linear Transformation (DLT) Solving for H size A is 8x9 or 12x9, but rank 8 Trivial solution is h=09T is not interesting 1-D null-space yields solution of interest pick for example the one with

Direct Linear Transformation (DLT) Over-determined solution No exact solution because of inexact measurement i.e. “noise” Find approximate solution Additional constraint needed to avoid 0, e.g. not possible, so minimize

DLT algorithm Objective Given n≥4 2D to 2D point correspondences {xi↔xi’}, determine the 2D homography matrix H such that xi’=Hxi Algorithm For each correspondence xi ↔xi’ compute Ai. Usually only two first rows needed. Assemble n 2x9 matrices Ai into a single 2nx9 matrix A Obtain SVD of A. Solution for h is last column of V Determine H from h

Inhomogeneous solution Since h can only be computed up to scale, pick hj=1, e.g. h9=1, and solve for 8-vector Solve using Gaussian elimination (4 points) or using linear least-squares (more than 4 points) However, if h9=0 this approach fails also poor results if h9 close to zero Therefore, not recommended Note h9=H33=0 if origin is mapped to infinity

Degenerate configurations x1 x1 x4 x1 x4 x4 x2 H? x2 H’? x2 x3 x3 x3 (case A) (case B) Constraints: i=1,2,3,4 Define: Then, H* is rank-1 matrix and thus not a homography If H* is unique solution, then no homography mapping xi→xi’(case B) If further solution H exist, then also αH*+βH (case A) (2-D null-space in stead of 1-D null-space)

Solutions from lines, etc. 2D homographies from 2D lines Minimum of 4 lines Minimum of 5 points or 5 planes 3D Homographies (15 dof) 2D affinities (6 dof) Minimum of 3 points or lines Conic provides 5 constraints Mixed configurations?

Cost functions Algebraic distance Geometric distance Reprojection error Comparison Geometric interpretation Sampson error

Algebraic distance DLT minimizes residual vector partial vector for each (xi↔xi’) algebraic error vector algebraic distance where Not geometrically/statistically meaningfull, but given good normalization it works fine and is very fast (use for initialization)

Geometric distance d(.,.) Euclidean distance (in image) measured coordinates estimated coordinates true coordinates d(.,.) Euclidean distance (in image) Error in one image e.g. calibration pattern Symmetric transfer error Reprojection error

Reprojection error

Comparison of geometric and algebraic distances Error in one image typical, but not , except for affinities For affinities DLT can minimize geometric distance Possibility for iterative algorithm

Geometric interpretation of reprojection error Estimating homography~fit surface to points X=(x,y,x’,y’)T in 4 represents 2 quadrics in 4 (quadratic in X) Analog to conic fitting Nonlinear, except for line fitting = affine homographies (no quadratic terms in this case!) (x,x’,y and x,y,y’) => linear solution

Sampson error between algebraic and geometric error Vector that minimizes the geometric error is the closest point on the variety to the measurement Sampson error: 1st order approximation of Find the vector that minimizes subject to

Find the vector that minimizes subject to Use Lagrange multipliers: minimize derivatives

Sampson error between algebraic and geometric error Vector that minimizes the geometric error is the closest point on the variety to the measurement Sampson error: 1st order approximation of Find the vector that minimizes subject to (Sampson error)

Sampson approximation A few points For a 2D homography X=(x,y,x’,y’) is the algebraic error vector is a 2x4 matrix, e.g. Similar to algebraic error in fact, same as Mahalanobis distance Sampson error independent of linear reparametrization (cancels out in between e and J) Must be summed for all points Close to geometric error, but much fewer unknowns

Statistical cost function and Maximum Likelihood Estimation Optimal cost function related to noise model Assume zero-mean isotropic Gaussian noise (assume outliers removed) Error in one image Maximum Likelihood Estimate

Statistical cost function and Maximum Likelihood Estimation Optimal cost function related to noise model Assume zero-mean isotropic Gaussian noise (assume outliers removed) Error in both images Maximum Likelihood Estimate

Error in two images (independent) Mahalanobis distance General Gaussian case Measurement X with covariance matrix Σ Error in two images (independent) Varying covariances

Next class: Parameter estimation (continued) Transformation invariance and normalization Iterative minimization Robust estimation

Upcoming assignment Take two or more photographs taken from a single viewpoint Compute panorama Use different measures DLT, MLE Use Matlab Due Feb. 13