Camera Calibration class 9 Multiple View Geometry Comp 290-089 Marc Pollefeys.

Slides:



Advertisements
Similar presentations
Projective 3D geometry class 4
Advertisements

Vanishing points  .
Camera Calibration.
Epipolar Geometry.
3D reconstruction.
Computing 3-view Geometry Class 18
Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer.
Multiple View Reconstruction Class 24 Multiple View Geometry Comp Marc Pollefeys.
Self-calibration.
Camera Models CMPUT 498/613 Richard Hartley and Andrew Zisserman, Multiple View Geometry, Cambridge University Publishers, 2000 Readings: HZ Ch 6, 7.
Epipolar Geometry class 11 Multiple View Geometry Comp Marc Pollefeys.
Structure from motion.
Scene Planes and Homographies class 16 Multiple View Geometry Comp Marc Pollefeys.
More on single-view geometry class 10 Multiple View Geometry Comp Marc Pollefeys.
3D reconstruction class 11
Projective 2D geometry (cont’) course 3
Used slides/content with permission from
Parameter estimation class 5 Multiple View Geometry Comp Marc Pollefeys.
Computing F and rectification class 14 Multiple View Geometry Comp Marc Pollefeys.
Structure from motion. Multiple-view geometry questions Scene geometry (structure): Given 2D point matches in two or more images, where are the corresponding.
Uncalibrated Geometry & Stratification Sastry and Yang
Parameter estimation class 6 Multiple View Geometry Comp Marc Pollefeys.
Computer Vision : CISC4/689
Camera models and single view geometry. Camera model.
Self-calibration Class 21 Multiple View Geometry Comp Marc Pollefeys.
Multiple View Geometry Comp Marc Pollefeys
3D photography Marc Pollefeys Fall 2007
Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai.
Multiple View Geometry in Computer Vision
Cameras and Projections Dan Witzner Hansen Course web page:
Single View Metrology Class 3. 3D photography course schedule (tentative) LectureExercise Sept 26Introduction- Oct. 3Geometry & Camera modelCamera calibration.
CMPUT 412 3D Computer Vision Presented by Azad Shademan Feb , 2007.
Assignment 2 Compute F automatically from image pair (putative matches, 8-point, 7-point, iterative, RANSAC, guided matching) (due by Wednesday 19/03/03)
Camera Models class 8 Multiple View Geometry Comp Marc Pollefeys.
 -Linearities and Multiple View Tensors Class 19 Multiple View Geometry Comp Marc Pollefeys.
Camera calibration and single view metrology Class 4 Read Zhang’s paper on calibration
Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai.
More on single-view geometry class 10 Multiple View Geometry Comp Marc Pollefeys.
Multiple View Reconstruction Class 23 Multiple View Geometry Comp Marc Pollefeys.
3D Urban Modeling Marc Pollefeys Jan-Michael Frahm, Philippos Mordohai Fall 2006 / Comp Tue & Thu 14:00-15:15.
Camera models and calibration Read tutorial chapter 2 and 3.1 Szeliski’s book pp
Algorithm Evaluation and Error Analysis class 7 Multiple View Geometry Comp Marc Pollefeys.
3D photography Marc Pollefeys Fall 2004 / Comp Tue & Thu 9:30-10:45
Projective 2D geometry course 2 Multiple View Geometry Comp Marc Pollefeys.
Computer Vision Calibration Marc Pollefeys COMP 256 Read F&P Chapter 2 Some slides/illustrations from Ponce, Hartley & Zisserman.
Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai.
The Trifocal Tensor Class 17 Multiple View Geometry Comp Marc Pollefeys.
Geometria della telecamera. the image of a point P belongs to the line (P,O) p P O p = image of P = image plane ∩ line(O,P) interpretation line of p:
Projective 2D geometry course 2 Multiple View Geometry Comp Marc Pollefeys.
Epipolar geometry The fundamental matrix and the tensor
Camera Calibration class 9 Multiple View Geometry Comp Marc Pollefeys.
Course 12 Calibration. 1.Introduction In theoretic discussions, we have assumed: Camera is located at the origin of coordinate system of scene.
Camera Model & Camera Calibration
Parameter estimation. 2D homography Given a set of (x i,x i ’), compute H (x i ’=Hx i ) 3D to 2D camera projection Given a set of (X i,x i ), compute.
Single-view geometry Odilon Redon, Cyclops, 1914.
ROBOT VISION Lesson 4: Camera Models and Calibration Matthias Rüther Slides partial courtesy of Marc Pollefeys Department of Computer Science University.
Single-view geometry Odilon Redon, Cyclops, 1914.
More on single-view geometry class 10 Multiple View Geometry Comp Marc Pollefeys.
Camera Models class 8 Multiple View Geometry Comp Marc Pollefeys.
Parameter estimation class 5 Multiple View Geometry CPSC 689 Slides modified from Marc Pollefeys’ Comp
Projective 2D geometry course 2 Multiple View Geometry Comp Marc Pollefeys.
Calibration ECE 847: Digital Image Processing Stan Birchfield Clemson University.
Parameter estimation class 5
Epipolar Geometry class 11
Multiple View Geometry Comp Marc Pollefeys
More on single-view geometry class 10
Multiple View Geometry Comp Marc Pollefeys
3D reconstruction class 11
Camera Calibration class 9
Parameter estimation class 6
Presentation transcript:

Camera Calibration class 9 Multiple View Geometry Comp Marc Pollefeys

Content Background: Projective geometry (2D, 3D), Parameter estimation, Algorithm evaluation. Single View: Camera model, Calibration, Single View Geometry. Two Views: Epipolar Geometry, 3D reconstruction, Computing F, Computing structure, Plane and homographies. Three Views: Trifocal Tensor, Computing T. More Views: N-Linearities, Multiple view reconstruction, Bundle adjustment, auto- calibration, Dynamic SfM, Cheirality, Duality

Multiple View Geometry course schedule (subject to change) Jan. 7, 9Intro & motivationProjective 2D Geometry Jan. 14, 16(no class)Projective 2D Geometry Jan. 21, 23Projective 3D Geometry(no class) Jan. 28, 30Parameter Estimation Feb. 4, 6Algorithm EvaluationCamera Models Feb. 11, 13Camera CalibrationSingle View Geometry Feb. 18, 20Epipolar Geometry3D reconstruction Feb. 25, 27Fund. Matrix Comp.Structure Comp. Mar. 4, 6Planes & HomographiesTrifocal Tensor Mar. 18, 20Three View ReconstructionMultiple View Geometry Mar. 25, 27MultipleView ReconstructionBundle adjustment Apr. 1, 3Auto-CalibrationPapers Apr. 8, 10Dynamic SfMPapers Apr. 15, 17CheiralityPapers Apr. 22, 24DualityProject Demos

Single view geometry Camera model Camera calibration Single view geom.

Pinhole camera

Camera anatomy Camera center Column points Principal plane Axis plane Principal point Principal ray

Affine cameras

Decomposition of P ∞ absorb d 0 in K 2x2 alternatives, because 8dof (3+3+2), not more

Summary parallel projection canonical representation calibration matrix principal point is not defined

A hierarchy of affine cameras Orthographic projection Scaled orthographic projection (5dof) (6dof)

A hierarchy of affine cameras Weak perspective projection (7dof)

1.Affine camera=camera with principal plane coinciding with  ∞ 2.Affine camera maps parallel lines to parallel lines 3.No center of projection, but direction of projection P A D=0 (point on  ∞ ) A hierarchy of affine cameras Affine camera (8dof)

Pushbroom cameras Straight lines are not mapped to straight lines! (otherwise it would be a projective camera) (11dof)

Line cameras (5dof) Null-space PC=0 yields camera center Also decomposition

Camera calibration

Resectioning

Basic equations

minimal solution Over-determined solution  5½ correspondences needed (say 6) P has 11 dof, 2 independent eq./points n  6 points minimize subject to constraint

Degenerate configurations More complicate than 2D case (see Ch.21) (i)Camera and points on a twisted cubic (ii)Points lie on plane or single line passing through projection center

Less obvious (i) Simple, as before (ii) Anisotropic scaling Data normalization

Line correspondences Extend DLT to lines (back-project line) (2 independent eq.)

Geometric error

Gold Standard algorithm Objective Given n≥6 2D to 2D point correspondences {X i ↔x i ’}, determine the Maximum Likelyhood Estimation of P Algorithm (i)Linear solution: (a)Normalization: (b)DLT: (ii)Minimization of geometric error: using the linear estimate as a starting point minimize the geometric error: (iii)Denormalization: ~~ ~

Calibration example (i)Canny edge detection (ii)Straight line fitting to the detected edges (iii)Intersecting the lines to obtain the images corners typically precision <1/10 (HZ rule of thumb: 5 n constraints for n unknowns

Errors in the world Errors in the image and in the world

Geometric interpretation of algebraic error note invariance to 2D and 3D similarities given proper normalization

Estimation of affine camera note that in this case algebraic error = geometric error

Gold Standard algorithm Objective Given n≥4 2D to 2D point correspondences {X i ↔x i ’}, determine the Maximum Likelyhood Estimation of P (remember P 3T =(0,0,0,1)) Algorithm (i)Normalization: (ii)For each correspondence (iii)solution is (iv)Denormalization:

Restricted camera estimation Minimize geometric error  impose constraint through parametrization  Image only  9   2n, otherwise  3n+9   5n Find best fit that satisfies skew s is zero pixels are square principal point is known complete camera matrix K is known Minimize algebraic error  assume map from param q  P=K[R|-RC], i.e. p=g(q)  minimize ||Ag(q)||

Reduced measurement matrix One only has to work with 12x12 matrix, not 2nx12

Restricted camera estimation Initialization Use general DLT Clamp values to desired values, e.g. s=0,  x =  y Note: can sometimes cause big jump in error Alternative initialization Use general DLT Impose soft constraints gradually increase weights

Exterior orientation Calibrated camera, position and orientation unkown  Pose estimation 6 dof  3 points minimal (4 solutions in general)

Covariance estimation ML residual error Example: n=197, =0.365, =0.37

Covariance for estimated camera Compute Jacobian at ML solution, then (variance per parameter can be found on diagonal) (chi-square distribution =distribution of sum of squares) cumulative -1

short and long focal length Radial distortion

Correction of distortion Choice of the distortion function and center Computing the parameters of the distortion function (i)Minimize with additional unknowns (ii)Straighten lines (iii)…

Next class: More Single-View Geometry Projective cameras and planes, lines, conics and quadrics. Camera calibration and vanishing points, calibrating conic and the IAC