Multiview geometry ECE 847: Digital Image Processing Stan Birchfield Clemson University.

Slides:



Advertisements
Similar presentations
Epipolar Geometry.
Advertisements

Lecture 11: Two-view geometry
MASKS © 2004 Invitation to 3D vision Lecture 7 Step-by-Step Model Buidling.
Jan-Michael Frahm, Enrique Dunn Spring 2012
Two-view geometry.
Lecture 8: Stereo.
Camera calibration and epipolar geometry
3D Computer Vision and Video Computing 3D Vision Topic 3 of Part II Stereo Vision CSc I6716 Spring 2011 Zhigang Zhu, City College of New York
Structure from motion.
Computer Vision : CISC 4/689
Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.
Epipolar Geometry Class 7 Read notes Feature tracking run iterative L-K warp & upsample Tracking Good features Multi-scale Transl. Affine.
Stanford CS223B Computer Vision, Winter 2006 Lecture 5 Stereo I
Epipolar geometry. (i)Correspondence geometry: Given an image point x in the first view, how does this constrain the position of the corresponding point.
Structure from motion. Multiple-view geometry questions Scene geometry (structure): Given 2D point matches in two or more images, where are the corresponding.
Epipolar Geometry and the Fundamental Matrix F
Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai.
Computer Vision : CISC 4/689
3D Computer Vision and Video Computing 3D Vision Topic 4 of Part II Stereo Vision CSc I6716 Spring 2008 Zhigang Zhu, City College of New York
3D Computer Vision and Video Computing 3D Vision Lecture 15 Stereo Vision (II) CSC 59866CD Fall 2004 Zhigang Zhu, NAC 8/203A
3D Computer Vision and Video Computing 3D Vision Lecture 14 Stereo Vision (I) CSC 59866CD Fall 2004 Zhigang Zhu, NAC 8/203A
Projected image of a cube. Classical Calibration.
May 2004Stereo1 Introduction to Computer Vision CS / ECE 181B Tuesday, May 11, 2004  Multiple view geometry and stereo  Handout #6 available (check with.
Lec 21: Fundamental Matrix
CSE473/573 – Stereo Correspondence
Computer Vision : CISC 4/689 CREDITS Rasmussen, UBC (Jim Little), Seitz (U. of Wash.), Camps (Penn. State), UC, UMD (Jacobs), UNC, CUNY.
Stereo Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with slides by James Rehg and.
COMP322/S2000/L271 Stereo Imaging Ref.V.S.Nalwa, A Guided Tour of Computer Vision, Addison Wesley, (ISBN ) Slides are adapted from CS641.
Multiple View Geometry. THE GEOMETRY OF MULTIPLE VIEWS Reading: Chapter 10. Epipolar Geometry The Essential Matrix The Fundamental Matrix The Trifocal.
CS 558 C OMPUTER V ISION Lecture IX: Dimensionality Reduction.
3-D Scene u u’u’ Study the mathematical relations between corresponding image points. “Corresponding” means originated from the same 3D point. Objective.
Epipolar geometry Class 5. Geometric Computer Vision course schedule (tentative) LectureExercise Sept 16Introduction- Sept 23Geometry & Camera modelCamera.
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography.
CSE 6367 Computer Vision Stereo Reconstruction Camera Coordinate Transformations “Everything should be made as simple as possible, but not simpler.” Albert.
Multi-view geometry. Multi-view geometry problems Structure: Given projections of the same 3D point in two or more images, compute the 3D coordinates.
Automatic Camera Calibration
Epipolar Geometry and Stereo Vision Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 03/05/15 Many slides adapted from Lana Lazebnik,
Lecture 11 Stereo Reconstruction I Lecture 11 Stereo Reconstruction I Mata kuliah: T Computer Vision Tahun: 2010.
Multi-view geometry.
Lecture 12 Stereo Reconstruction II Lecture 12 Stereo Reconstruction II Mata kuliah: T Computer Vision Tahun: 2010.
Projective cameras Motivation Elements of Projective Geometry Projective structure from motion Planches : –
Epipolar geometry Epipolar Plane Baseline Epipoles Epipolar Lines
CS654: Digital Image Analysis Lecture 8: Stereo Imaging.
Stereo Vision ECE 847: Digital Image Processing Stan Birchfield Clemson University.
Multiview Geometry and Stereopsis. Inputs: two images of a scene (taken from 2 viewpoints). Output: Depth map. Inputs: multiple images of a scene. Output:
Stereo Course web page: vision.cis.udel.edu/~cv April 11, 2003  Lecture 21.
Binocular Stereo #1. Topics 1. Principle 2. binocular stereo basic equation 3. epipolar line 4. features and strategies for matching.
Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning.
EECS 274 Computer Vision Stereopsis.
Bahadir K. Gunturk1 Phase Correlation Bahadir K. Gunturk2 Phase Correlation Take cross correlation Take inverse Fourier transform  Location of the impulse.
Two-view geometry. Epipolar Plane – plane containing baseline (1D family) Epipoles = intersections of baseline with image planes = projections of the.
stereo Outline : Remind class of 3d geometry Introduction
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography.
Computer vision: models, learning and inference M Ahad Multiple Cameras
Stereo March 8, 2007 Suggested Reading: Horn Chapter 13.
Multiple View Geometry and Stereo. Overview Single camera geometry – Recap of Homogenous coordinates – Perspective projection model – Camera calibration.
Lec 26: Fundamental Matrix CS4670 / 5670: Computer Vision Kavita Bala.
Multi-view geometry. Multi-view geometry problems Structure: Given projections of the same 3D point in two or more images, compute the 3D coordinates.
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
Epipolar Geometry and Stereo Vision
Two-view geometry Computer Vision Spring 2018, Lecture 10
Epipolar geometry.
Epipolar Geometry class 11
3D Photography: Epipolar geometry
Two-view geometry.
Reconstruction.
Two-view geometry Epipolar geometry F-matrix comp. 3D reconstruction
Two-view geometry.
Two-view geometry.
Multi-view geometry.
Presentation transcript:

Multiview geometry ECE 847: Digital Image Processing Stan Birchfield Clemson University

Stereo geometry for pinhole cameras with flat retinas C,C’,x,x’ and X are coplanar Left cameraRight camera world point center of projection epipolar plane epipolar line for x epipole baseline M. Pollefeys,

epipoles e,e’ = intersection of baseline with image plane = projection of projection center in other image = vanishing point of camera motion direction an epipolar plane = plane containing baseline (1-D family) an epipolar line = intersection of epipolar plane with image (always come in corresponding pairs) Epipolar geometry M. Pollefeys,

What if only C,C’,x are known? Epipolar geometry epipole center of projection baseline All points on p project on l and l’ M. Pollefeys,

Family of planes  and lines l and l’ Intersection in e and e’ Epipolar geometry M. Pollefeys,

Example: Converging cameras M. Pollefeys,

e e’ Example: Forward motion M. Pollefeys,

Epipolar geometry and Fundamental matrix epipolar line (epipole: intersection of all epipolar lines) (computed with 20 points, non-normalized algorithm)

(computed with 50 points, non-normalized algorithm) Epipolar geometry and Fundamental matrix epipolar line (epipole: intersection of all epipolar lines)

Epipolar geometry and Fundamental matrix using OpenCV implementation best

Fundamental matrix point in image 1 point in image 2 fundamental matrix

Epipolar lines (1) epipolar line in image 2 associated with (x,y) in image 1

Epipolar lines (2) epipolar line in image 1 associated with (x’,y’) in image 2

Computing the fundamental matrix (Eight-point algorithm) known unknown

known unknown Computing the fundamental matrix (Eight-point algorithm)

1.Construct A (nx9) from correspondences 2.Compute SVD of A: A = U  V T 3.Unit-norm solution of Af=0 is given by v n (the right-most singular vector of A) 4.Reshape f into F 1 5.Compute SVD of F 1 : F 1 =U F  F V F T 6.Set  F (3,3)=0 to get  F ’ (The enforces rank(F)=2) 7.Reconstruct F=U F  F ’V F T 8.Now x T F is epipolar line in image 2, and Fx’ is epipolar line in image 1 Note: Need to normalize coordinates to get good results Computing the fundamental matrix (Eight-point algorithm)

Normalizing coordinates Be careful: order of x and x’ has switched!

Normalizing coordinates Step 1: Compute normalizing transformation matrices: Step 2: Normalize coordinates: Step 3: Use 8-point algorithm to compute : Step 4: Transform back to original coordinate system:

(simple for stereo  rectification) Example: Motion parallel to image plane M. Pollefeys,

Example: Motion parallel to image scanlines Epipoles are at infinity Scanlines are the epipolar lines In this case, the images are said to be “rectified” Tsukuba stereo images courtesy of Y. Ohta and Y. Nakamura at the University of Tsukuba

Standard (rectified) stereo geometry pure translation along X-axis M. Pollefeys,

Perspective projection X x X x M. Pollefeys,

Rectified geometry xLxL xRxR XLXL -X R b

Standard stereo geometry disparity is inversely proportional to depth stereo vision is less useful for distant objects M. Pollefeys,

Stereo camera configurations (Slide from Pascal Fua)

More cameras Multi-baseline stereo [Okutomi & Kanade]

Applications Take 3 pictures, reconstruct 3D geometry

3D coordinates of cardboard cutouts (u,v)  (x,y,z) (u,v 0 )  (x,y,0)(u 1,v 0 )  (x 1,y 1,0) z =√ (x 1 -x) 2 +(y 1 -y) 2 Assuming vertical object, vertical image plane, and unity aspect ratio: