Mixing Catadioptric and Perspective Cameras

Slides:



Advertisements
Similar presentations
Epipolar Geometry.
Advertisements

Lecture 11: Two-view geometry
3D reconstruction.
The Trifocal Tensor Multiple View Geometry. Scene planes and homographies plane induces homography between two views.
MASKS © 2004 Invitation to 3D vision Lecture 7 Step-by-Step Model Buidling.
Geometry 1: Projection and Epipolar Lines Introduction to Computer Vision Ronen Basri Weizmann Institute of Science.
Self-calibration.
Omnidirectional camera calibration
Dr. Hassan Foroosh Dept. of Computer Science UCF
Camera calibration and epipolar geometry
A Generic Concept for Camera Calibration Peter Sturm and Srikumar Ramaligam Sung Huh CPSC 643 Individual Presentation 4 April 15, 2009.
Structure from motion.
Self-calibration and multi-view geometry Class 10 Read Chapter 6 and 3.2.
3D reconstruction class 11
Parameter estimation class 5 Multiple View Geometry Comp Marc Pollefeys.
Epipolar geometry. (i)Correspondence geometry: Given an image point x in the first view, how does this constrain the position of the corresponding point.
Structure from motion. Multiple-view geometry questions Scene geometry (structure): Given 2D point matches in two or more images, where are the corresponding.
Uncalibrated Geometry & Stratification Sastry and Yang
Epipolar Geometry and the Fundamental Matrix F
Lecture 21: Multiple-view geometry and structure from motion
Multiple View Geometry
Multiple-view Reconstruction from Points and Lines
CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 1 The plan for today Leftovers and from last time Camera matrix Part A) Notation,
Lecture 20: Two-view geometry CS6670: Computer Vision Noah Snavely.
Projected image of a cube. Classical Calibration.
Lec 21: Fundamental Matrix
Multiple View Geometry
The Trifocal Tensor Class 17 Multiple View Geometry Comp Marc Pollefeys.
Multiple View Geometry. THE GEOMETRY OF MULTIPLE VIEWS Reading: Chapter 10. Epipolar Geometry The Essential Matrix The Fundamental Matrix The Trifocal.
Structure Computation. How to compute the position of a point in 3- space given its image in two views and the camera matrices of those two views Use.
3-D Scene u u’u’ Study the mathematical relations between corresponding image points. “Corresponding” means originated from the same 3D point. Objective.
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography.
CSE 6367 Computer Vision Stereo Reconstruction Camera Coordinate Transformations “Everything should be made as simple as possible, but not simpler.” Albert.
Automatic Camera Calibration
Computer vision: models, learning and inference
Lecture 11 Stereo Reconstruction I Lecture 11 Stereo Reconstruction I Mata kuliah: T Computer Vision Tahun: 2010.
Multi-view geometry.
Epipolar geometry The fundamental matrix and the tensor
1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained.
Projective cameras Motivation Elements of Projective Geometry Projective structure from motion Planches : –
Projective geometry ECE 847: Digital Image Processing Stan Birchfield Clemson University.
Lecture 04 22/11/2011 Shai Avidan הבהרה : החומר המחייב הוא החומר הנלמד בכיתה ולא זה המופיע / לא מופיע במצגת.
Projective cameras Trifocal tensors Euclidean/projective SFM Self calibration Line geometry Purely projective cameras Je ne suis pas la la semaine prochaine.
Multiview Geometry and Stereopsis. Inputs: two images of a scene (taken from 2 viewpoints). Output: Depth map. Inputs: multiple images of a scene. Output:
Two-view geometry Epipolar geometry F-matrix comp. 3D reconstruction
Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning.
Parameter estimation. 2D homography Given a set of (x i,x i ’), compute H (x i ’=Hx i ) 3D to 2D camera projection Given a set of (X i,x i ), compute.
December 12 th, 2001C. Geyer/K. Daniilidis GRASP Laboratory Slide 1 Structure and Motion from Uncalibrated Catadioptric Views Christopher Geyer and Kostas.
Geometry of Multiple Views
Feature Matching. Feature Space Outlier Rejection.
Multi-linear Systems and Invariant Theory
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography.
Computer vision: models, learning and inference M Ahad Multiple Cameras
Rectification of Stereo Images Милош Миловановић Урош Поповић.
Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : –
Parameter estimation class 5 Multiple View Geometry CPSC 689 Slides modified from Marc Pollefeys’ Comp
Lec 26: Fundamental Matrix CS4670 / 5670: Computer Vision Kavita Bala.
Projective 2D geometry course 2 Multiple View Geometry Comp Marc Pollefeys.
Computer vision: geometric models Md. Atiqur Rahman Ahad Based on: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince.
Multi-view geometry. Multi-view geometry problems Structure: Given projections of the same 3D point in two or more images, compute the 3D coordinates.
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
René Vidal and Xiaodong Fan Center for Imaging Science
A Unified Algebraic Approach to 2D and 3D Motion Segmentation
Parameter estimation class 5
Two-view geometry Computer Vision Spring 2018, Lecture 10
Epipolar geometry.
More on single-view geometry class 10
Omnidirectional epipolar geometry
3D reconstruction class 11
Reconstruction.
Presentation transcript:

Mixing Catadioptric and Perspective Cameras Peter Sturm INRIA Rhône-Alpes France

Introduction

Introduction

Introduction • Existing results - epipolar geometry between omnidirectional cameras - motion estimation - (self-) calibration - ...

Our Goals • Study the geometry of hybrid stereo systems (omnidirectional and perspective cameras) - epipolar geometry - trifocal tensors - plane homographies • Applications - motion estimation - calibration - self-calibration, calibration transfer - 3D reconstruction - ...

Plan  Camera models • Epipolar geometry of hybrid systems • Derivation of matching tensors • Applications • Conclusions

Camera Models Perspective and affine cameras

Camera Models Central catadioptric cameras • mirror (surface of revolution of a conic) • camera • virtual optical center

Camera Models Central catadioptric cameras • mirror (surface of revolution of a conic) • camera • virtual optical center • calibration

Camera Models Types of central catadioptric cameras • hyperbola + perspective camera • parabola + affine camera • ...

Plan • Camera models  Epipolar geometry of hybrid systems • Derivation of matching tensors • Applications • Conclusions

Epipolar Geometry epipolar plane d epipolar line d’ epipolar line q’ q epipole

Epipolar Geometry epipolar conic

Epipolar Geometry epipolar line epipole epipolar conic epipoles

Epipolar Geometry Example

Epipolar Geometry “ conic ~ F q’ “ There exists with concretely: Interpretation: epipolar line d’ q’ d q epipole epipolar conic epipoles purely perspective case

Epipolar Geometry There exists with Interpretation: “ conic ~ F q’ “ concretely: Interpretation: “lifted coordinates” epipolar line epipole epipolar conic epipoles

Epipolar Geometry Example

Epipolar Geometry BUT... Until now, linear epipolar relation only found for: • any combination of perspective, affine or para-catadioptric cameras (parabolic mirrors) Not yet for: • other omnidirectional cameras than para-catadioptric ones (e.g. based on hyperbolic mirrors)

Epipolar Geometry Special case: • combination of perspective and para-catadioptric cameras • epipolar conics are circles • F is of dimension 4x3 • the « lifted coordinates » are:

Epipolar Geometry Epipoles: • is of rank 2 • The epipole of the perspective camera is the right null-vector of F • F has a one-dimensional left null-space  the two epipoles of the catadioptric camera are the left null-vectors that are valid lifted coordinates (quadratic constraint):

Plan • Camera models • Epipolar geometry of hybrid systems  Derivation of matching tensors • Applications • Conclusions

Matching Tensors • Multi-linear relations between coordinates of correponding image points • Purely perspective case: derivation based on linear equations representing projections (3D  2D) • Here: equations for back-projection (2D  3D directions) - perspective cameras with

Matching Tensors • Linear equations representing back-projections - perspective cameras - para-catadioptric cameras • is of dimension 3x4 • it depends - on the mirror’s intrinsic parameters - on the affine camera’s intrinsic parameters

Matching Tensors • Putting the equations together

Matching Tensors • Putting the equations together This matrix has a kernel  its rank is lower than 6  its determinant is zero  bilinear equation on the coefficients of and

Matching Tensors • Straightforward extension to more than 2 views ...

Plan • Camera models • Epipolar geometry of hybrid systems • Derivation of matching tensors  Applications - Self-calibration of omnidirectional cameras from fundamental matrices - Calibration transfer from an omnidirectional to a perspective camera - Self-calibration of omnidirectional cameras from a plane homography • Conclusions

Applications Self-calibration of omnidirectional cameras from fundamental matrices: • para-catadioptric camera has 3 intrinsic parameters • representation as a 4-vector of homogeneous coordinates • this vector is in the left null-space of the fundamental matrix of this camera, defined with respect to any other camera (perspective, affine, catadioptric)  self-calibration is possible from two or more fundamental matrices

Applications Calibration transfer from an omnidirectional to a perspective camera: • input: - calibration of a para-catadioptric camera - fundamental matrix with a perspective camera  closed-form solution for the focal length of the perspective camera

Applications Self-calibration of omnidirectional cameras from a plane homography: • input: - plane homography H with a perspective camera  recovery of intrinsic parameters of para-catadioptric camera (given by null-vector of H)

Plan • Camera models • Epipolar geometry of hybrid systems • Derivation of matching tensors • Applications  Conclusions

Conclusions Open questions Perspectives • Multi-linear matching relations between perspective, affine and para-catadioptric cameras • Applications in calibration, self-calibration, motion estimation, 3D reconstruction, ... Open questions • Fundamental matrix etc. for hyper-catadioptric cameras ? • Plane homographies « for the inverse direction » ? Perspectives • Hybrid trifocal tensors for line images • Multi-view 3D reconstruction for hybrid systems

Mixing Catadioptric and Perspective Cameras Peter Sturm INRIA Rhône-Alpes France