Feature Matching

Feature Space Outlier Rejection

After Outlier Rejection

RANdom SAmple Consensus

RANSAC for Homography

Example: Panorama

Homography Transform,Warping

Image Warping

Forward Warping

Inverse Warping

Forward vs. Inverse Warping

Two View Geometry  When a camera changes position and orientation, the scene moves rigidly relative to the camera 3-D Scene u u’u’ Rotation + Translation

Two View Geometry (simple cases)  In two cases this results in homography: 1. Camera rotates around its focal point 2. The scene is planar Then: Point correspondence forms 1:1mapping depth cannot be recovered

The General Case: Epipolar Lines epipolar line

Epipolar Plane epipolar plane epipolar line Baseline P O O’

Epipole  Every plane through the baseline is an epipolar plane  It determines a pair of epipolar lines (one in each image)  Two systems of epipolar lines are obtained  Each system intersects in a point, the epipole  The epipole is the projection of the center of the other camera epipolar lines Baseline O O’

Epipolar Lines epipolar plane epipolar line Baseline P O O’ To define an epipolar plane, we define the plane through the two camera centers O and O’ and some point P. This can be written algebraically (in some world coordinates) as follows:

Essential Matrix (algebraic constraint between corresponding image points)  Set world coordinates around the first camera  What to do with O’P? Every rotation changes the observed coordinate in the second image  We need to de-rotate to make the second image plane parallel to the first  Replacing by image points

Essential Matrix (cont.)  Denote this by:  Then  Define  E is called the “essential matrix”

Properties of the Essential Matrix  E is homogeneous  9 parameters  E can be recovered up to scale using 8 points.  The constraint det E=0  7 points suffices  In fact, there are only 5 degrees of freedom in E, 3 for rotation 2 for translation (up to scale), determined by epipole

 Background The lens optical axis does not coincide with the sensor  We model this using a 3x3 matrix the Calibration matrix Camera Internal Parameters or Calibration matrix

Camera Calibration matrix  The difference between ideal sensor and the real one is modeled by a 3x3 matrix K:  (c x,c y ) camera center, (a x,a y ) pixel dimensions, b skew  We end with

Fundamental Matrix  F, is the fundamental matrix.

Properties of the Fundamental Matrix  F is homogeneous  9 parameters  F can be recovered up to scale using 8 points.  The constraint det F=0  7 points suffices

Epipolar Plane l’ l Baseline P O O’ Other derivations Hartley & Zisserman p. 223 x X’ e e’ e’

HomographyEpipolar Form ShapeOne-to-one mapConcentric epipolar lines D.o.f.88/5 F/E Eqs/pnt21 Minimal configuration 45+ (8, linear) Depth NoYes, up to scale Scene Planar (or no translation) 3D scene Two-views geometry Summary:

Stereo Vision  Objective: 3D reconstruction  Input: 2 (or more) images taken with calibrated cameras  Output: 3D structure of scene  Steps: Rectification Matching Depth estimation

Rectification Image Reprojection  reproject image planes onto common plane parallel to baseline Notice, only focal point of camera really matters (Seitz)

Rectification  Any stereo pair can be rectified by rotating and scaling the two image planes (=homography)  Images have to be rectified so that Image planes of cameras are parallel. Focal points are at same height. Focal lengths same.  Then, epipolar lines fall along the horizontal scan lines of the images

References   mography_estimation.pdf mography_estimation.pdf      463/2005_fall/www/Lectures/RANSAC.pdf 463/2005_fall/www/Lectures/RANSAC.pdf   2.ppt 2.ppt