CSE 6367 Computer Vision Stereo Reconstruction Camera Coordinate Transformations “Everything should be made as simple as possible, but not simpler.” Albert Einstein Farhad Kamangar Computer Science and Engineering Department The University of Texas at Arlington

Stereo Constraints (Review) p p’ ? Given p in left image, find the corresponding point p’ in right image be?

Stereo X1X1 Y1Y1 Z1Z1 O1O1 Image plane Focal plane P p p’ Y2Y2 X2X2 Z2Z2 O2O2 Epipolar Line Epipole

Stereo X1X1 Y1Y1 Z1Z1 O1O1 Image plane Focal plane P p p’ Y2Y2 X2X2 Z2Z2 O2O2 Epipolar Line Epipole All epipolar lines pass through the epipole

Stereo X1X1 Y1Y1 Z1Z1 O1O1 Image plane Focal plane P p p’ Y2Y2 X2X2 Z2Z2 O2O2 Epipolar Line Epipole All epipolar lines pass through the epipole

Stereo X1X1 Y1Y1 Z1Z1 O1O1 Image plane Focal plane P p p’ Y2Y2 X2X2 Z2Z2 O2O2 Epipolar Line Epipole T

Stereo X1X1 Y1Y1 Z1Z1 O1O1 Image plane Focal plane P p p’ Y2Y2 X2X2 Z2Z2 O2O2 Epipolar Line Epipole T

Stereo X1X1 Y1Y1 Z1Z1 O1O1 Image plane Focal plane P p p’ Y2Y2 X2X2 Z2Z2 O2O2 Epipolar Line Epipole T T=O 2 -O 1 O1, O2, p, and p’ are on the same plane

Stereo X1X1 Y1Y1 Z1Z1 O1O1 Image plane Focal plane P p p’ Y2Y2 X2X2 Z2Z2 O2O2 Epipolar Line Epipole T O1, O2, p, and p’ are on the same plane

Stereo X1X1 Y1Y1 Z1Z1 O1O1 Image plane Focal plane P p p’ Y2Y2 X2X2 Z2Z2 O2O2 Epipolar Line Epipole T E is Essential matrix

The Essential Matrix U and v are the coordinates of point p in image plane Equation of epipolar line which corresponds to point p

The Essential Matrix Essential Matrix: Based on the Relative Geometry of the Cameras Cameras are assumed to be calibrated Has five independent parameters

Fundamental Matrix Assuming that the image coordinates of points p and p’ are u and u’ F is the fundamental matrix

Fundamental Matrix Fundamental Matrix, F, is a 3 by 3 matrix (singular with rank 2) F has 7 parameters up to scale and can be estimated from 7 point correspondences Direct Simpler Method requires 8 correspondences

Estimating Fundamental Matrix

The 8-point Algorithm

Camera Parameters Assume a camera (any camera) where the Projection Reference Point ( PRP ), is located at: Normal vector to the projection plane (View Plane Normal VPN) is given as: The View Up (VUP) vector is given as:

Transform Camera to World The general form of the matrix that transforms the camera coordinate system to the world coordinate system is,

Two Cameras Now assume we have two cameras, left camera and right camera For left camera: For right camera:

Point in the World Assume a point in the homogenous world coordinate system:

The coordinates of this point in the left camera coordinate system is: The coordinates of this point in the right camera coordinate system is:

Left Camera to World The matrix that transforms the left camera coordinate system to the world coordinate system

Right Camera to World The matrix that transforms the right camera coordinate system to the world coordinate system

Right Camera to Left Camera The matrix that transforms the right camera coordinate system to the left camera coordinate system

Given point P in the world coordinate system, the coordinate of this point in the left camera coordinate system will be:

Assuming that the focal length of the camera is f, Assume that the image of the point P on the left image plane is P’. The coordinate of the point P’ in the left camera coordinate system will be:

Assume that the image of the point P on the right image plane is P’’. The coordinate of the point P’’ in the right camera coordinate system will be:

Problem: given all the camera parameters and only the x and y coordinates of the points P’ and P’’, find the coordinates of the point P in the world coordinate system. In other words; given the projections of the point P on the left and right image planes, find the coordinates of the original point P.

From Left Camera to Right Camera

Calculate Camera Coordinates (1)

Calculate Camera Coordinates (2)

Calculate Camera Coordinates (3)