Imaging Geometry for the Pinhole Camera Outline: Motivation |The pinhole camera

Example 1: Self-Localisation View 3 View 2 View 1

Example 2: Build a Panorama (register many images into a common frame) M. Brown and D. G. Lowe. Recognising Panoramas. ICCV 2003

Example 3: 3D Reconstruction: Detect Correspondences and triangulate

Example 4: Camera motion tracking ⇒ image stabilization background part of the image registered original stabilized

Example 5: Medical imaging – non-rigid image registration for change detection from the atlas test slice deform. field before registrationafter

Unis, 3 Oct Example 6: Recognition and Localisation of Objects Object Models: What objects are in the image? Where are they?

Unis, 3 Oct Example 7: Inspection and visual measurement ( in the registered view angles and lengths can be checked )

Imaging Geometry: Pinhole Camera Model This part of the talk follows A. Zisserman’s EPSRC9 tutorial Image formation by common cameras is well modeled by a perspective projection: If expressed as a linear mapping between homogeneous coordinates: 9

Imaging Geometry: Internal camera parameters C is the camera calibration matrix. (u 0, v 0 ) is the principal point, the intersection of the optical axis and the image plane  u =f k u,  v = f k v define scaling in x and y directions Moving from image plane (x,y) to (u,v) pixel coordinates:

Imaging Geometry: From World to Camera Coordinates The Euclidean transformation (rigid motion of the camera) is described by X c = R X w + T. Chaining all the transformations: This defines a 3x4 projection matrix P from Euclidean 3-space to an image:

Imaging Geometry: Plane projective transformations Choose the world coordinates so that the plane of the points has zero Z coordinate. The 3x4 projection matrix P reduces to:

Image Geometry: Computing Plane Projective Transform 1 The plane projective transform is called a homography Four point-to-point correspondences define a homography From the model of pinhole camera, we know the form ( » denotes similarity up to scale): or, equivalently:

Image Geometry: Computing Plane Projective Transform 2 Multiplying out: Each point correspondence defines two constraints: Two approaches can be used to address the scale ambiguity. We will use the simpler one that sets h 33 =1. This is OK unless points at infinity are involved

Image Geometry: Computing Plane Projective Transform 3 The constrains from four points can be expressed as a linear (in unknowns h ij ) into an 8x8 matrix:

Removing Perspective Distortion 1.Have coordinates of four points on the object plane 2.Solve for H in x’=Hx from the and corresponding image coordinates. 3.Then x=H -1 x’ 4.(E.g.) inspect the part, checking distances or angle

Taxonomy of planar projective transforms II Notes: Properties of the more general transforms are inherited by transformations lower in the table R = [r ij ] is a rotation matrix, i.e. R R > =1, also

Taxonomy of planar projective transforms I In many circumstances, we know from the imaging set- up, that the image-to-image transformation is simpler than homography or can be well approximated by a transformation with a lower number of degrees of freedom. Three types of transforms are commonly encountered: –Euclidean (shifted and rotated, e.g. two flatbed scans of the same image ) –Similarity (shift, rotation, isotropic scaling, e.g. two photos from the same spot with different zoom) –Affine transformation

Image Geometry: Computing Affine Transform An affine transform is defined as: Each point-to-point correspondence provides to constraints, 3 correspondences are needed to uniquely define the transformation. Solving the problem requires inversion of a single 3x3 matrix:

Unis, 3 Oct