Projective Geometry and Camera Models ENEE 731 Image Understanding Kaushik Mitra

Camera 3D to 2D mapping 2D to 3D mapping

Preliminaries Projective geometry Projective spaces (P2) Homogenous coordinates Projective transformations (Homography)

Why Projective Geometry? Parallel lines converge to a point 2D -2D transformation Projective transformation (Homography)

Projective space P2 Projective space: set of points Each point is a line in R3 passing through origin Motivated from camera geometry Alternative way P=(x,y,z) and P’=(x’,y’,z’) equivalent if and only if P=λP’ λ ≠ 0

P2=R2 U P1 (x, y, z) ≈ (λx, λy, λz) P2 can be divided into two subsets Points with z ≠ 0 Points with z= 0 If z ≠ 0, (x, y, z) ≈ (x/z, y/z, 1) one-to-one mapping with R2 If z=0 (x, y, 0) points at infinity line at infinity P1 P2 = R2 U P1 Homogenous coordinates P = (x, y, z) (up to a scale)

Projective Transformation (Homography) Invertible mapping h from P2 to P2 Lines maps to lines x1, x2, x3 lie on a line  h(x1), h(x2), h(x3) lie on a line

Homography matrix Theorem x’=Hx h:P2->P2 is homography  h(x)=Hx, H non-singular x’=Hx H, a homogenous matrix (up to scale) 8 dof

Hierarchy of Transformations Euclidean < Similarity < Affine < Projective Invariants Quantities that are preseved Euclidean: rotation and translation x’ = HEx = [R t; 0T 1] Invariants: length, angle, area

Hierarchy of Transformations Similarity: isotropic scale + (Euclidean) x’ = HSx = [sR t; 0T 1] Invariants: angle, ratios of length and area Affine: non-isotropic scales and skew x’ = HAx = [A t; 0T 1]; A non-singular Invariants: Parallel lines, ratio of areas Projective: x’ = HPx = [A t; vT u]x Note: parallel lines not preserved Invariants: Colinearity, cross-ratio

Generalization: Pn P3 = R3 + P2 P2: plane at infinity Homogenous coordinates: X= (X1, X2, X3, X4) Projective transformation X’ = HX

Camera Models

Camera Models Mapping 3D to 2D: Camera Matrices Central projection Pin-hole camera Finite projective camera Parallel projection Orthographic camera Affine camera

Pinhole Camera Geometry Camera center, C Image plane Principle axis Principle point Camera coordinate system C as origin Image plane at Z=f (X, Y, Z)T -> (fX/Z, fY/Z)T

Camera Matrix (X, Y, Z)T -> (fX/Z, fY/Z)T Homogenous coordinates (X, Y, Z, 1)T -> (fX, fY, Z)T Transformation: diag(f f 1)[I | 0] Camera projection matrix: x= PX P = diag(f, f, 1)[I | 0 ], a 3×4 matrix

Principle Point Offset (px, py): principle point (X, Y, Z) -> (fX/Z+px, fY/Z+py) x = K[I | 0]X Camera matrix: K[I | 0] K: Internal parameter matrix

Camera Rotation and Translation World coordinate system , in inhomogenous coordinates P = K[R | t] K: Internal parameters R,t: External parameters

Finite Projective Camera Generalize K αx, αy: unequal scale factors s: skew parameter P = [KR | Kt] = [M | p4] M = KR non-singular, rank 3 General projective cameras Homogenous 3×4 matrix of rank 3

General Projective Camera Given P, what can we say about the camera? Camera center? P is a 3×4 matrix PC=0 Consider the line joining C and A: X(λ) = λA + (1-λ)C x = PX(λ) = λPA C is the camera center

From 2D to 3D Given a point x, find the ray Two points on ray Camera center Another point P+x, where P+ is the pseudo-inverse X(λ) = λP+x + (1-λ)C

Parallel Projection Central Projection: P=[I | 0] Parallel Projection: Projection along Z-axis

Hierarchy of Parallel Projection Orthographic projection Scaled orthographic projection Weak perspective projection Affine projection

Orthographic Projection Projection along Z-axis Dof: 5

Generalization of Orthographic Projection (O.P.) Scaled orthographic projection O. P. followed by isotropic scaling Dof : 6 Weak perspective projection O.P. with non-isotropi c scaling Dof: 7 Affine camera (projection) Plus skew Dof: 8

Properties of Affine Camera Last row is (0 0 0 1) Parallel world line maps to parallel image lines Camera center at infinity

How to get an Affine Camera?

Computation of Camera Matrix P

Approach for Computing P x = PX Estimate P from 3D-2D correspondences Xi ↔ xi Compute K, R, t from P

Basic Equations for Computing P Each Xi ↔ xi satisfy xi = PXi (upto a scale) xi × PXi = 0 Linear in P Aip = 0, where p = vec(P) 2 linearly independent eqns. per corrs. P is a 3×4 homogenous matrix => 11 dofs # of correspondences ≥ 6

Computing P Want to solve: Ap = 0, with p≠0 In presence of noise, Eigen-value solution Algebraic cost Geometric cost: ML estimate of P Non-linear optimization: use Newton’s method

Summary for Computing P Form A from 3D-2D correspondences Normalization step (see reference 1) Solve algebraic cost Solve geometric cost starting from algebraic soln. Denormalization step (see reference 1)

Computing K from P P = [M | p4] We know P = K[R | t] = [KR | Kt] Decompose M using QR decomposition Get K and R Obtain t as K-1p4

Summary Projective geometry: P2 and P3 Camera models Central projection (Pin hole camera) Parallel projection (affine camera) Estimation of Camera Model P Estimation of internal parameter K

References 1) Multi-view Geometry (Ch 2, 6, 7) Hartley and Zisserman 2) Computer Vision: Algorithms and Applications Richard Szeliski

Principle Plane PX = (x, y, 0)T P3 represents principle plane => P3TX = 0, where P = [P1T; P2T; P3T] P3 represents principle plane