1. Projective Geometry and Camera Models
ENEE 731 Image Understanding
Kaushik Mitra

2. Camera
3D to 2D mapping
2D to 3D mapping

3. Preliminaries Projective geometry Projective spaces (P2)
Homogenous coordinates
Projective transformations (Homography)

4. Why Projective Geometry?
Parallel lines converge to a point
2D -2D transformation
Projective transformation (Homography)

5. Projective space P2 Projective space: set of points
Each point is a line in R3 passing through origin
Motivated from camera geometry
Alternative way
P=(x,y,z) and P’=(x’,y’,z’) equivalent if and only if P=λP’
λ ≠ 0

6. P2=R2 U P1 (x, y, z) ≈ (λx, λy, λz) P2 can be divided into two subsets
Points with z ≠ 0
Points with z= 0
If z ≠ 0, (x, y, z) ≈ (x/z, y/z, 1)
one-to-one mapping with R2
If z=0 (x, y, 0)
points at infinity
line at infinity P1
P2 = R2 U P1
Homogenous coordinates P = (x, y, z) (up to a scale)

7. Projective Transformation (Homography)
Invertible mapping h from P2 to P2
Lines maps to lines
x1, x2, x3 lie on a line  h(x1), h(x2), h(x3) lie on a line

8. Homography matrix Theorem x’=Hx
h:P2->P2 is homography  h(x)=Hx, H non-singular
x’=Hx
H, a homogenous matrix (up to scale)
8 dof

9. Hierarchy of Transformations
Euclidean < Similarity < Affine < Projective
Invariants
Quantities that are preseved
Euclidean: rotation and translation
x’ = HEx = [R t; 0T 1]
Invariants: length, angle, area

10. Hierarchy of Transformations
Similarity: isotropic scale + (Euclidean)
x’ = HSx = [sR t; 0T 1]
Invariants: angle, ratios of length and area
Affine: non-isotropic scales and skew
x’ = HAx = [A t; 0T 1]; A non-singular
Invariants: Parallel lines, ratio of areas
Projective:
x’ = HPx = [A t; vT u]x
Note: parallel lines not preserved
Invariants: Colinearity, cross-ratio

11. Generalization: Pn P3 = R3 + P2
P2: plane at infinity
Homogenous coordinates: X= (X1, X2, X3, X4)
Projective transformation
X’ = HX

12. Camera Models

13. Camera Models Mapping 3D to 2D: Camera Matrices Central projection
Pin-hole camera
Finite projective camera
Parallel projection
Orthographic camera
Affine camera

14. Pinhole Camera Geometry
Camera center, C
Image plane
Principle axis
Principle point
Camera coordinate system
C as origin
Image plane at Z=f
(X, Y, Z)T -> (fX/Z, fY/Z)T

15. Camera Matrix (X, Y, Z)T -> (fX/Z, fY/Z)T Homogenous coordinates
(X, Y, Z, 1)T -> (fX, fY, Z)T
Transformation: diag(f f 1)[I | 0]
Camera projection matrix: x= PX
P = diag(f, f, 1)[I | 0 ], a 3×4 matrix

16. Principle Point Offset
(px, py): principle point
(X, Y, Z) -> (fX/Z+px, fY/Z+py)
x = K[I | 0]X
Camera matrix: K[I | 0]
K: Internal parameter matrix

17. Camera Rotation and Translation
World coordinate system
, in inhomogenous coordinates
P = K[R | t]
K: Internal parameters
R,t: External parameters

18. Finite Projective Camera
Generalize K
αx, αy: unequal scale factors
s: skew parameter
P = [KR | Kt] = [M | p4]
M = KR non-singular, rank 3
General projective cameras
Homogenous 3×4 matrix of rank 3

19. General Projective Camera
Given P, what can we say about the camera?
Camera center?
P is a 3×4 matrix
PC=0
Consider the line joining C and A:
X(λ) = λA + (1-λ)C
x = PX(λ) = λPA
C is the camera center

20. From 2D to 3D Given a point x, find the ray Two points on ray
Camera center
Another point
P+x, where P+ is the pseudo-inverse
X(λ) = λP+x + (1-λ)C

21. Parallel Projection Central Projection: P=[I | 0] Parallel Projection:
Projection along Z-axis

22. Hierarchy of Parallel Projection
Orthographic projection
Scaled orthographic projection
Weak perspective projection
Affine projection

23. Orthographic Projection
Projection along Z-axis
Dof: 5

24. Generalization of Orthographic Projection (O.P.)
Scaled orthographic projection
O. P. followed by isotropic scaling
Dof : 6
Weak perspective projection
O.P. with non-isotropi c scaling
Dof: 7
Affine camera (projection)
Plus skew
Dof: 8

25. Properties of Affine Camera
Last row is ( )
Parallel world line maps to parallel image lines
Camera center at infinity

26. How to get an Affine Camera?

27. Computation of Camera Matrix P

28. Approach for Computing P
x = PX
Estimate P from 3D-2D correspondences
Xi ↔ xi
Compute K, R, t from P

29. Basic Equations for Computing P
Each Xi ↔ xi satisfy
xi = PXi (upto a scale)
xi × PXi = 0
Linear in P
Aip = 0, where p = vec(P)
2 linearly independent eqns. per corrs.
P is a 3×4 homogenous matrix => 11 dofs
# of correspondences ≥ 6

30. Computing P Want to solve: Ap = 0, with p≠0 In presence of noise,
Eigen-value solution
Algebraic cost
Geometric cost:
ML estimate of P
Non-linear optimization: use Newton’s method

31. Summary for Computing P
Form A from 3D-2D correspondences
Normalization step (see reference 1)
Solve algebraic cost
Solve geometric cost starting from algebraic soln.
Denormalization step (see reference 1)

32. Computing K from P P = [M | p4] We know P = K[R | t] = [KR | Kt]
Decompose M using QR decomposition
Get K and R
Obtain t as K-1p4

33. Summary Projective geometry: P2 and P3 Camera models
Central projection (Pin hole camera)
Parallel projection (affine camera)
Estimation of Camera Model P
Estimation of internal parameter K

34. References 1) Multi-view Geometry (Ch 2, 6, 7)
Hartley and Zisserman
2) Computer Vision: Algorithms and Applications
Richard Szeliski

35. Principle Plane PX = (x, y, 0)T P3 represents principle plane
=> P3TX = 0, where P = [P1T; P2T; P3T]
P3 represents principle plane