1. Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai

2. 2 Outline 2-D Projective geometry 3-D Projective geometry Chapters 1,2 and 5 of “Multiple View Geometry in Computer Vision” by Hartley and Zisserman

3. 3 Points, lines & conics (last week) Transformations & invariants Projective 2D Geometry

4. 4 Projective transformations A projectivity is an invertible mapping h from P 2 to itself such that three points x 1,x 2,x 3 lie on the same line if and only if h(x 1 ),h(x 2 ),h(x 3 ) do. Definition: A mapping h : P 2  P 2 is a projectivity if and only if there exist a non-singular 3x3 matrix H such that for any point in P 2 reprented by a vector x it is true that h(x)=Hx Theorem: Definition: Projective transformation or 8DOF projectivity=collineation=projective transformation=homography

5. 5 Mapping between planes central projection may be expressed by x’=Hx (application of theorem)

6. 6 Removing projective distortion select four points in a plane with known coordinates (linear in h ij ) (2 constraints/point, 8DOF  4 points needed) Remark: no calibration at all necessary, better ways to compute (see later)

7. 7 More examples

8. 8 Transformation of lines and conics Transformation for lines Transformation for conics Transformation for dual conics For a point transformation

9. 9 A hierarchy of transformations Projective linear group Affine group (last row (0,0,1)) Euclidean group (upper left 2x2 orthogonal) Oriented Euclidean group (upper left 2x2 det 1) Alternative, characterize transformation in terms of elements or quantities that are preserved or invariant e.g. Euclidean transformations leave distances unchanged

10. 10 Class I: Isometries (iso=same, metric=measure) orientation preserving: orientation reversing: special cases: pure rotation, pure translation 3DOF (1 rotation, 2 translation) Invariants: length, angle, area

11. 11 Class II: Similarities (isometry + scale) also know as equi-form (shape preserving) metric structure = structure up to similarity (in literature) 4DOF (1 scale, 1 rotation, 2 translation) Invariants: ratios of length, angle, ratios of areas, parallel lines

12. 12 Class III: Affine transformations non-isotropic scaling! (2DOF: scale ratio and orientation) 6DOF (2 scale, 2 rotation, 2 translation) Invariants: parallel lines, ratios of parallel lengths, ratios of areas

13. 13 Class VI: Projective transformations Action non-homogeneous over the plane 8DOF (2 scale, 2 rotation, 2 translation, 2 line at infinity) Invariants: cross-ratio of four points on a line (ratio of ratios)

14. 14 Action of affinities and projectivities on line at infinity Line at infinity becomes finite, allows to observe vanishing points, horizon Line at infinity stays at infinity, but points move along line

15. 15 Decomposition of projective transformations upper-triangular, decomposition unique (if chosen s>0) Example:

16. 16 Overview of transformations Projective 8dof Affine 6dof Similarity 4dof Euclidean 3dof Concurrency, collinearity, order of contact (intersection, tangency, inflection, etc.), cross ratio Parallellism, ratio of areas, ratio of lengths on parallel lines (e.g midpoints), linear combinations of vectors (centroids). The line at infinity l ∞ Ratios of lengths, angles. The circular points I,J lengths, areas.

17. 17 Projective 3D Geometry Points, lines, planes and quadrics Transformations П ∞, ω ∞ and Ω ∞

18. 18 Hierarchy of transformations Projective 15dof Affine 12dof Similarity 7dof Euclidean 6dof Intersection and tangency Parallellism of planes, Volume ratios, centroids, The plane at infinity π ∞ The absolute conic Ω ∞ Volume

19. 19 The plane at infinity The plane at infinity π  is a fixed plane under a projective transformation H iff H is an affinity 1.canical position 2.contains all vanishing points 3.two planes are parallel  line of intersection in π ∞ 4.line // line (or plane)  point of intersection in π ∞ 5.fixed as set under affinities 6.Other planes may be fixed under some affinities, but π ∞ is fixed under all affinities

20. 20 The absolute conic The absolute conic Ω ∞ is a fixed conic under the projective transformation H iff H is a similarity The absolute conic Ω ∞ is a (point) conic on π . In a metric frame: or conic for directions: (with no real points) 1.Ω ∞ is only fixed as a set 2.Circles intersect Ω ∞ in two points 3.Spheres intersect π ∞ in Ω ∞

21. 21 The absolute dual quadric The absolute conic Ω * ∞ is a fixed conic under the projective transformation H iff H is a similarity 1.8 dof 2.plane at infinity π ∞ is the nullvector of Ω ∞

22. 22 Outline 2-D Projective geometry 3-D Projective geometry Camera model re-visited

23. 23 linear projection in homogeneous coordinates! Pinhole camera model

24. 24 Pinhole camera model

25. 25 principal point Pinhole point offset

26. 26 calibration matrix Pinhole point offset

27. 27 Camera rotation and translation

28. 28 CCD camera

29. 29 1  arctan(1/s) for CCD/CMOS, always s=0 Image from image, s≠0 possible (non coinciding principal axis) When is skew non-zero?

30. 30 non-singular 11 dof (5+3+3) decompose P in K,R,C? {finite cameras}={P 4x3 | det M≠0} If rank P=3, but rank M<3, then cam at infinity Finite projective camera

31. 31 Camera center Column points Principal plane Axis plane Principal point Principal ray Camera anatomy

32. 32 null-space camera projection matrix For all A all points on AC project on image of A, therefore C is camera center Image of camera center is (0,0,0) T, i.e. undefined Finite cameras: Infinite cameras: Camera center

33. 33 Image points corresponding to X,Y,Z directions and origin Column vectors

34. 34 note: p 1,p 2 dependent on image reparametrization Row vectors

35. 35 principal point The principal point

36. 36 Forward projection Back-projection (pseudo-inverse) Action of projective camera on point

37. 37 Finding the camera center (use SVD to find null-space) Finding the camera orientation and internal parameters (use RQ decomposition ~QR) Q R =( ) -1 = -1 -1 Q R (if only QR, invert) Camera matrix decomposition

38. 38 general projective interpretation Meaningfull decomposition in K,R,t requires Euclidean image and space Camera center is still valid in projective space Principal plane requires affine image and space Principal ray requires affine image and Euclidean space Euclidean vs. projective