Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai

2 Outline 2-D Projective geometry 3-D Projective geometry Chapters 2,3 and 6 of “Multiple View Geometry in Computer Vision” by Hartley and Zisserman

3 Points, lines & conics (last week) Transformations & invariants Projective 2D Geometry

4 Projective transformations A projectivity is an invertible mapping h from P 2 to itself such that three points x 1,x 2,x 3 lie on the same line if and only if h(x 1 ),h(x 2 ),h(x 3 ) do. Definition: A mapping h : P 2  P 2 is a projectivity if and only if there exist a non-singular 3x3 matrix H such that for any point in P 2 reprented by a vector x it is true that h(x)=Hx Theorem: Definition: Projective transformation or 8DOF projectivity=collineation=projective transformation=homography

5 Mapping between planes central projection may be expressed by x’=Hx (application of theorem)

6 Removing projective distortion select four points in a plane with known coordinates (linear in h ij ) (2 constraints/point, 8DOF  4 points needed) Remark: no calibration at all necessary, better ways to compute (see later)

7 Transformation of lines and conics Transformation for lines Transformation for conics Transformation for dual conics For a point transformation

8 A hierarchy of transformations Projective linear group Affine group (last row (0,0,1)) Euclidean group (upper left 2x2 orthogonal) Oriented Euclidean group (upper left 2x2 det 1) Alternative, characterize transformation in terms of elements or quantities that are preserved or invariant e.g. Euclidean transformations leave distances unchanged

9 Class I: Isometries (iso=same, metric=measure) orientation preserving: orientation reversing: special cases: pure rotation, pure translation 3DOF (1 rotation, 2 translation) Invariants: length, angle, area

10 Class II: Similarities (isometry + scale) also know as equi-form (shape preserving) metric structure = structure up to similarity (in literature) 4DOF (1 scale, 1 rotation, 2 translation) Invariants: ratios of length, angle, ratios of areas, parallel lines

11 Class III: Affine transformations non-isotropic scaling! (2DOF: scale ratio and orientation) 6DOF (2 scale, 2 rotation, 2 translation) Invariants: parallel lines, ratios of parallel lengths, ratios of areas

12 Class VI: Projective transformations Action non-homogeneous over the plane 8DOF (2 scale, 2 rotation, 2 translation, 2 line at infinity) Invariants: cross-ratio of four points on a line (ratio of ratios)

13 Action of affinities and projectivities on line at infinity Line at infinity becomes finite, allows to observe vanishing points, horizon Line at infinity stays at infinity, but points move along line

14 Decomposition of projective transformations upper-triangular, decomposition unique (if chosen s>0) Example:

15 Overview of transformations Projective 8dof Affine 6dof Similarity 4dof Euclidean 3dof Concurrency, collinearity, order of contact (intersection, tangency, inflection, etc.), cross ratio Parallellism, ratio of areas, ratio of lengths on parallel lines (e.g midpoints), linear combinations of vectors (centroids). The line at infinity l ∞ Ratios of lengths, angles. The circular points I,J lengths, areas.

16 Projective 3D Geometry Points, lines, planes and quadrics Transformations П ∞, ω ∞ and Ω ∞

17 Hierarchy of transformations Projective 15dof Affine 12dof Similarity 7dof Euclidean 6dof Intersection and tangency Parallellism of planes, Volume ratios, centroids, The plane at infinity π ∞ The absolute conic Ω ∞ Volume

18 The plane at infinity The plane at infinity π  is a fixed plane under a projective transformation H iff H is an affinity 1.canical position 2.contains all vanishing points 3.two planes are parallel  line of intersection in π ∞ 4.line // line (or plane)  point of intersection in π ∞ 5.fixed as set under affinities 6.Other planes may be fixed under some affinities, but π ∞ is fixed under all affinities

19 The absolute conic The absolute conic Ω ∞ is a fixed conic under the projective transformation H iff H is a similarity The absolute conic Ω ∞ is a (point) conic on π . In a metric frame: or conic for directions: (with no real points) 1.Ω ∞ is only fixed as a set 2.Circles intersect Ω ∞ in two points 3.Spheres intersect π ∞ in Ω ∞

20 The absolute dual quadric The absolute conic Ω * ∞ is a fixed conic under the projective transformation H iff H is a similarity 1.8 dof 2.plane at infinity π ∞ is the nullvector of Ω ∞

21 Outline 2-D Projective geometry 3-D Projective geometry Camera model re-visited

22 linear projection in homogeneous coordinates! Pinhole camera model

23 Intrinsic parameters Camera deviates from pinhole s: skew f x ≠ f y : different magnification in x and y (c x c y ): optical axis does not pierce image plane exactly at the center Usually: rectangular pixels: square pixels: principal point known: or

24 Extrinsic parameters Scene motion Camera motion

25 Camera rotation and translation

26 non-singular 11 dof (5+3+3) decompose P in K,R,C? {finite cameras}={P 3x4 | det M≠0} If rank P=3, but rank M<3, then cam at infinity Finite projective camera

27 Camera center Column points Principal plane Axis plane Principal point Principal ray Camera anatomy

28 null-space of camera projection matrix For all A all points on AC project on image of A, therefore C is camera center Image of camera center is (0,0,0) T, i.e. undefined Finite cameras: Infinite cameras: Camera center

29 Image points corresponding to X,Y,Z directions and origin Column vectors

30 note: p 1,p 2 dependent on image reparametrization Row vectors

31 principal point The principal point

32 Forward projection Back-projection (pseudo-inverse) Action of projective camera on point D=[d 1 d 2 d 3 0] T

33 Finding the camera center (use SVD to find null-space) Finding the camera orientation and internal parameters (use RQ decomposition ~QR) Q R =( ) -1 = Q R (if only QR, invert) Camera matrix decomposition

34 general projective interpretation Meaningfull decomposition in K,R,t requires Euclidean image and space Camera center is still valid in projective space Principal plane requires affine image and space Principal ray requires affine image and Euclidean space Euclidean vs. projective