1. Robot Vision SS 2008 Matthias Rüther 1 ROBOT VISION Lesson 2: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer Science University of North Carolina, Chapel Hill

2. Robot Vision SS 2008 Matthias Rüther 2 Content  Linear Algebra: basic vector and matrix operations  Background: Projective geometry (2D, 3D), Parameter estimation, Algorithm evaluation.  Single View: Camera model, Calibration, Single View Geometry.  Two Views: Epipolar Geometry, 3D reconstruction, Computing F, Computing structure, Plane and homographies.  Three Views: Trifocal Tensor, Computing T.  More Views: N-Linearities, Multiple view reconstruction, Bundle adjustment, auto-calibration, Dynamic SfM, Cheirality, Duality

3. Robot Vision SS 2008 Matthias Rüther 3 Vector Basics v x2x2 x1x1 v v‘ v+v‘ v w Φ

4. Robot Vision SS 2008 Matthias Rüther 4 Vector Spaces  Linear combination –Given vectors v 1,..., v k and scalars c 1,..., c k, the vector v  c 1 v 1  c 2 v 2 ...  c k v k is obtained as a linear combination of the vectors v 1,..., v k –Example: all vectors in R 3 are linear combinations of i  (1, 0, 0), j  (0, 1, 0), and k  (0, 0, 1) v  (a, b, c)  a(1, 0, 0)  b(0, 1, 0)  c(0, 0, 1)  Spanning –S=(v 1, v 2,..., v k ) spans a space W if every vector in W can be written as a linear combination of the vectors in S  Linear Indepedence –v1,..., vk is linearly indepedent if c 1 v 1  c 2 v 2 ...  c k v k  0 implies c 1  c 2 ...  c k  0

5. Robot Vision SS 2008 Matthias Rüther 5 Vector Spaces  Basis –S=(v 1,..., v k ) is a basis for a vector space W if v j are linearly indepedent S spans W ! S need not be orthogonal !  Dimension –All bases for W have the same number of vectors. –The dimension of W is the size of its sets of basis vectors.  Coordinates Relative to a Basis –If S=(v 1,..., v n ) is a basis for a vector space W, then any v  W has a unique vector expansion:

6. Robot Vision SS 2008 Matthias Rüther 6  Example: –In R 3 the vectors relative toand relative to are identical. Vector Spaces

7. Robot Vision SS 2008 Matthias Rüther 7 Matrix Basics  Addition/Subtraction C  A  B implies c ij  a ij  b ij C  A  B implies c ij  a ij  b ij  Multiplication C m  q = A m  n *B n  q implies  Determinants –det(A)=0  A is singular  A is not invertible –det(A- I) is characteristic polynomial, roots are the eigenvalues

8. Robot Vision SS 2008 Matthias Rüther 8 Matrix Basics  Inverse –AA  1 =A  1 A=I –Exists, iff det(A) != 0, A is not singular –Properties:  Pseudo-Inverse A   (A T A)  1 A T A + A=I  Rank Maximum number of linearly independent columns or rows. –If A is m  n, rank(A) <= min(m,n) –If A is n  n, rank(A) = n iff A nonsingular (invertible, det(A)!=0) –If A is n  n, rank(A) < n iff A singular

9. Robot Vision SS 2008 Matthias Rüther 9 Solving Linear Equation Systems  Common problem: solve the equation system A m  n x=b –If m>n, the system is over-determined –If m<n, the system is under-determined –If b=0, the system is homogeneous –The system has either no solutions, exactly one or infinitely many –Methods for solving A m  n x=b include Gauss elimination, LU factorization, computing pseudo-inverse using SVD etc.  Conditions for solutions of Ax=b (A is square) –If A is invertible, Ax=b has exactly one solution. –A is invertible  Ax=0 has only trivial solution –If rank(A|b)>rank(A), the system has no solution –If rank(A)<n, and rank(A|b)=rank(A), the system has infinite number of solutions –Ax=0 has non-trivial solutions iff rank(A)<n

10. Robot Vision SS 2008 Matthias Rüther 10  Points, lines & conics  Transformations & invariants  1D projective geometry and the Cross-ratio Projective 2D Geometry

11. Robot Vision SS 2008 Matthias Rüther 11  Euclidean 1D vector space: –Geometric Primitives: points –Representation: 1D vector: p = (x) –Transformations: Translation: p‘ = (x+t) Scaling: p‘ = (s*x) Translation and Scaling: p‘ = (s*x+t) 1D Projective Geometry ℝ1ℝ1 p1p1 p2p2 p3p3 p4p4 x p1p1 p2p2 p3p3 p4p4 x p‘ 1 p‘ 2 p‘ 3 p‘ 4 x p‘ 1 p‘ 2 p‘ 3 p‘ 4 x d p1p2

12. Robot Vision SS 2008 Matthias Rüther 12  1D Projective space: 1D Projective Geometry ℙ1ℙ1 p1p1 p2p2 p3p3 p4p4 x w W=1 Representation: 1D vector: p‘ = (x‘ w‘) T, where p = (x) = (x‘/w‘) e.g. p‘ = (2 1) T = (4 2) T = (1 0,5) T are all equivalent to p = (2)

13. Robot Vision SS 2008 Matthias Rüther 13 1D Projective Geometry  1D Projective space: –Geometric Primitives: points –Representation: 1D vector: p = (x w) T –Transformations: Translation: p‘ = T * p1DOF Scaling: p‘ = S * p1DOF Translation and Scaling: p‘ = M * p, where M=S*T 2DOF Projective Mapping: p‘ = P * p, where P is a 2x2 Matrix 3DOF

14. Robot Vision SS 2008 Matthias Rüther 14 1D Projective Geometry  Translation: x w W=1 p‘ 1 p‘ 2 p‘ 3 p‘ 4

15. Robot Vision SS 2008 Matthias Rüther 15 1D Projective Geometry  Scaling: x w W=1 p‘ 1 p‘ 2 p‘ 3 p‘ 4

16. Robot Vision SS 2008 Matthias Rüther 16 1D Projective Geometry  Projective Mapping: x w W=1 p‘ 1 p‘ 2 p‘ 3 p‘ 4 Analogy to central projective camera!

17. Robot Vision SS 2008 Matthias Rüther 17 1D Projective Geometry  Invariants: –transformations form a hierarchy, some geometric features remain unchanged under the transformations –Translation (isometry) Length, overall scale –Translation and Scaling (similarity) Ratio of lengths (d12 : d23) –Projective Mapping (projectivity) Cross ratio (ratio of ratios): d12*d34 = d13*d24

18. Robot Vision SS 2008 Matthias Rüther 18 Homogeneous coordinates Homogeneous representation of lines equivalence class of vectors, any vector is representative Set of all equivalence classes in R 3  (0,0,0) T forms P 2 Homogeneous representation of points on if and only if The point x lies on the line l if and only if x T l = l T x = 0 Homogeneous coordinates Inhomogeneous coordinates but only 2DOF

19. Robot Vision SS 2008 Matthias Rüther 19 Points from lines and vice-versa Intersections of lines The intersection of two lines and is Line joining two points The line through two points and is Example

20. Robot Vision SS 2008 Matthias Rüther 20 Ideal points and the line at infinity Intersections of parallel lines Example Ideal points Line at infinity tangent vector normal direction Note that in P 2 there is no distinction between ideal points and others

21. Robot Vision SS 2008 Matthias Rüther 21 A model for the projective plane exactly one line through two points exaclty one point at intersection of two lines

22. Robot Vision SS 2008 Matthias Rüther 22 Duality Duality principle: To any theorem of 2-dimensional projective geometry there corresponds a dual theorem, which may be derived by interchanging the role of points and lines in the original theorem

23. Robot Vision SS 2008 Matthias Rüther 23 Conics Curve described by 2 nd -degree equation in the plane or homogenized or in matrix form with 5DOF:

24. Robot Vision SS 2008 Matthias Rüther 24 Five points define a conic For each point the conic passes through or stacking constraints yields

25. Robot Vision SS 2008 Matthias Rüther 25 Tangent lines to conics The line l tangent to C at point x on C is given by l=Cx l x C

26. Robot Vision SS 2008 Matthias Rüther 26 Dual conics A line tangent to the conic C satisfies Dual conics = line conics = conic envelopes In general ( C full rank):

27. Robot Vision SS 2008 Matthias Rüther 27 Degenerate conics A conic is degenerate if matrix C is not of full rank e.g. two lines (rank 2) e.g. repeated line (rank 1) Degenerate line conics: 2 points (rank 2), double point (rank1) Note that for degenerate conics

28. Robot Vision SS 2008 Matthias Rüther 28 Projective transformations A projectivity is an invertible mapping h from P 2 to itself such that three points x 1,x 2,x 3 lie on the same line if and only if h(x 1 ),h(x 2 ),h(x 3 ) do. Definition: A mapping h : P 2  P 2 is a projectivity if and only if there exist a non-singular 3x3 matrix H such that for any point in P 2 reprented by a vector x it is true that h(x)=Hx Theorem: Definition: Projective transformation or 8DOF projectivity=collineation=projective transformation=homography

29. Robot Vision SS 2008 Matthias Rüther 29 Mapping between planes central projection may be expressed by x’=Hx (application of theorem)

30. Robot Vision SS 2008 Matthias Rüther 30 Removing projective distortion select four points in a plane with known coordinates (linear in h ij ) (2 constraints/point, 8DOF  4 points needed) Remark: no calibration at all necessary, better ways to compute (see later)

31. Robot Vision SS 2008 Matthias Rüther 31 More examples

32. Robot Vision SS 2008 Matthias Rüther 32 Transformation of lines and conics Transformation for lines Transformation for conics Transformation for dual conics For a point transformation

33. Robot Vision SS 2008 Matthias Rüther 33 A hierarchy of transformations Projective linear group Affine group (last row (0,0,1)) Euclidean group (upper left 2x2 orthogonal) Oriented Euclidean group (upper left 2x2 det 1) Alternative, characterize transformation in terms of elements or quantities that are preserved or invariant e.g. Euclidean transformations leave distances unchanged

34. Robot Vision SS 2008 Matthias Rüther 34 Class I: Isometries (iso=same, metric=measure) orientation preserving: orientation reversing: special cases: pure rotation, pure translation 3DOF (1 rotation, 2 translation) Invariants: length, angle, area

35. Robot Vision SS 2008 Matthias Rüther 35 Class II: Similarities (isometry + scale) also know as equi-form (shape preserving) metric structure = structure up to similarity (in literature) 4DOF (1 scale, 1 rotation, 2 translation) Invariants: ratios of length, angle, ratios of areas, parallel lines

36. Robot Vision SS 2008 Matthias Rüther 36 Class III: Affine transformations non-isotropic scaling! (2DOF: scale ratio and orientation) 6DOF (2 scale, 2 rotation, 2 translation) Invariants: parallel lines, ratios of parallel lengths, ratios of areas

37. Robot Vision SS 2008 Matthias Rüther 37 Class VI: Projective transformations Action non-homogeneous over the plane 8DOF (2 scale, 2 rotation, 2 translation, 2 line at infinity) Invariants: cross-ratio of four points on a line (ratio of ratio)

38. Robot Vision SS 2008 Matthias Rüther 38 Action of affinities and projectivities on line at infinity Line at infinity becomes finite, allows to observe vanishing points, horizon, Line at infinity stays at infinity, but points move along line

39. Robot Vision SS 2008 Matthias Rüther 39 Overview of transformations Projective 8dof Affine 6dof Similarity 4dof Euclidean 3dof Concurrency, collinearity, order of contact (intersection, tangency, inflection, etc.), cross ratio Parallellism, ratio of areas, ratio of lengths on parallel lines (e.g midpoints), linear combinations of vectors (centroids). The line at infinity l ∞ Ratios of lengths, angles. The circular points I,J lengths, areas.