Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai

Fundamental matrix (3x3 rank 2 matrix) Epipolar geometry Underlying structure in set of matches for rigid scenes Computable from corresponding points Simplifies matching Allows to detect wrong matches Related to calibration C1 C2 l2 P l1 e1 e2 C1 C2 l2 P l1 e1 e2 m1 L1 m2 L2 M l2 C1 m1 L1 m2 L2 M C2 m1 m2 lT1 l2 Fundamental matrix (3x3 rank 2 matrix)

Outline The trifocal tensor 2-D Projective geometry Chapters 2 and 3 of “Multiple View Geometry in Computer Vision” by Hartley and Zisserman

The trifocal tensor Three back-projected lines have to meet in a single line Incidence relation provides constraint on lines (impossible in two-view case) Constraints contained in 3x3x3 trifocal tensor

Line-line-line relation

Point-line-line relation

Point-line-point relation

Point-point-point relation

Point-point-point relation Given point correspondence in two images, the point cannot always be determined in third image (if it is on trifocal plane)

Outline The trifocal tensor 2-D Projective geometry

Projective 2D Geometry Points, lines & conics Transformations & invariants (next week)

Homogeneous coordinates Homogeneous representation of lines equivalence class of vectors, any vector is representative Set of all equivalence classes in R3(0,0,0)T forms P2 Homogeneous representation of points on if and only if The point x lies on the line l if and only if xTl=lTx=0 Homogeneous coordinates Inhomogeneous coordinates but only 2DOF

Points from lines and vice-versa Intersections of lines The intersection of two lines and is Line joining two points The line through two points and is Example

Ideal points and the line at infinity Intersections of parallel lines Example tangent vector normal direction Ideal points Line at infinity Note that in P2 there is no distinction between ideal points and others

Duality Duality principle: To any theorem of 2-dimensional projective geometry there corresponds a dual theorem, which may be derived by interchanging the role of points and lines in the original theorem

Outline The trifocal tensor 2-D Projective geometry

Projective 3D Geometry Points, lines, planes and quadrics Transformations П∞, ω∞ and Ω ∞

3D points 3D point in R3 in P3 projective transformation (4x4-1=15 dof)

Planes 3D plane Transformation Euclidean representation Dual: points ↔ planes, lines ↔ lines

Planes from points Or implicitly from coplanarity condition (solve as right nullspace of ) Or implicitly from coplanarity condition

Points from planes (solve as right nullspace of )

Lines (4dof: 2 for each point on the planes) Example: X-axis Span of WT is pencil of points: Span of W*T is pencil of planes: (4dof: 2 for each point on the planes) Example: X-axis

Conics Curve described by 2nd-degree equation in the plane or homogenized or in matrix form with 5DOF:

Five points define a conic For each point the conic passes through or stacking constraints yields

Tangent lines to conics The line l tangent to C at point x on C is given by l=Cx l x C

Dual conics A line tangent to the conic C satisfies In general (C full rank): Dual conics = line conics = conic envelopes

Quadrics and dual quadrics (Q : 4x4 symmetric matrix) 9 d.o.f. in general 9 points define quadric det Q=0 ↔ degenerate quadric (plane ∩ quadric)=conic transformation 3. and thus defined by less points 4. 5. Derive X’QX=x’M’QMx=0 relation to quadric (non-degenerate) transformation

Quadric classification Rank Sign. Diagonal Equation Realization 4 (1,1,1,1) X2+ Y2+ Z2+1=0 No real points 2 (1,1,1,-1) X2+ Y2+ Z2=1 Sphere (1,1,-1,-1) X2+ Y2= Z2+1 Hyperboloid (1S) 3 (1,1,1,0) X2+ Y2+ Z2=0 Single point 1 (1,1,-1,0) X2+ Y2= Z2 Cone (1,1,0,0) X2+ Y2= 0 Single line (1,-1,0,0) X2= Y2 Two planes (1,0,0,0) X2=0 Single plane Signature sigma= sum of diagonal,e.g. +1+1+1-1=2,always more + than -, so always positive…

Quadric classification Projectively equivalent to sphere: sphere ellipsoid hyperboloid of two sheets paraboloid Ruled quadrics: hyperboloids of one sheet Ruled quadric: two family of lines, called generators. Hyperboloid of 1 sheet topologically equivalent to torus! Degenerate ruled quadrics: cone two planes

Hierarchy of transformations Projective 15dof Intersection and tangency Parallellism of planes, Volume ratios, centroids, The plane at infinity π∞ Affine 12dof Similarity 7dof The absolute conic Ω∞ Euclidean 6dof Volume

Screw decomposition Any particular translation and rotation is equivalent to a rotation about a screw axis and a translation along the screw axis. screw axis // rotation axis

The plane at infinity The plane at infinity π is a fixed plane under a projective transformation H iff H is an affinity Represents 3DOF between projective and affine canonical position contains directions two planes are parallel  line of intersection in π∞ line // line (or plane)  point of intersection in π∞

The absolute conic The absolute conic Ω∞ is a (point) conic on π. In a metric frame: or conic for directions: (with no real points) The absolute conic Ω∞ is a fixed conic under the projective transformation H iff H is a similarity Represent 5 DOF between affine and similarity Ω∞ is only fixed as a set Circles intersect Ω∞ in two points Spheres intersect π∞ in Ω∞

The absolute conic Euclidean: Projective: (orthogonality=conjugacy) Given plane at infinity and absolute conic Euclidean: Projective: (orthogonality=conjugacy) normal Orthogonality is conjugacy with respect to Absolute Conic plane Pole-polar relationship. (out of scope for now)

The absolute dual quadric The absolute conic Ω*∞ is a fixed conic under the projective transformation H iff H is a similarity 1, not equation like abs conic 8 dof plane at infinity π∞ is the nullvector of Ω∞ Angles:

Next week Projective transformations Why do we need the Absolute Conic, the Absolute Quadric and their images