1. The Trifocal Tensor Multiple View Geometry

2. Scene planes and homographies plane induces homography between two views

3. 6-point algorithm x 1,x 2,x 3,x 4 in plane, x 5,x 6 out of plane Compute H from x 1,x 2,x 3,x 4

4. Three-view geometry

5. The trifocal tensor Three back-projected lines have to meet in a single line Incidence relation provides constraint on lines Let us derive the corresponding algebraic constraint…

6. Notations

7. Incidence e.g.  is part of bundle formed by  ’ and  ”

8. Incidence relation

9. The Trifocal Tensor Trifocal Tensor = {T 1,T 2,T 3 } Only depends on image coordinates and is thus independent of 3D projective basis Also and but no simple relation General expression not as simple as DOF T: 3x3x3=27 elements, 26 up to scale 3-view relations: 11x3-15=18 dof 8(=26-18) independent algebraic constraints on T (compare to 1 for F, i.e. rank-2)

10. Homographies induced by a plane

11. Line-line-line relation Eliminate scale factor: (up to scale)

12. Point-line-line relation

13. Point-line-point relation note: valid for any line through x”, e.g. l”=[x”] x x” arbitrary

14. Point-point-point relation note: valid for any line through x’, e.g. l’=[x’] x x’ arbitrary

15. Overview incidence relations

16. Non-incident configuration incidence in image does not guarantee incidence in space

17. Epipolar lines if l’ is epipolar line, then satisfied for arbitrary l” inversely, epipolar lines are right and left null-space of

18. Epipoles With points becomes respectively Epipoles are intersection of right resp. left null-space of (e=P’C and e”=P”C)

19. Algebraic properties of T i matrices

20. Extracting F good choice for l” is e” (V 3 T e”=0)

21. Computing P,P‘,P“ ? ok, but not specifically, (no derivation)

22. matrix notation is impractical Use tensor notation instead

23. Definition affine tensor Collection of numbers, related to coordinate choice, indexed by one or more indices Valency = ( n+m ) Indices can be any value between 1 and the dimension of space ( d (n+m) coefficients)

24. Conventions Einstein’s summation: (once above, once below) Index rule: Contravariant indices Covariant indices

25. More on tensors Transformations (covariant) (contravariant)

26. Some special tensors Kronecker delta Levi-Cevita epsilon (valency 2 tensor) (valency 3 tensor)

28. Trilinearities

29. Compute F and P from T

30. matrix notation is impractical Use tensor notation instead

31. Definition affine tensor Collection of numbers, related to coordinate choice, indexed by one or more indices Valency = ( n+m ) Indices can be any value between 1 and the dimension of space ( d (n+m) coefficients)

32. Conventions Contraction: (once above, once below) Index rule:

33. More on tensors Transformations (covariant) (contravariant)

34. Some special tensors Kronecker delta Levi-Cevita epsilon (valency 2 tensor) (valency 3 tensor)

36. Trilinearities

37. Transfer: epipolar transfer

38. Transfer: trifocal transfer Avoid l’=epipolar line

39. Transfer: trifocal transfer point transfer line transfer degenerate when known lines are corresponding epipolar lines

40. Computation of Trifocal Tensor Linear method (7-point) Minimal method (6-point) Geometric error minimization method RANSAC method

41. Basic equations Three points Correspondence Relation #lin. indep.Eq. 4 Two points, one line One points, two line 2 1 2 Three lines At=0 (26 equations) (more equations) min||At|| with ||t||=1

42. Normalized linear algorithm At=0 Points Lines or Normalization: normalize image coordinates to ~1

43. Normalized linear algorithm Objective Given n  7 image point correspondences across 3 images, or a least 13 lines, or a mixture of point and line corresp., compute the trifocal tensor. Algorithm (i)Find transformation matrices H,H’,H” to normalize 3 images (ii)Transform points with H and lines with H -1 (iii)Compute trifocal tensor T from At=0 (using SVD) (iv)Denormalize trifocal tensor

44. Internal constraints 27coefficients 1 free scale 18 parameters 8 internal consistency constraints (not every 3x3x3 tensor is a valid trifocal tensor!) (constraints not easily expressed explicitly) Trifocal Tensor satisfies all intrinsic constraints if it corresponds to three cameras {P,P’,P”}

45. Maximum Likelihood Estimation data cost function parameterization (24 parameters+3N) also possibility to use Sampson error (24 parameters)

46. Objective Compute the trifocal tensor between two images Algorithm (i)Interest points: Compute interest points in each image (ii)Putative correspondences: Compute interest correspondences (and F) between 1&2 and 2&3 (iii)RANSAC robust estimation: Repeat for N samples (a) Select at random 6 correspondences and compute T (b) Calculate the distance d  for each putative match (c) Compute the number of inliers consistent with T (d  <t) Choose T with most inliers (iv)Optimal estimation: re-estimate T from all inliers by minimizing ML cost function with Levenberg-Marquardt (v)Guided matching: Determine more matches using prediction by computed T Optionally iterate last two steps until convergence Automatic computation of T

47. 108 putative matches 18 outliers 88 inliers 95 final inliers (26 samples) (0.43) (0.23) (0.19)

48. additional line matches

49. Matrix formulation for m-View Consider one object point X and its m images: i x i =P i X i, i=1, ….,m: i.e. rank(M) < m+4.

50. Laplace expansions The rank condition on M implies that all (m+4)*(m+4) minors of M are equal to 0. These can be written as sums of products of camera matrix parameters and image coordinates.

51. Matrix formulation for non-trivially zero minors, one row has to be taken from each image (m). 4 additional rows left to choose

52. only interesting if 2 or 3 rows from view

53. The three different types 1.Take the 2 remaining rows from one image block and the other two from another image block, gives the 2-view constraints. 2.Take the 2 remaining rows from one image block 1 from another and 1 from a third, gives the 3-view constraints. 3.Take 1 row from each of four different image blocks, gives the 4-view constraints.