1. Multiple View Geometry

2. THE GEOMETRY OF MULTIPLE VIEWS
Epipolar Geometry
The Essential Matrix
The Fundamental Matrix
The Trifocal Tensor
The Quadrifocal Tensor
Reading: Chapter 10.

3. Epipolar Geometry
Epipolar Plane
Baseline
Epipoles
Epipolar Lines

4. Potential matches for p have to lie on the corresponding
Epipolar Constraint
Potential matches for p have to lie on the corresponding
epipolar line l’.
Potential matches for p’ have to lie on the corresponding
epipolar line l.

5. Epipolar Constraint: Calibrated Case
Essential Matrix
(Longuet-Higgins, 1981)

6. E p’ is the epipolar line associated with p’.
Properties of the Essential Matrix T
E p’ is the epipolar line associated with p’.
ETp is the epipolar line associated with p.
E e’=0 and ETe=0.
E is singular.
E has two equal non-zero singular values
(Huang and Faugeras, 1989). T

7. Epipolar Constraint: Small Motions
To First-Order:
Pure translation:
Focus of Expansion

8. Epipolar Constraint: Uncalibrated Case
Fundamental Matrix
(Faugeras and Luong, 1992)

9. Properties of the Fundamental Matrix
F p’ is the epipolar line associated with p’.
FT p is the epipolar line associated with p.
F e’=0 and FT e=0.
F is singular. T T

10. The Eight-Point Algorithm (Longuet-Higgins, 1981)
|F | =1.
Minimize:
under the constraint 2

11. Non-Linear Least-Squares Approach
(Luong et al., 1993)
Minimize
with respect to the coefficients of F , using an
appropriate rank-2 parameterization.

12. Problem with eight-point algorithm
linear least-squares:
unit norm vector F yielding smallest residual
What happens when there is noise?

13. The Normalized Eight-Point Algorithm (Hartley, 1995)
Center the image data at the origin, and scale it so the
mean squared distance between the origin and the data
points is sqrt(2) pixels: q = T p , q’ = T’ p’.
Use the eight-point algorithm to compute F from the
points q and q’ .
Enforce the rank-2 constraint.
Output T F T’. i i i i i i T

14. Weak-calibration Experiments

15. Epipolar geometry example

16. courtesy of Andrew Zisserman
Example: converging cameras
courtesy of Andrew Zisserman

17. Trinocular Epipolar Constraints
These constraints are not independent!

18. Trinocular Epipolar Constraints: Transfer
Given p and p , p can be computed
as the solution of linear equations. 1 2 3

19. problem for epipolar transfer in trifocal plane!
Trinocular Epipolar Constraints: Transfer
problem for epipolar transfer in trifocal plane!
There must be more to trifocal geometry…
image from Hartley and Zisserman

20. Trifocal Constraints

21. Trifocal Constraints Calibrated Case All 3x3 minors must be zero!
Trifocal Tensor

22. Trifocal Constraints
Uncalibrated Case
Trifocal Tensor

23. Trifocal Constraints: 3 Points
Pick any two lines l and l through p and p . 2 3 2 3
Do it again.
T( p , p , p )=0 1 2 3

24. For any matching epipolar lines, l G l = 0.
Properties of the Trifocal Tensor
For any matching epipolar lines, l G l = 0.
The matrices G are singular.
They satisfy 8 independent constraints in the
uncalibrated case (Faugeras and Mourrain, 1995). T i 2 1 3 i 1
Estimating the Trifocal Tensor
Ignore the non-linear constraints and use linear least-squares
Impose the constraints a posteriori.

25. For any matching epipolar lines, l G l = 0.2 1 3
The backprojections of the two lines do not define a line!

26. courtesy of Andrew Zisserman
Trifocal
Tensor
Example
108 putative matches
18 outliers
(26 samples)
88 inliers
95 final inliers
(0.43)
(0.23)
(0.19)
courtesy of Andrew Zisserman

27. Trifocal Tensor Example
additional line matches
images courtesy of Andrew Zisserman

28. Transfer: trifocal transfer
(using tensor notation)
doesn’t work if l’=epipolar line
image courtesy of Hartley and Zisserman

29. Image warping using T(1,2,N)
(Avidan and Shashua `97)

30. Multiple Views (Faugeras and Mourrain, 1995)

31. Two Views
Epipolar Constraint

32. Three Views
Trifocal Constraint

33. Four Views
Quadrifocal Constraint
(Triggs, 1995)

34. Geometrically, the four rays must intersect in P..

35. Quadrifocal Tensor
and Lines

36. Quadrifocal tensor determinant is multilinear
thus linear in coefficients of lines !
There must exist a tensor with 81 coefficients containing all possible combination of x,y,w coefficients for all 4 images: the quadrifocal tensor

37. Scale-Restraint Condition from Photogrammetry