1. Geometry of Multiple Views
EECS 274 Computer Vision
Geometry of Multiple Views

2. Geometry of Multiple Views
Epipolar geometry
Essential matrix
Fundamental matrix
Trifocal tensor
Quadrifocal tensor
Reading: FP Chapter 10, S Chapter 7

3. Epipolar geometry Epipolar plane defined by P, O, O’, p and p’
Baseline OO’
p’ lies on l’ where the epipolar plane intersects with image plane π’
l’ is epipolar line associated with p and intersects baseline OO’ on e’
e’ is the projection of O observed from O’
Epipolar lines l, l’
Epipoles e, e’

4. Epipolar constraint
Potential matches for p have to lie on the corresponding
epipolar line l’
Potential matches for p’ have to lie on the corresponding
epipolar line l

5. Epipolar Constraint: Calibrated Case
M’=(R t)
Essential Matrix
(Longuet-Higgins, 1981)
3 ×3 skew-symmetric matrix: rank=2

6. Properties of essential matrix
E is defined by 5 parameters (3 for rotation and 2 for translation)
E p’ is the epipolar line associated with p’
E T p is the epipolar line associated with p
Can write as l .p = 0
The point p lies on the epipolar line associated with the vector
E p’

7. Properties of essential matrix (cont’d)
E e’=0 and ETe=0 (E e’=-RT[tx]e=0 )
E is singular
E has two equal non-zero singular values (Huang and Faugeras, 1989)

8. Epipolar Constraint: Small Motions To First-Order:
Pure translation: Focus of Expansion e
The motion field at every point in
the image points to focus of expansion

9. Epipolar Constraint: Uncalibrated Case
Fundamental Matrix
(Faugeras and Luong, 1992)
are normalized image coordinate

10. Properties of fundamental matrix
F has rank 2 and is defined by 7 parameters
F p’ is the epipolar line associated with p’ in the 1st image
F T p is the epipolar line associated with p in the 2nd image
F e’=0 and F T e=0
F is singular

11. Rank-2 constraint F admits 7 independent parameter
Possible choice of parameterization using e=(α,β)T and e’=(α’,β’)T and epipolar transformation
Can be written with 4 parameters: a, b, c, d

12. Weak calibration In theory:
E can be estimated with 5 point correspondences
F can be estimated with 7 point correspondences
Some methods estimate E and F matrices from a minimal number of parameters
Estimating epipolar geometry from a redundant set of point correspondences with unknown intrinsic parameters

13. The Eight-Point Algorithm (Longuet-Higgins, 1981)
Homogenous system, set F33=1
|F | =1.
Minimize:
under the constraint 2

14. Least-squares minimization
|F | =1
Minimize:
under the constraint
Error function:

15. Non-Linear Least-Squares Approach (Luong et al., 1993)
8 point algorithm with least-squares minimization
ignores the rank 2 property
First use least squares to find epipoles e and e’ that
minimizes |FT e|2 and |Fe’|2
Minimize
with respect to the coefficients of F , using an
appropriate rank-2 parameterization (4 parameters
instead of 8)

16. The Normalized Eight-Point Algorithm (Hartley, 1995)
Estimation of transformation parameters suffer form poor numerical condition problem
Center the image data at the origin, and scale it so the
mean squared distance between the origin and the data
points is 2 pixels: q = T p , q’ = T’ p’
Use the eight-point algorithm to compute F from the
points q and q’
Enforce the rank-2 constraint
Output T F T’ i i i i i i T

17. Weak calibration experiment

18. Trinocular Epipolar Constraints
Optical centers O1O2O3 defines a trifocal plane
Generally, P does not lie on trifocal plane formed
Trifocal plane intersects retinas along t1, t2, t3
Each line defines two epipoles, e.g., t2 defines e12, e32, wrt O1 and O3
These constraints are not independent!

19. Trinocular Epipolar Constraints: Transfer
Given p1 and p2 , p3 can be computed
as the solution of linear equations.
Geometrically, p1 is found as the intersection
of epipolar lines associated with p2 and p3

20. Trifocal Constraints P
The set of points that project onto an image line l is the plane L that contains the line and pinhole
Point P in L is projected onto p on line l (l=(a,b,c)T)
Recall

21. Trifocal Constraints Calibrated Case All 3×3 minors must be zero!P
Calibrated Case
All 3×3 minors
must be zero!
line-line-line correspondence
Trifocal Tensor

22. Trifocal Constraints Calibrated Case
Given 3 point correspondences, p1, p2, p3 of the same point P, and two lines l2, l3, (passing through p2, and p3), O1p1 must intersect the line l, where the planes L2 and L3 intersect
point-line-line correspondence

23. Trifocal Constraints P
Uncalibrated Case

24. Trifocal Constraints
Uncalibrated Case
Trifocal Tensor

25. Trifocal Constraints: 3 Points
Pick any two lines l and l through p and p . 2 3 2 3
Do it again.
T( p , p , p )=0 1 2 3

26. Properties of the Trifocal Tensor
For any matching epipolar lines, l G l = 0
The matrices G are singular
Each triple of points  4 independent equations
Each triple of lines  2 independent equations
4p+2l ≥ 26  need 7 triples of points or 13 triples of lines
The coefficients of tensor satisfy 8 independent constraints
in the uncalibrated case (Faugeras and Mourrain, 1995)
 Reduce the number of independent parameters from 26 to 18 2 1 3 T i i 1
Estimating the Trifocal Tensor
Ignore the non-linear constraints and use linear least-squares
a posteriori
Impose the constraints a posteriori

27. Multiple Views (Faugeras and Mourrain, 1995)
All 4 × 4 minors
have zero
determinants

28. Two Views
6 minors
Epipolar Constraint

29. Three Views
48 minors
Trifocal Constraint

30. Quadrifocal Constraint (Triggs, 1995)
Four Views
16 minors
Quadrifocal Constraint
(Triggs, 1995)

31. Geometrically, the four rays must intersect in P..

32. Quadrifocal Tensor and Lines
Given 4 point correspondences, p1, p2, p3, p4 of the same point P, and 3 lines l2, l3, l4 (passing through p2, and p3, p4), O1p1 must intersect the line l, where the planes L2 , L3, and L4

33. Scale-Restraint Condition from Photogrammetry
Trinocular constraints in the presence of calibration or measurement errors