1. Computer Vision : CISC 4/689
CREDITS
Rasmussen, UBC (Jim Little), Seitz (U. of Wash.), Camps (Penn. State), UC, UMD (Jacobs), UNC, CUNY
Computer Vision : CISC 4/689

2. Computer Vision : CISC 4/689
Multi-View Geometry
3D World Points
Relates
Camera Orientations
Camera Centers
Computer Vision : CISC 4/689

3. Computer Vision : CISC 4/689
Multi-View Geometry
3D World Points
Relates
Camera Centers
Camera Intrinsic Parameters
Image Points
Camera Orientations
Computer Vision : CISC 4/689

4. Computer Vision : CISC 4/689
Stereo
scene point
image plane
optical center
Computer Vision : CISC 4/689

5. Computer Vision : CISC 4/689
Stereo
Basic Principle: Triangulation
Gives reconstruction as intersection of two rays
Requires
calibration
point correspondence
Computer Vision : CISC 4/689

6. Computer Vision : CISC 4/689
Stereo Constraints p’ p ?
Given p in left image, where can the corresponding point p’ in right image be?
Computer Vision : CISC 4/689

7. Computer Vision : CISC 4/689
Stereo Constraints M p’
Image plane
Epipolar Line Y2 X2 Z2 O2 Y1 p O1 Z1 X1
Epipole
Focal plane
Computer Vision : CISC 4/689

8. Computer Vision : CISC 4/689
Stereo
The geometric information that relates two different viewpoints of the same scene is entirely contained in a mathematical construct known as fundamental matrix.
The geometry of two different images of the same scene is called the epipolar geometry.
Computer Vision : CISC 4/689

9. Stereo/Two-View Geometry
The relationship of two views of a scene taken from different camera positions to one another
Interpretations
“Stereo vision” generally means two synchronized cameras or eyes capturing images
Could also be two sequential views from the same camera in motion
Assuming a static scene
Computer Vision : CISC 4/689

10. Computer Vision : CISC 4/689
3D from two-views
There are two ways of extracting 3D from a pair of images.
Classical method, called Calibrated route, we need to calibrate both cameras (or viewpoints) w.r.t some world coordinate system. i.e, calculate the so-called epipolar geometry by extracting the essential matrix of the system.
Second method, called uncalibrated route, a quantity known as fundamental matrix is calculated from image correspondences, and this is then used to determine the 3D.
Either way, principle of binocular vision is triangulation. Given a single image, the 3D location of any visible object point must lie on the straight line that passes through COP and image point (see fig.). Intersection of two such lines from two views is triangulation.
Computer Vision : CISC 4/689

11. Mapping Points between Images
What is the relationship between the images x, x’ of the scene point X in two views?
Intuitively, it depends on:
The rigid transformation between cameras (derivable from the camera matrices P, P’)
The scene structure (i.e., the depth of X)
Parallax: Closer points appear to move more
Computer Vision : CISC 4/689

12. Example: Two-View Geometry
courtesy of F. Dellaert
Is there a transformation relating the points xi to x’i ?
Computer Vision : CISC 4/689

13. Computer Vision : CISC 4/689
Epipolar Geometry
Baseline: Line joining camera centers C, C’
Epipolar plane ¦: Defined by baseline and scene point X
Computer Vision : CISC 4/689
baseline
from Hartley
& Zisserman

14. Computer Vision : CISC 4/689
Epipolar Lines
Epipolar lines l, l’: Intersection of epipolar plane ¦ with image planes
Epipoles e, e’: Where baseline intersects image planes
Equivalently, the image in one view of the other camera center. C C’
Computer Vision : CISC 4/689
from Hartley
& Zisserman

15. Computer Vision : CISC 4/689
Epipolar Pencil
As position of X varies, epipolar planes “rotate” about the baseline (like a book with pages)
This set of planes is called the epipolar pencil
Epipolar lines “radiate” from epipole—this is the pencil of epipolar lines
Computer Vision : CISC 4/689
from Hartley
& Zisserman

16. Computer Vision : CISC 4/689
Epipolar Constraint
Camera center C and image point define ray in 3-D space that projects to epipolar line l’ in other view (since it’s on the epipolar plane)
3-D point X is on this ray, so image of X in other view x’ must be on l’
In other words, the epipolar geometry defines a mapping x ! l’, of points in one image to lines in the other x’ C C’
Computer Vision : CISC 4/689
from Hartley
& Zisserman

17. Example: Epipolar Lines for Converging Cameras
Left view
Right view
from Hartley
& Zisserman
Intersection of epipolar lines = Epipole !
Indicates direction of other camera
Computer Vision : CISC 4/689

18. Special Case: Translation Parallel to Image Plane
Note that epipolar lines are parallel and corresponding points lie on correspond-
ing epipolar lines (the latter is true for all kinds of camera motions)
Computer Vision : CISC 4/689

19. From Geometry to AlgebraP p p’
Computer Vision : CISC 4/689

20. From Geometry to AlgebraP p p’
Computer Vision : CISC 4/689

21. Computer Vision : CISC 4/689
Rotation arrow should be at the other end, to get p in p’ coordinates
Linear Constraint: Should be able to express as matrix multiplication.
Computer Vision : CISC 4/689

22. Review: Matrix Form of Cross Product
Computer Vision : CISC 4/689

23. Review: Matrix Form of Cross Product
Computer Vision : CISC 4/689

24. Computer Vision : CISC 4/689
Matrix Form
Computer Vision : CISC 4/689

25. Computer Vision : CISC 4/689
The Essential Matrix
If calibrated, p gets multiplied by Intrisic matrix, K
Computer Vision : CISC 4/689

26. The Fundamental Matrix F
Mapping of point in one image to epipolar line in other image x ! l’ is expressed algebraically by the fundamental matrix F
Write this as l’ = F x
Since x’ is on l’, by the point-on-line definition we know that
x’T l’ = 0
Substitute l’ = F x, we can thus relate corresponding points in the camera pair (P, P’) to each other with the following:
x’T F x = 0
line
point
Computer Vision : CISC 4/689