1. EECS 274 Computer Vision
Stereopsis

2. Stereopsis Stereopsis: Fusion and Reconstruction
Correlation-Based Fusion
Multi-Scale Edge Matching
Dynamic Programming
Using Three or More Cameras
Reading: FP Chapter 11, S Chapter 11

3. An application: mobile robot navigation
The INRIA Mobile Robot, 1990.
The Stanford Cart,
H. Moravec, 1979.
Courtesy O. Faugeras and H. Moravec.

4. Stereo vision Fusion: match points observed by two or more cameras
Reconstruction: find the pre-image of the matching points in 3D world
Assume calibrated camera and essential matrix/trifocal tensors associated with 3 cameras are known

5. Reconstruction / triangulation

6. (Binocular) Fusion

7. Non-Linear Method: find Q minimizing
Reconstruction
Geometric
approach
Algebraic approach
Linear Method:
find P such that
Non-Linear Method: find Q minimizing

8. Rectification
All epipolar lines are parallel in the rectified image plane

9. Reconstruction from rectified images
Disparity: d=u’-u
Depth: z = -B/d × f

10. Correlation methods (1970--)
Slide the window along the epipolar line until w.w’ is maximized. 2
Minimize |w-w’|
Normalized Correlation: minimize q instead.

11. Correlation methods: foreshortening problems
Solution: add a second pass using disparity estimates to warp
the correlation windows, e.g. Devernay and Faugeras (1994)

12. Multi-scale edge matching (Marr, Poggio and Grimson, 1979-81)
Edges are found by repeatedly smoothing the image and detecting
the zero crossings of the second derivative (Laplacian)
Matches at coarse scales are used to offset the search for matches
at fine scales (equivalent to eye movements)

13. Multi-scale edge matching (Marr, Poggio and Grimson, 1979-81)
One of the two
input images
Image Laplacian
Zeros of the Laplacian

14. Multi-scale edge matching (Marr, Poggio and Grimson, 1979-81)

15. The ordering constraint
In general the points
are in the same order
on both epipolar lines.
But it is not always the case..

16. Ordering constraints points are not necessarily in order
d-b-a c’-b’-d’

17. Dynamic programming (Baker and Binford, 1981)
Find the minimum-cost path going monotonically
down and right from the top-left corner of the
graph to its bottom-right corner.
Nodes = matched feature points (e.g., edge points).
Arcs = matched intervals along the epipolar lines.
Arc cost = discrepancy between intervals.

18. Dynamic programming (Ohta and Kanade, 1985)
use inter-scanline f or point
correspondence on vertical
edges
use DP for both
intra-scanline and
inter-scanline search

19. The third eye can be used for verification..
Three views
The third eye can be used for verification..
b1-a2 match is wrong as thee is no corresponding point in camera 3

20. More views (Okutami and Kanade, 1993)
Pick a reference image, and slide the corresponding
window along the corresponding epipolar lines of all
other images, using inverse depth relative to the first
image as the search parameter.
Use the sum of correlation scores to rank matches.

21. I1 I2
I10

22. Correspondence Extensive literature Smooth surfaces Region based
Graph cut
Mutual information
Belief propagation
Conditional random field
Smooth surfaces

23. Middlebury stereo dataset
De facto data set with ground truth, code, and comprehensive performance evaluation

24. Performance evaluation