1. Computer Vision
Stereo Vision

2. Pinhole Camera
Bahadir K. Gunturk

3. Perspective Projection
Bahadir K. Gunturk

4. Stereo Vision Two cameras. Known camera positions. Recover depth. p p’
scene point p p’
image plane
optical center
Bahadir K. Gunturk

5. Correspondences p p’
Bahadir K. Gunturk

6. Matrix form of cross product
a=axi+ayj+azk
a×b=|a||b|sin(η)u
b=bxi+byj+bzk
Bahadir K. Gunturk

7. The Essential Matrix
Essential matrix
Bahadir K. Gunturk

8. Stereo Constraints M Image plane Epipolar Line Y1 p p’ Y2 X2 O1 Z1 X1
Epipole
Focal plane
Bahadir K. Gunturk

9. A Simple Stereo System Right image: Left image: target reference
LEFT CAMERA
RIGHT CAMERA
baseline
Elevation Zw
disparity
Depth Z
Right image:
target
Left image:
reference
Zw=0
Bahadir K. Gunturk

10. Stereo View
Left View
Right View
Bahadir K. Gunturk
Disparity

11. Stereo Disparity
The separation between two matching objects is called the stereo disparity.
Bahadir K. Gunturk

12. Parallel Cameras P Z xl xr f pl pr Ol Or Disparity: T
T is the stereo baseline
Bahadir K. Gunturk

13. Finding Correspondences
Bahadir K. Gunturk

14. Correlation Approach
LEFT IMAGE
(xl, yl)
(0). Essential Equation represents actually the epipolar plane in either the left or the right image
(1). Epipolar line in the right image given pl
(Epl)Tpr=0
zr = fr
extension of the equations in pr = (xr,yr,fr)
(2). Epipolar line in the left image given pr
(prTE) pl=0
zl = fl
For Each point (xl, yl) in the left image, define a window centered at the point
Bahadir K. Gunturk

15. Correlation Approach
RIGHT IMAGE
(xl, yl)
(0). Essential Equation represents actually the epipolar plane in either the left or the right image
(1). Epipolar line in the right image given pl
(Epl)Tpr=0
zr = fr
extension of the equations in pr = (xr,yr,fr)
(2). Epipolar line in the left image given pr
(prTE) pl=0
zl = fl
… search its corresponding point within a search region in the right image
Bahadir K. Gunturk

16. Correlation Approach
RIGHT IMAGE
(xr, yr) dx
(xl, yl)
(0). Essential Equation represents actually the epipolar plane in either the left or the right image
(1). Epipolar line in the right image given pl
(Epl)Tpr=0
zr = fr
extension of the equations in pr = (xr,yr,fr)
(2). Epipolar line in the left image given pr
(prTE) pl=0
zl = fl
… the disparity (dx, dy) is the displacement when the correlation is maximum
Bahadir K. Gunturk

17. = ? f g
Comparing Windows
Most
popular
Bahadir K. Gunturk

18. Comparing Windows Minimize Sum of Squared Differences Maximize
Cross correlation
Bahadir K. Gunturk

19. Correspondence Difficulties
Why is the correspondence problem difficult?
Some points in each image will have no corresponding points in the other image.
(1) the cameras might have different fields of view.
(2) due to occlusion.
A stereo system must be able to determine the image parts that should not be matched.
Bahadir K. Gunturk

20. Structured Light Structured lighting
Feature-based methods are not applicable when the objects have smooth surfaces (i.e., sparse disparity maps make surface reconstruction difficult).
Patterns of light are projected onto the surface of objects, creating interesting points even in regions which would be otherwise smooth.
Finding and matching such points is simplified by knowing the geometry of the projected patterns.
Bahadir K. Gunturk

21. Stereo results Data from University of Tsukuba Scene Ground truth
(Seitz)
Bahadir K. Gunturk

22. Results with window correlation
Estimated depth of field
(a fixed-size window)
Ground truth
(Seitz)
Bahadir K. Gunturk

23. Results with better method
A state of the art method
Boykov et al., Fast Approximate Energy Minimization via Graph Cuts,
International Conference on Computer Vision, September 1999.
Ground truth
Bahadir K. Gunturk
(Seitz)