1. Chapter 11: Stereopsis
Stereopsis: Fusing the pictures taken by two cameras
and exploiting the difference (or disparity) between
them to obtain the depth information of the scene.

2. ○ Stereo imaging (1) Bring the first camera to coincide with the world
coordinate system. The coordinates of w
Similarly,
(2)
(3) Substitute (2) into (1)
(4) Solve for Z

3. Image Rectification
-- Replace the input images with pictures parallel
to the baseline

4. Idea: Make the rectified
epipolar lines lie on
the scanline, which
is paralleled to the
baseline

5. -- Recover 3D information from 2D data
11.1 3D Reconstruction
-- Recover 3D information from 2D data
○ Stereo vision involves:
Feature extraction
Feature correspondence
Depth estimation
3D shape reconstruction
。Epipolar constraint restricts feature
correspondence along epipolar lines

6. ○ Difficulties:
。 Sparse depth information

7. intersect due to calibration and feature localization
。 Measurement errors: The rays R and R’ will never
intersect due to calibration and feature localization
( p and p’) errors

8. ○ Solutions:
。Heuristic approach
(1) Find the line segment perpendicular to R and R’
(2) Take its mid-point P as the preimage of p and p’

9. Given (a) projection matrices: M and M’ (b) matching points: p and p’
。Algebraic approach:
Given (a) projection matrices: M and M’
(b) matching points: p and p’
From perspective projection relations,
i) This is an over-constrained system of four
(2 for p and 2 for p’) independent linear
equations in P (3 unknowns x, y, z)
ii) Solve using linear least-squares techniques.

10. 。Optimization method: Reconstruct the scene
point Q by
where p, p’: discrete pixels
q , q’: actual images
Solve using nonlinear least-squares techniques

11. 11.3 Binocular Fusion
○ Random dot stereogram
-- A pair of images obtained by randomly
spraying black dots on a small square
plate floating over a larger one

12. Stereoscopically, the image pair gives the impression of a square in front of
the surround

13. 。Cooperative stereopsis algorithm (Marr and
Poggio, 1976)
-- relies on three constraints:
(a) Compatibility -- Two image features
can only match if they possess
similar properties
(b) Uniqueness -- A feature in one image
matches at most one feature in the
other picture
(c) Continuity -- The disparity of matches
varies smoothly in the image

15. Correlation
-- Find pixel correspondences by comparing
intensity profiles in the neighborhood of
potential matches

16. ○ Problem of assuming that the observed surface is
(locally) parallel to the two image planes
Two-pass algorithm:
Use initial estimates
of the disparity to
warp the windows
to compensate for
unequal amount of
foreshortening
ii) Find disparity and its derivatives that maximize
the correlation between the two windows

17. 11.3.2 Multi-Scale Edge Matching
-- Edge matching preferred to point matching
-- Correspondences are found at a variety of
scales

18. ○ Multi-scale binocular fusion algorithm
。 Larger scale -> fewer noises, less precise in location
。 Smaller scale -> more noises, more precise in location

19. Single scale matching Multiscale matching superimposition Image 2

22. 11.3.3 Dynamic Programming ○ Ordering constraint
In case 1, the order of feature points along the two
epipolar lines is the same. In case 2, a small object
lies in front of a larger one. Some surface points are
not visible in one of the images (e.g., A is not visible
in the right image), and the order of the image points
(e.g., B and D) is not the same in the two pictures.
Case 2
Case 1

23. ○ Example: Match intervals between two
corresponding epipolar lines

26. -- Additional cameras eliminate ambiguity
11.4 More Cameras
Three Cameras
-- Additional cameras eliminate ambiguity
○ The third image can be used to check
hypothetical matches between the first two
pictures
i) A 3D point is first constructed from two images
and is then projected onto the third image
ii) Given three cameras and two images of a point,
predict its position in a third image by intersecting
corresponding epipolar lines

28. ○ Multicamera method (Okutami and Kanade 1993)
Multiple Cameras
○ Multicamera method (Okutami and Kanade 1993)
Matches are found using all pictures.