1. Image Rectification for Stereo Vision
Charles Loop
Zhengyou Zhang
Microsoft Research
Please remind the audience that this is all completely confidential. We are in the midst of the patent process and certainly do not want to jeopardize that effort in any way. 1

2. Problem Statement rectification
Compute a pair of 2D projective transforms (homographies)
rectification
Original images
Rectified images

3. Motivations To simplify stereo matching:
Instead of comparing pixels on skew lines, we now only compare pixels on the same scan lines.
Graphics applications: view morphing
Problem:
Rectifying homographies are not unique
Goal: to develop a technique based on
geometrically well-defined criteria minimizing image distortion due to rectification

4. Epipolar Geometry Epipoles anywhere Epipole at Fundamental matrixC C’
Epipoles anywhere
Fundamental matrix
F: a 3x3 rank-2 matrix
Epipole at
Fundamental matrix

5. Stereo Image Rectification
Compute H and H’ such that
Compute rectified image points:
Problem:
H and H’ are not unique.

6. Properties of H and H’ (I)
Consider each row of H and H’ as a line:
Recall: both e and e’ are sent to [1 0 0]T
Observations (I):
v and w must go through the epipole e
v’ and w’ must go through the epipole e’
u and u’ are irrelevant to rectification

7. Properties of H and H’ (II)
Observation (II):
Lines v and v’, and lines w and w’ must be corresponding epipolar lines.
Observation (III):
Lines w and w’ define the rectifying plane.

8. Decomposition of H Special projective transform: Similarity transform:
Shearing transform:

9. Special Projective Transform (I)
Sends the epipole to infinity
epipolar lines become parallel
Captures all image distortion due to projective transformation
Subgoal: Make Hp as affine as possible.

10. Special Projective Transform (II)
How to do it?
Let original image point be
the transformed point will be
Observation:
If all weights are equal, then there is no distortion.
Key idea:
minimize the variation of wi over all pixels
with weight

11. Similarity Transform
Rotate and translate images such that the epipolar lines are horizontally aligned.
Images are now rectified.

12. Shearing Transform
Free to scale and translate in the horizontal direction.
Subgoal:
Preserve original image resolution as close as possible.

13. Example
Original image pair

14. Intermediate result
After special projective transform:

15. Intermediate result
After similarity transform:

16. Final result
After shearing transform