Computational Vision CSCI 363, Fall 2016 Lecture 15 Stereopsis

Binocular Stereo The image in each of our two eyes is slightly different. Images in the plane of fixation fall on corresponding locations on the retina. Images in front of the plane of fixation are shifted outward on each retina. They have crossed disparity. Images behind the plane of fixation are shifted inward on the retina. They have uncrossed disparity.

Crossed and uncrossed disparity 1 uncrossed (negative) disparity plane of fixation 2 crossed (positive) disparity

Stereo processing To determine depth from stereo disparity: Extract the "features" from the left and right images For each feature in the left image, find the corresponding feature in the right image. Measure the disparity between the two images of the feature. Use the disparity to compute the 3D location of the feature.

The Correspondence problem How do you determine which features from one image match features in the other image? (This problem is known as the correspondence problem). This could be accomplished if each image has well defined shapes or colors that can be matched. Problem: Random dot stereograms. Left Image Right Image Making a stereogram

Random Dot Stereogram

Problem with Random Dot Stereograms In 1980's Bela Julesz developed the random dot stereogram. The stereogram consists of 2 fields of random dots, identical except for a region in one of the images in which the dots are shifted by a small amount. When one image is viewed by the left eye and the other by the right eye, the shifted region is seen at a different depth. No cues such as color, shape, texture, shading, etc. to use for matching. How do you know which dot from left image matches which dot from the right image?

Using Constraints to Solve the Problem To solve the correspondence problem, we need to make some assumptions (constraints) about how the matching is accomplished. Constraints used by many computer vision stereo algorithms: Uniqueness: Each point has at most one match in the other image. Similarity: Each feature matches a similar feature in the other image (i.e. you cannot match a white dot with a black dot). Continuity: Disparity tends to vary slowly across a surface. (Note: this is violated at depth edges). Epipolar constraint: Given a point in the image of one eye, the matching point in the image for the other eye must lie along a single line.

The epipolar constraint Feature in left image Possible matches in right image

It matters where you look If the observer is fixating a point along a horizontal plane through the middle of the eyes, the possible positions of a matching point in the other image lie along a horizontal line. If the observer is looking upward or downward, the line will be tilted. Most stereo algorithms for machine vision assume the epipolar lines are horizontal. For biological systems, the stereo computation must take into account where the eyes are looking (e.g. upward or downward).