Media Processor Lab. Media Processor Lab. Trellis-based Parallel Stereo Matching Media Processor Lab. Sejong univ. Dong-seok Kim

Media Processor Lab. Media Processor Lab. # 2. Contents  Introduction  Stereo Vision Model  Center-referenced space  Constraints on disparity  Estimating optimal disparity  Experimental results  Conclusion

Media Processor Lab. Media Processor Lab. # 3. Introduction  Stereo vision is an inverse process that attempts to restore the original scene from a pair of images.  In this paper a new basis for disparity based on center-referenced coordinates is presented that is concise and complete in terms of constraint representation.

Media Processor Lab. Media Processor Lab. # 4. Stereo Vision Model (1)  Projection Model  Assumption : coplanar image planes, parallel optical axes, equal focal lengths l, and matching epipolar lines  The inverse match space I is the finite set of points, represented by solid dots, that are reconstructable by matching image pixels.  Left image scan line : f l = [f l 1 ··· f l N ]  Right image scan line : f r = [f r 1 ··· f r N ]

Media Processor Lab. Media Processor Lab. # 5. Stereo Vision Model (2)  Representation of Correspondence  Each element of each scan line can  have a corresponding element in the other image scan line, denoted (f l i, f r j )  be occluded in the other image scan line, denoted (f l i, Ø) for a left image element (right occlusion) and (Ø, f r i ) for a right image element (left occlusion)  Left-referenced disparity map : d l = [d l 1 ··· d l N ]  Disparity value : d l i ⇔ (f l i, f r i+d l i )  Right-referenced disparity map : d r = [d r 1 ··· d r N ]  Disparity value : d r j ⇔ (f l i+d r j,f r j ).

Media Processor Lab. Media Processor Lab. # 6. Center-referenced space (1)  Using only left- or right-referenced disparity, it is difficult to represent common matching constraints with respect to both images.  An alternate center-referenced projection  The focal point p c located at the midpoint between the focal points for the left and right image plane  Plane with 2N + 1 and focal length of 2l  The projection lines intersect with the horizontal iso- disparity lines forms the inverse space D.

Media Processor Lab. Media Processor Lab. # 7. Center-referenced space (2)  Center-referenced disparity vector d = [d 0 ··· d 2N ]  disparity value d i indicates the depth index of a real world point along the projection line from site i on the center image plane  If d i is a match point : (f l (i – d j + 1)/2, f r (i + d j + 1)/2 )  (f l i, f r j ) is denoted by the disparity d i + j – 1 = j – i  The odd function o(x) is used to indicate if d i is a match point, that is o(i + di) = 1 when d i is a match point.

Media Processor Lab. Media Processor Lab. # 8. Center-referenced space (3)  Represent occlusions by assigning the highest possible disparity (Fig. 3)  The correspondence (f l 5, f r 8 ) creates a right occlusion for which the real object could lie anywhere in the triangular Right Occlusion Region (ROR)  If only I is used, then the match points (solid dots) in the ROR are used.  D contains additional occlusion points (open dots) in the ROR that are further to the right, which are used instead.

Media Processor Lab. Media Processor Lab. # 9. Constraints on disparity  Parallel axes : d i ≥ 0  Endpoints : d 0 = d 2N = 0  Cohesiveness : d i – d i – 1 ∈ {–1, 0, 1}  Uniqueness : o(i + d i ) = 1 ⇒ d i–1 = d i = d i+1

Media Processor Lab. Media Processor Lab. # 10. Estimating optimal disparity  DP techniques progressing through the trellis from left to right (site i = 0, …, 2N). In recursive form, the shortest path algorithm for disparity is formally given by:  Initialization : Endpoint has zero disparity  Recursion : At each site i = 1, … 2N, find the best path into each node j. if i+j is even, otherwise  Termination : i = 2N and j = 0.  Reconstruction : Backtrack the decisions..

Media Processor Lab. Media Processor Lab. # 11. Experimental results

Media Processor Lab. Media Processor Lab. # 12. Conclusion  We have used a center-referenced projection to represent the discrete inverse space for stereo correspondence.  This space D contains additional occlusion points which we exploit to create a concise representation of correspondence and occlusion.  The algorithm was tested on both real and synthetic image pairs with good results.