1. Multiresolution stereo image matching using complex wavelets Julian Magarey CRC for Sensor Signal and Information Processing Anthony Dick Dept of Computer Science University of Adelaide

2. Stereo Vision Problem

3. A Stereo Pair AIM: To recover 3D shape from stereo pair

4. Stereo Matching b Find a point in each image which represents the same point in the scene corresponding pointscorresponding points corresponding points disparity

5. Feature Based Matching b Detect and match distinctive features b Problems featureless areasfeatureless areas occluded featuresoccluded features same feature may appear differentsame feature may appear different

6. Multiresolution Matching b Match points at several levels of detail MATCH COARSE FINE Left Image Right Image

7. Wavelet Transform Original Image Resolution i,j Level 1 Res i/2, j/2 Level 2 Res i/4, j/4

8. Multiresolution Matching b Now have multiresolution representation b Level m similarity distance measure: where x is a pixel in the level m representation of the left image x’ is a pixel in the level m representation of the right image

9. Similarity distance surface  Can extrapolate similarity surface about x’ the surface minimum, the location of the surface minimum, a 2x2 curvature matrix, are derived from where

10. Stereo Matching Algorithm b Now have basic matching algorithm perform wavelet transform on imagesperform wavelet transform on images minimise SD(x,x’) for all x at top levelminimise SD(x,x’) for all x at top level use as starting point for finer level matchinguse as starting point for finer level matching b What if top level match is wrong? b How do we interpolate matches to finer level?

11. Coping with Mismatches b Find a field of disparity vectors which minimises where is a directed measure of the difference between {u} and the unsmoothed disparity field is a measure of the uniformity of {u} is a scalar controlling their relative influence

12. Regularisation Features b Based on Anandan [IJCV, 1989]  Use curvature matrix κ to smooth more in directions of less certainty Smooth more in this direction Smooth less in this direction Similarity surface contours

13. Coarse-to-fine interpolation b Robust disparity interpolation = level m pixel= level m+1 pixel D(a) = choice of {D(A), D(B), D(C), D(D)} which minimises similarity distance AB C D a c b d

14. Results b Calibrated camera setup b Projective reconstruction b Form textured VRML surface C1C2 Left Camera Right Camera 3D Surface

15. Results

16. Results

17. Conclusion b Future work lightinglighting geometric constraint incorporationgeometric constraint incorporation colour imagescolour images camera self-calibrationcamera self-calibration more than two imagesmore than two images b Already, results are promising!