1. A Region Based Stereo Matching Algorithm Using Cooperative Optimization Zeng-Fu Wang, Zhi-Gang Zheng University of Science and Technology of China Computer Vision and Pattern Recognition, 2008 1

2. Outline Introduction Algorithmn Experimental Results Conclusion 2

3. Introduction The stereo correspondence problem is a key point in computer vision. Goal : Find a more reasonable disparity map that closes to the ground truth data. 3

4. Outline Introduction Algorithmn Experimental Results Conclusion 4

5. Algorithmn 5

6. 6

7. Mean-shift algorithm [19] No assumptions about probability distributions. Find local maxima clusters close in space and range correspond to classes. 7 [19] D. Comanicu, P. Meer : “Mean shift: A robust approach toward feature space analysis”, IEEE Trans. Pattern Anal. Machine Intell., May 2002.

8. Mean-shift algorithm [19] 8 [19] D. Comanicu, P. Meer : “Mean shift: A robust approach toward feature space analysis”, IEEE Trans. Pattern Anal. Machine Intell., May 2002.

9. Algorithmn 9

10. Window stereo matching Step 1 Compute a matching cost for each pixel at each disparity Step 2 Aggregate the costs across pixels at the same disparity Step 3 Calculating the best disparities based on the aggregated costs Step 4 Optionally refine the disparities 10

11. Adaptive correlation window stereo matching algorithm [16] Assumption : depth discontinuities occur at colour boundaries Reduce the outliers wieght A variation window sizes on the recurrsive moving average implementation 11 [16] Mark Gerrits and Philippe Bekaert. “Local Stereo Matching with Segmentation-based Outlier Rejection. ” Third Canadian Conference on Computer and Robot Vision, June 2006.

12. Adaptive correlation window stereo matching algorithm [16] 12 [16] Mark Gerrits and Philippe Bekaert. “Local Stereo Matching with Segmentation-based Outlier Rejection. ” Third Canadian Conference on Computer and Robot Vision, June 2006.

13. Algorithmn 13

14. Disparity plane fitting 14

15. Disparity plane fitting algorithm based on voting 15

16. Disparity plane fitting algorithm based on voting (parameter a) 16 Step 1 Do the similar calculations for all possible point pairs on the same lines belonging to the region. Step 2 Make a histogram by a voting operation, where the - coordinate is and the -coordinate is the count number of. Step 3 Do a smoothness operation by a Gaussian filter. Step 4 The maximum of the histogram will be the estimation of.

17. Disparity plane fitting algorithm comparison 17 The comparison of the plane fitting results based on the RANSAC algorithm(blue) and the voting algorithm(red).

18. Disparity plane fitting algorithm based on voting 18 The disparities obtained by the plane fitting algorithm based on voting

19. Algorithmn 19

20. The cooperative optimization Goal : To optimize the disparity plane parameters of each region such that the disparity plane parameters of the adjacent regions keep consistent. The total energy function E(x) of all regions is defined as E(x) = E 1 (x) + E 2 (x) +... + E n (x) E i (x) : the energy function of the ith region 20

21. The cooperative optimization 21 The sketch map for optimization of sub-targets

22. Energy functional of each region 22

23. Energy functional of each region 23

24. Energy functional of each region 24

25. Energy functional of each region 25

26. Energy functional of each region 26, otherwise

27. The cooperative optimization 27

28. The cooperative optimization  The optimization is carried out until the algorithm converges or the number of iteration is reached. 28

29. The cooperative optimization 29

30. The cooperative optimization 30

31. Algorithmn 31

32. Outline Introduction Algorithmn Experimental Results Conclusion 32

33. Experimental Results Device : A notebook with CPU of PM1.6G Settings parameters: = 0.5, = 0.5, = 0.5 are set according to [17] Source : Middlebury http://vision.middlebury.edu/stereo/ http://vision.middlebury.edu/stereo/ Time : 20s ( 4 iterations ) Segmentation : 8s 33 [17] Xiaofei Huang. “Cooperative Optimization for Energy Minimization: A Case Study of Stereo Matching”, cs.CV/0701057, Jan 2007. http://front.math.ucdavis.edu/author/X.Huanghttp://front.math.ucdavis.edu/author/X.Huang,

34. Experimental Results 34

35. Experimental Results 35 Black : occluded border regions White : discontinuities

36. Experimental Results 36

37. 37

38. Experimental Results 38

39. Outline Introduction Algorithmn Experimental Results Conclusion 39

40. Conclusion Contributions Combine some known techniques to obtain the high quality disparity map. The algorithm only requests the initial estimation of disparities is roughly correct. Future works Improve the plane fitting by introducing B-spline fitting technique. Develop a more efficient segmentation algorithm. 40