A Region Based Stereo Matching Algorithm Using Cooperative Optimization Zeng-Fu Wang, Zhi-Gang Zheng University of Science and Technology of China Computer Vision and Pattern Recognition,

Outline Introduction Algorithmn Experimental Results Conclusion 2

Introduction The stereo correspondence problem is a key point in computer vision. Goal : Find a more reasonable disparity map that closes to the ground truth data. 3

Outline Introduction Algorithmn Experimental Results Conclusion 4

Algorithmn 5

6

Mean-shift algorithm [19] No assumptions about probability distributions. Find local maxima clusters close in space and range correspond to classes. 7 [19] D. Comanicu, P. Meer : “Mean shift: A robust approach toward feature space analysis”, IEEE Trans. Pattern Anal. Machine Intell., May 2002.

Mean-shift algorithm [19] 8 [19] D. Comanicu, P. Meer : “Mean shift: A robust approach toward feature space analysis”, IEEE Trans. Pattern Anal. Machine Intell., May 2002.

Algorithmn 9

Window stereo matching Step 1 Compute a matching cost for each pixel at each disparity Step 2 Aggregate the costs across pixels at the same disparity Step 3 Calculating the best disparities based on the aggregated costs Step 4 Optionally refine the disparities 10

Adaptive correlation window stereo matching algorithm [16] Assumption : depth discontinuities occur at colour boundaries Reduce the outliers wieght A variation window sizes on the recurrsive moving average implementation 11 [16] Mark Gerrits and Philippe Bekaert. “Local Stereo Matching with Segmentation-based Outlier Rejection. ” Third Canadian Conference on Computer and Robot Vision, June 2006.

Adaptive correlation window stereo matching algorithm [16] 12 [16] Mark Gerrits and Philippe Bekaert. “Local Stereo Matching with Segmentation-based Outlier Rejection. ” Third Canadian Conference on Computer and Robot Vision, June 2006.

Algorithmn 13

Disparity plane fitting 14

Disparity plane fitting algorithm based on voting 15

Disparity plane fitting algorithm based on voting (parameter a) 16 Step 1 Do the similar calculations for all possible point pairs on the same lines belonging to the region. Step 2 Make a histogram by a voting operation, where the - coordinate is and the -coordinate is the count number of. Step 3 Do a smoothness operation by a Gaussian filter. Step 4 The maximum of the histogram will be the estimation of.

Disparity plane fitting algorithm comparison 17 The comparison of the plane fitting results based on the RANSAC algorithm(blue) and the voting algorithm(red).

Disparity plane fitting algorithm based on voting 18 The disparities obtained by the plane fitting algorithm based on voting

Algorithmn 19

The cooperative optimization Goal : To optimize the disparity plane parameters of each region such that the disparity plane parameters of the adjacent regions keep consistent. The total energy function E(x) of all regions is defined as E(x) = E 1 (x) + E 2 (x) E n (x) E i (x) : the energy function of the ith region 20

The cooperative optimization 21 The sketch map for optimization of sub-targets

Energy functional of each region 22

Energy functional of each region 23

Energy functional of each region 24

Energy functional of each region 25

Energy functional of each region 26, otherwise

The cooperative optimization 27

The cooperative optimization  The optimization is carried out until the algorithm converges or the number of iteration is reached. 28

The cooperative optimization 29

The cooperative optimization 30

Algorithmn 31

Outline Introduction Algorithmn Experimental Results Conclusion 32

Experimental Results Device : A notebook with CPU of PM1.6G Settings parameters: = 0.5, = 0.5, = 0.5 are set according to [17] Source : Middlebury Time : 20s ( 4 iterations ) Segmentation : 8s 33 [17] Xiaofei Huang. “Cooperative Optimization for Energy Minimization: A Case Study of Stereo Matching”, cs.CV/ , Jan

Experimental Results 34

Experimental Results 35 Black : occluded border regions White : discontinuities

Experimental Results 36

37

Experimental Results 38

Outline Introduction Algorithmn Experimental Results Conclusion 39

Conclusion Contributions Combine some known techniques to obtain the high quality disparity map. The algorithm only requests the initial estimation of disparities is roughly correct. Future works Improve the plane fitting by introducing B-spline fitting technique. Develop a more efficient segmentation algorithm. 40