Efficient Stereo Matching Based on a New Confidence Metric Won-Hee Lee, Yumi Kim, and Jong Beom Ra Department of Electrical Engineering, KAIST, Daejeon, Korea 20th European Signal Processing Conference (EUSIPCO 2012)
Outline Introduction Related Work Proposed Algorithm Experimental Results Conclusion
Introduction
Introduction For the TV application, stereo matching should be performed in real-time. Aggregation kernel size is to be small Aggregation process takes large computation loads May cause problems in a textureless area Texture area information incorrect textureless area Propose a new confidence metric for stereo matching For efficient refinement (with small kernel size) Objective:
Introduction 35X35 5X35 [4] K. Zhang, J. Lu, and G. Lafruit, “Cross-based local stereo matching using orthogonal integral images,” IEEE TCSVT, 2009.
Related Work
Related Work Cross-based stereo matching algorithm[4] Raw matching cost: Aggregated cost: Ud(x) : local support region Ud(x) : the number of pixels in Ud(x)
Related Work Cross-based stereo matching algorithm[4] Winner-take-all: d0(x) : the initial disparity dmax(x) : the maximum disparity
Related Work Confidence metrics[5]: Several metrics were proposed to measure the confidence level of match Utilizing: Aggregated cost Curvature of the cost curve Left-right consistency [5] X. Hu and P. Mordohai, “Evaluation of stereo confidence indoors and outdoors,” in CVPR, 2010
Confidence metrics 1) Matching score metric (MSM) C : aggregated cost di : the disparity that reveals the ith minimum cost White(High confidence) Black(Low confidence)
Confidence metrics 2) Curvature of cost curve metric (CUR)
Confidence metrics 3) Naive peak ratio metric (PKRN)
Confidence metrics 4) Naive winner margin metric (WMNN) computes a margin between two minimum costs normalize it with the sum of total costs
Confidence metrics 5) Left right difference metric (LRD) min{cR(x - d1, dR)}: the minimum value of a cost curve at the corresponding pixel in the right image.
Proposed Algorithm
Framework
Proposed Confidence Metric ‧:Correct estimated pixels X : Incorrect estimated pixels
Proposed Confidence Metric The new metric is proposed as Characteristic: extracts the curvature information across a range larger than that including three cost values in the CUR metric Ud(x) : improve the metric performance for a cost graph with a small curvature. LoG : a Laplacian of Gaussian filter of n-taps
Refinement Weighted median filter Weight 𝒘 𝒊 : 𝒅 𝒊 𝟎 : the initial disparity of neighboring pixels (same color segment) : duplication operator offset a slope of function
Refinement Histogram-based color segmentation algorithm[6]: [6] J. Delon, A. Desolneux, J. L. Lisani, and A. B. Petro, “A nonparametric approach for histogram segmentation,” IEEE TIP, 2007. Refinement Histogram-based color segmentation algorithm[6]:
Refinement The filtering is applied only to the limited number if pixels Due to small size of filtering kernel To enlarge the filtering range Vertically propagate the filtered result of a current pixel Propagation Data After weighted median filtering… Datapropagate = DataA Filtered Disparity A If Weightpropagate > WeightB DisparityB = Disparitypropagate Else Datapropagate = DataB Weight B (current) Color segment index C
Experimental Results
Experimental Results Parameters: Aggregation kernel Filtering kernel T n σ τ 5 x 35 5 x 63 60 7 10 2 n : Laplacian of Gaussian filter of n-taps offset a slope of function
Experimental Results Initial disparity map Bad pixel Confidence map
AUC: Area Under the Curve Experimental Results AUC: Area Under the Curve Venus Tsukuba Teddy Cones
Experimental Results Error rate (Threshold = 1) Cross-based Adaptive support-weight
[4] Experimental Results [10] Proposed
After Aggregation
After Aggregation
After Aggregation
Conclusion
Conclusion Presented an efficient stereo matching algorithm Applying a weighted median filter that is based on the proposed confidence metric. Successfully refine initial disparities. Competitive to the existing algorithms with a large size of aggregation kernel.