Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP 2013 International Conference on Computer Vision Theory and Application

Outline Introduction Related Works Proposed Method Weighted Joint Bilateral Filter Slope Depth Compensation Filter Experimental Results Conclusion

Introduction

Introduction Goal : Using two filters to get more accurate disparity map in real-time. Consideration Noise reduction Correct edges Blurring control Goal

Related Works

Related Works Stereo Matching Left Image Right Image

(Optimization methods) Related Works Local Global Estimate accuracy Low High Calculation cost Methods Pixel matching Block matching (Optimization methods) Graph cuts Belief propagation Example

Related Works Flow Chart (Local) Matching Cost Computation 1 Matching Cost Computation 2 Cost Aggregation 3 Disparity Map Computation/Optimization 4 Disparity Map Refinement Disparity Map Refinement

Related Works Depth map refinement with filter Median filter Bilateral filter Input depth map Filter Output depth map

Related Works Bilateral filter Smoothing Space weight:Near pixels has large weight Color weight:Similar color pixels has large weight Smoothing Keep edges Weak in spike noise

Related Works Joint bilateral filter Add in the reference image Color weight is calculated by the reference Keep object edges of the reference Reference : Low noise Target : High noise Filtered image

Related Works Joint bilateral filter Multilateral filter Noise reduction O Correct edge O Blurring X Mixed depth values Spreading error regions Multilateral filter Space + Color + Depth Boundary recovering X

Proposed Method

Proposed Method Weighted joint bilateral filter Noise reduction Edge correction Slope depth compensation filter Blurring control

Weighted Joint Bilateral Filter 𝐷: Depth value 𝑝: Coordinate of current pixel 𝑠: Coordinate of support pixel 𝑁: Aggregation set of support pixel 𝑤(),𝑐(): Space/color weight 𝜎𝑠,𝜎𝑐: Space/color Gaussian distribution 𝑅𝑠: Weight map

Weighted Joint Bilateral Filter Add in the weight map Controlling amount of influence on a pixel Weight of the edge and error is small Joint bilateral filter - Mixed depth values - Spreading error regions

Weighted Joint Bilateral Filter Making weight map Space/color/disparity weight Sum of nearness of space, color, and disparity between center pixel and surrounding pixels.

Weighted Joint Bilateral Filter Mask image is made by Speckle Filter Detecting speckle noise Weight of speckle region is 0 Red region: speckle noise Weight = 0

Weighted Joint Bilateral Filter

Slope Depth Compensation Filter Weighted joint bilateral filter Remaining small blurring Difference between foreground and background color is small Slope depth compensation filter Reason of blurring is mixed depth value Convert mixed value to non-blurred candidate using initial depth map Removing remaining blur

Slope Depth Compensation Filter X in Dx ∈ {INITIAL;WJBF;SDCF}

Slope Depth Compensation Filter

Proposed Method Initial disparity Stereo matching Noise reduction/ edge correction Weighted Joint Bilateral F. Blurring control Slope Depth Compensention F.

Experimental Results

Experimental Results Evaluating accuracy improvement for various types of depth maps Block Matching (BM) Semi-Global Matching (SGM) Efficient Large-Scale (ELAS) Dynamic Programing (DP) Double Belief Propagation (DBP)

Experimental Results

Experimental Results

Experimental Results Comparing proposed method with cost volume refinement(Teddy). 32 times slower Yang, Q., Wang, L., and Ahuja, N. A constantspace belief propagation algorithm for stereo matching. In Computer Vision and Pattern Recognition(2010).

Experimental Results

Experimental Results Device : Intel Core i7-920 2.93GHz Comparing running time (ms) of BM plus proposed filter with selected stereo methods.

Experimental Results

Experimental Results Use the proposed filter for depth maps from Microsoft Kinect.

Conclusion

Conclusion Contribution The proposed methods can reduce depth noise and correct object boundary edge without blurring. Amount of improvement is large when an input depth map is not accurate. Future Works Investigating dependencies of input natural images and depth maps.