1. Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi
Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement
Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi
VISAPP 2013
International Conference on Computer Vision Theory and Application

2. Outline Introduction Related Works Proposed Method
Weighted Joint Bilateral Filter
Slope Depth Compensation Filter
Experimental Results
Conclusion

3. Introduction

4. Introduction
Goal : Using two filters to get more accurate disparity map in real-time.
Consideration
Noise reduction
Correct edges
Blurring control
Goal

5. Related Works

6. Related Works
Stereo Matching
Left Image Right Image

7. (Optimization methods)
Related Works
Local
Global
Estimate accuracy
Low
High
Calculation cost
Methods
Pixel matching
Block matching
(Optimization methods)
Graph cuts
Belief propagation
Example

8. Related Works Flow Chart (Local) Matching Cost Computation1
Matching Cost Computation 2
Cost Aggregation 3
Disparity Map Computation/Optimization 4
Disparity Map Refinement
Disparity Map Refinement

9. Related Works Depth map refinement with filter Median filter
Bilateral filter
Input depth map
Filter
Output depth map

10. Related Works Bilateral filter Smoothing
Space weight:Near pixels has large weight
Color weight:Similar color pixels has large weight
Smoothing
Keep edges
Weak in spike noise

11. Related Works Joint bilateral filter Add in the reference image
Color weight is calculated by the reference
Keep object edges of the reference
Reference : Low noise Target : High noise Filtered image

12. Related Works Joint bilateral filter Multilateral filter
Noise reduction O
Correct edge O
Blurring X
Mixed depth values
Spreading error regions
Multilateral filter
Space + Color + Depth
Boundary recovering X

13. Proposed Method

14. Proposed Method Weighted joint bilateral filter
Noise reduction
Edge correction
Slope depth compensation filter
Blurring control

15. Weighted Joint Bilateral Filter
𝐷: Depth value
𝑝: Coordinate of current pixel
𝑠: Coordinate of support pixel
𝑁: Aggregation set of support pixel
𝑤(),𝑐(): Space/color weight
𝜎𝑠,𝜎𝑐: Space/color Gaussian distribution
𝑅𝑠: Weight map

16. Weighted Joint Bilateral Filter
Add in the weight map
Controlling amount of influence on a pixel
Weight of the edge and error is small
Joint bilateral filter
- Mixed depth values
- Spreading error regions

17. Weighted Joint Bilateral Filter
Making weight map
Space/color/disparity weight
Sum of nearness of space, color, and disparity between center pixel and surrounding pixels.

18. Weighted Joint Bilateral Filter
Mask image is made by Speckle Filter
Detecting speckle noise
Weight of speckle region is 0
Red region: speckle noise
Weight = 0

19. Weighted Joint Bilateral Filter

20. Slope Depth Compensation Filter
Weighted joint bilateral filter
Remaining small blurring
Difference between foreground and background color is small
Slope depth compensation filter
Reason of blurring is mixed depth value
Convert mixed value to non-blurred candidate using initial depth map
Removing remaining blur

21. Slope Depth Compensation Filter
X in Dx ∈ {INITIAL;WJBF;SDCF}

22. Slope Depth Compensation Filter

23. Proposed Method Initial disparity Stereo matching
Noise reduction/ edge correction Weighted Joint Bilateral F.
Blurring control Slope Depth Compensention F.

24. Experimental Results

25. Experimental Results
Evaluating accuracy improvement for various types of depth maps
Block Matching (BM)
Semi-Global Matching (SGM)
Efficient Large-Scale (ELAS)
Dynamic Programing (DP)
Double Belief Propagation (DBP)

26. Experimental Results

27. Experimental Results

31. Experimental Results
Comparing proposed method with cost volume refinement(Teddy).
32 times slower
Yang, Q., Wang, L., and Ahuja, N. A constantspace belief propagation algorithm for stereo matching.
In Computer Vision and Pattern Recognition(2010).

32. Experimental Results

33. Experimental Results Device : Intel Core i7-920 2.93GHz
Comparing running time (ms) of BM plus proposed filter with selected stereo methods.

34. Experimental Results

35. Experimental Results
Use the proposed filter for depth maps from Microsoft Kinect.

37. Conclusion

38. Conclusion Contribution
The proposed methods can reduce depth noise and correct object boundary edge without blurring.
Amount of improvement is large when an input depth map is not accurate.
Future Works
Investigating dependencies of input natural images and depth maps.