1. Cross-Based Local Multipoint Filtering
Jiangbo Lu1, Keyang Shi2, Dongbo Min1,
Liang Lin2, and Minh N. Do3
1Advanced Digital Sciences Center,
2Sun Yat-Sen University,
3Univ. of Illinois at Urbana-Champaign
Computer Vision and Pattern Recognition(CVPR), 2012.

2. Outline Introduction Related Work Proposed Algorithm
Experimental Results
Conclusion

3. Introduction

4. Background Edge-preserving smoothing filtering:
A key component for many computer vision applications
Goal :
remove noise or fine details
the structure/edge should be well preserved
Bilateral filter(BF), Guided filter(GF)

5. Cross-Based Local Multipoint Filtering(CLMF)
Objective
Present a cross-based framework of performing local multipoint filtering efficiently.
Two main steps:
1) multipoint estimation
2) aggregation
CLMF-0、CLMF-1
Guided Filter (GF)
fixed-sized square window
Cross-Based Local Multipoint Filtering(CLMF)
adaptive window size

6. Related work

7. Cross-based local support decision[19]
[19] K. Zhang, J. Lu, and G. Lafruit. Cross-based local stereo matching using orthogonal integral images. IEEE Trans. CSVT, 19(7):1073–1079, July 2009.

8. Bilateral Filter[15]
[15] C. Tomasi and R. Manduchi. Bilateral filtering for gray and color images. In Proc. of ICCV, 1998.

9. Guided Filter[6]
[6] K. He, J. Sun, and X. Tang. Guided image filtering. In Proc. of ECCV, 2010.

10. Guided Filter[6]

11. Guided Filter[6]

12. Proposed Algorithm

13. Definition Z : Filter input I : Guidance image Y : Filter output
Yi = Zi - ni Z
Z : Filter input
I : Guidance image
Y : Filter output
𝑝 : estimation point
𝑘 : observation point(support pixel)
Ωp : local support region of 𝑝
Wp : square window of a radius r
{hp, hp, hp, hp } : cross skeleton Y
Yi = aIi + b 1 2 3

14. Adaptive Scale Selection
Decide for each direction an appropriate arm length
Cross-based method[19]
Running average of the intensity of all the pixels covered by the current span h
More robust against the measurement noise
[19] :
If , =1
Otherwise, =0 p h
span h (right arm)

15. Adaptive Scale Selection
Gradient reversal artifact

16. Generalization of Local Multipoint Filtering
Zero-order (order m = 0 ) or first-order polynomial(m=1) model:
The model should be biased toward low-order polynomials to avoid over-fitting and gradient increase.

17. Generalization of Local Multipoint Filtering
Zero-order (order m = 0 ) or first-order polynomial(m=1) model:
Use “least squares” to fit the data (Similar with GF) :
ϵ is a regularization parameter to discourage the choices of large (i≥1)

18. Generalization of Local Multipoint Filtering
Zero-order (order m = 0 ) or first-order polynomial(m=1) model:
Solutions:
Ω 𝑘 : the number of pixels in Ω 𝑘
𝜇 𝑘 : mean of I in Ω 𝑘
𝜎 𝑘 : variance of I in Ω 𝑘
m=0 2
m=1

19. Generalization of Local Multipoint Filtering
Guided Filter(GF) : multipoint estimates are averaged together
CLMF : weighted averaged

20. Summary & Comparison O(1) time linear regression and aggregation
(independent of the window radius r)

21. Experimental Results

22. Implementation Raw matching cost[9]:
Winner-Take-All / Occlusion detection and filling[14]
r = 17, R = 3, τ = 20, and τs = 20
1.Scanline filling : the lowest disparity of the spatially closest nonoccluded pixel 2.Median filter :
[9] X. Mei, X. Sun, M. Zhou, S. Jiao, H. Wang, and X. Zhang. On building an accurate stereo matching system on graphics hardware. In Proc. of GPUCV, 2011.
[14] C. Rhemann, A. Hosni, M. Bleyer, C. Rother, and M. Gelautz. Fast cost-volume filtering for visual correspondence and beyond. In Proc. of CVPR, 2011.

23. CLMF-1
Ground Truth
CLMF-0

24. Experimental Results
Middlebury evaluation
Rank:23
Tsukuba

25. Conclusion

26. Conclusion
Propose a generic framework of performing cross-based local multipoint filtering efficiently
CLMF-0 and CLMF-1 find very competitive applications into many computer vision
More generalized than GF
Cross-based technique is very friendly for GPUs[20]
Plan to map the filters onto GPUs for speedup
[20] K. Zhang, J. Lu, Q. Yang, G. Lafruit, R. Lauwereins, and L. V. Gool. Real-time and accurate stereo: A scalable approach with bitwise fast voting on CUDA. IEEE Trans. CSVT, 21(7):867–878, July 2011.

27. Full-Image Guided Filtering for Fast Stereo Matching
Qingqing Yang, Dongxiao Li, Member, IEEE,
Lianghao Wang, and Ming Zhang
IEEE SIGNAL PROCESSING LETTERS, VOL. 20, NO. 3, MARCH 2013

28. Outline
Objective
Proposed Algorithm
Experimental Results
Conclusion

29. Objective Propose a novel full-image guided filtering method
A novel scheme called weight propagation is proposed to compute support weights.
Edge-preserving
Low-complexity

30. Proposed Algorithm

31. Filter Modeling C : filter input C’i : filter output at pixel i
Wi.j : weight of pixel pair (i,j)
Ni : normalizing constant
(p.q)∈Pi,j : adjacent nodes on the path Pi,j
Tp.q(I) : propagation function
Best path : minimum propagation weight
→ high complexity
Choose horizontal first policy
Ω : smoothness parameter

32. Implementation Two pass model 1)Horizontal direction in separate rows
2)the same way in separate columns Pr

33. Implementation Horizontal:Pr
Horizontal:
For an element r in a row, the intermediate sum of weighted value :
u- : the left neighbor of u
u+ : the right neighbor of u
Can be further accelerated by using the two-pass scan paradigm[15]
The intermediate results are stored in two temporary arrays.

34. horizontal path → vertical path
Implementation Pr
Horizontal:
The scan process is a sequential computation of weighted cumulative sum:
Simply computed by:
AL : the weighted cumulative sums calculated from the left to right
horizontal path → vertical path
reduce the complexity : 4 multiplication and 8 additions (each element)

35. Implementation Cost Volume C: Winner-Take-All: Post-processing:
[18] S. Birchfield and C. Tomasi, “A pixel dissimilarity measure that is insensitive to image sampling,” IEEE Trans. Patt. Anal. Mach. Intell., vol. 20, no. 4, pp. 401–406, 1998.
Implementation
Cost Volume C:
CBT : BT measure[18]
CGD : absolute difference of gradient
Winner-Take-All:
Post-processing:
Cross checking : occlusions / mismatch pixels are filled by the lowest disparity value of the nearest non-occluded pixel
Weighted median filter

36. Comparison Employ as many related pixels as possible
Important for cost filtering in large textureless regions
Bilateral
filter
Proposed

37. Comparison
Bilateral
filter
Proposed

38. Experimental Results

39. Experimental Results Core Duo 3.16 GHz CPU 2 GB 800MHz RAM
No parallelism technique is utilized.
The average runtime for cost-volume filtering : 68 ms
(on the Middlebury benchmark data sets)
27 × faster than the approach [13] using guided image filtering (1850 ms).
[13] C. Rhemann, A. Hosni, M. Bleyer, C. Rother, and M. Gelautz. Fast cost-volume filtering for visual correspondence and beyond. In Proc. of CVPR, 2011.

41. Experimental Results

42. Experimental Results

43. Conclusion

44. Conclusion
The novel weight propagation method ensures support elements are assigned.
All elements in the input signal contribute to the filtering approach.
Outperforms all local methods on the Middlebury benchmark in terms of both speed and accuracy