1. M.S. Student, Hee-Jong Hong
Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Dongbo Min, Member, IEEE,
Jiangbo Lu, Member, IEEE,
Minh N. Do, Senior Member, IEEE
M.S. Student, Hee-Jong Hong
Sep 24, 2013

2. Proposed Method : Improve Cost Aggregation Experimental Results
Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Outline
Introduction
Related Works
Proposed Method : Improve Cost Aggregation
Experimental Results
Conclusion

3. Introduction Goal：Perform efficient cost aggregation.
Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Introduction
Goal：Perform efficient cost aggregation.
Solution : Joint histogram + reduce redundancy
Advantage : Low complexity but keep high-quality.

4. Related Works Complexity of aggregation：O(NBL)
Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Related Works
N : all pixels (W*H)
B : window size
L : disparity level
Complexity of aggregation：O(NBL)
Reduce complexity approach
Scale image : Multi Scale Approach D. Min and K. Sohn, “Cost aggregation and occlusion handling with WLS in stereo matching,” IEEE Trans. on Image Processing, 2008.
Bilateral filter : Bilateral Approximation C. Richardt, D. Orr, I. P. Davies, A. Criminisi, and N. A. Dodgson, “Real-time spatiotemporal stereo matching using the dual-cross- bilateral grid,” in European Conf. on Computer Vision, 2010 S. Paris and F. Durand, “A fast approximation of the bilateral filter using a signal processing approach,” International Journal of Computer Vision, 2009.
Guided filter : Run in constant time => O(NL) C.Rhemann,A.Hosni,M.Bleyer,C.Rother,andM.Gelautz,“Fast cost-volume filtering for visual correspondence and beyond,” in IEEE Conf. on Computer Vision and Pattern Recognition, 2011

5. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Proposed Method

6. Local Method Algorithm
Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Local Method Algorithm
Cost initialization : Truncated Absolute Difference
=>
Cost aggregation : Weighted filter
Disparity computation : Winner take all
[4,8]

7. Improve Cost Aggregation
Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Improve Cost Aggregation
New formulation for aggregation
Remove normalization
Joint histogram representation
Compact representation for search range
Reduce disparity levels
Spatial sampling of matching window
Regularly sampled neighboring pixels
Pixel-independent sampling

8. New formulation for aggregation
Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
New formulation for aggregation
Remove normalization
=>
Joint histogram representation
J. Ding, J. Liu, W. Zhou, H. Yu, Y. Wang, and X. Gong, “Realtime stereo vision system using adaptive weight cost aggregation approach,” EURASIP J. Image and Video Processing, vol. 2011, p. 20,

9. Compact Search Range Cost aggregation O( NBD ) => O( NBDc )
Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Compact Search Range
Cost aggregation
=>
MC(q)：a subset of disparity levels whose size is Dc.
N : all pixels (W*H)
B : window size
D : disparity level
O( NBD )
O( NBDc )

10. Compact Search Range Non-occluded region of ‘Teddy’ image
Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Compact Search Range
Non-occluded region of ‘Teddy’ image
Dc = 5 (Best)
Final Accuracy = 94.2%
Dc = 6
Final Accuracy = 94.1%
Dc = 60
Final Accuracy = 93.7%

11. Spatial Sampling of Matching Window
Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Spatial Sampling of Matching Window
Reason
A large matching window and a well-defined weighting function leads to high complexity.
Pixels should aggregate in the same object, NOT in the window.
Solution
Color segmentation => Time consuming (Heavy)
Spatial Sampling => Easy but powerful
Regular Sampling => Independent from reference pixel => Reduce Complexity

12. Spatial Sampling of Matching Window
Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Spatial Sampling of Matching Window
Cost aggregation
=>
S : sampling ratio
O( NBDc )
O( NBDc / S2)

13. Parameter definition Pre-procseeing N : size of image
Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Parameter definition
N : size of image
B : size of matching window
N(p)=W×W
MD : disparity levels
size=D
MC : The subset of disparity
size=DC<<D
S : Sampling ratio
Pre-procseeing

14. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Experimental Result

15. Experimental Results Pre-processing Post-processing
Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Experimental Results
Pre-processing
5*5 Box filter
Post-processing
Cross-checking technique
Weighted median filter (WMF)
Device：Intel Xeon 2.8-GHz CPU (using a single core only) and a 6-GB RAM
Parameter setting
( ) = (1.5, 1.7, 31*31, 0.11, 13.5, 2.0)

16. Experimental Results (a) (b) (c) (d)
Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Experimental Results
(a)
(b)
(c)
(d)

17. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Experimental Results
Using too large box windows (7×7, 9×9) deteriorates the quality, and incurs more computational overhead.
Pre-filtering can be seen as the first cost aggregation step and serves the removal of noise.

18. Experimental Results The smaller S, the better
Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Experimental Results
The smaller S, the better
Fig. 5. Performance evaluation: average percent (%) of bad matching pixels for ‘nonocc’, ‘all’ and ‘disc’ regions according to Dc and S.
2 better than 1

19. Experimental Results The smaller S, the longer
Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Experimental Results
The smaller S, the longer
The bigger Dc, the longer

20. Experimental Results APBP : Average Percentage of Bad Pixels
Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Experimental Results
APBP : Average Percentage of Bad Pixels

21. Experimental Results Ground truth Error maps Results Original images
Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Experimental Results
Ground truth
Error maps
Results
Original images

22. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Experimental Results

23. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Conclusion

24. Conclusion Contribution Future work
Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013
Conclusion
Contribution
Re-formulate the problem with the relaxed joint histogram.
Reduce the complexity of the joint histogram-based aggregation.
Achieved both accuracy and efficiency.
Future work
More elaborate algorithms for selecting the subset of label hypotheses.
Estimate the optimal number Dc adaptively.
Extend the method to an optical flow estimation.

25. Thank you!