M.S. Student, Hee-Jong Hong

Slides:



Advertisements
Similar presentations
An Adaptive Learning Method for Target Tracking across Multiple Cameras Kuan-Wen Chen, Chih-Chuan Lai, Yi-Ping Hung, Chu-Song Chen National Taiwan University.
Advertisements

Improved Census Transforms for Resource-Optimized Stereo Vision
Aggregating local image descriptors into compact codes
Spatial-Temporal Consistency in Video Disparity Estimation ICASSP 2011 Ramsin Khoshabeh, Stanley H. Chan, Truong Q. Nguyen.
Stereo Many slides adapted from Steve Seitz. Binocular stereo Given a calibrated binocular stereo pair, fuse it to produce a depth image Where does the.
Cuong Cao Pham and Jae Wook Jeon, Member, IEEE
Real-Time Accurate Stereo Matching using Modified Two-Pass Aggregation and Winner- Take-All Guided Dynamic Programming Xuefeng Chang, Zhong Zhou, Yingjie.
Low Complexity Keypoint Recognition and Pose Estimation Vincent Lepetit.
Adviser : Ming-Yuan Shieh Student ID : M Student : Chung-Chieh Lien VIDEO OBJECT SEGMENTATION AND ITS SALIENT MOTION DETECTION USING ADAPTIVE BACKGROUND.
Depth Coding Using a Boundary Reconstruction Filter for 3-D Video Systems Kwan-Jung Oh, Anthony Vetro, Fellow, IEEE, Yo-Sung Ho, Senior Member, IEEE CSVT,
{ Fast Disparity Estimation Using Spatio- temporal Correlation of Disparity Field for Multiview Video Coding Wei Zhu, Xiang Tian, Fan Zhou and Yaowu Chen.
Modeling Pixel Process with Scale Invariant Local Patterns for Background Subtraction in Complex Scenes (CVPR’10) Shengcai Liao, Guoying Zhao, Vili Kellokumpu,
HCI Final Project Robust Real Time Face Detection Paul Viola, Michael Jones, Robust Real-Time Face Detetion, International Journal of Computer Vision,
Real-time Embedded Face Recognition for Smart Home Fei Zuo, Student Member, IEEE, Peter H. N. de With, Senior Member, IEEE.
A Multi-Scale Tikhonov Regularization Scheme for Implicit Surface Modelling Jianke Zhu, Steven C.H. Hoi and Michael R. Lyu Department of CS&E, The Chinese.
Adaptive Rao-Blackwellized Particle Filter and It’s Evaluation for Tracking in Surveillance Xinyu Xu and Baoxin Li, Senior Member, IEEE.
High-Quality Video View Interpolation
Object Recognition Using Geometric Hashing
Effective Gaussian mixture learning for video background subtraction Dar-Shyang Lee, Member, IEEE.
100+ Times Faster Weighted Median Filter [cvpr ‘14]
Estimating the Driving State of Oncoming Vehicles From a Moving Platform Using Stereo Vision IEEE Intelligent Transportation Systems 2009 M.S. Student,
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 20, NO. 11, NOVEMBER 2011 Qian Zhang, King Ngi Ngan Department of Electronic Engineering, the Chinese university.
Presented By : Murad Tukan
Fast Cost-volume Filtering For Visual Correspondence and Beyond Asmaa Hosni, Member, IEEE, Christoph Rhemann, Michael Bleyer, Member, IEEE, Carsten Rother,
Recursive Bilateral Filtering F Reference Yang, Qingxiong. "Recursive bilateral filtering." ECCV Deriche, Rachid. "Recursively implementating.
Joint Histogram Based Cost Aggregation For Stereo Matching Dongbo Min, Member, IEEE, Jiangbo Lu, Member, IEEE, Minh N. Do, Senior Member, IEEE IEEE TRANSACTION.
Stereo Matching Information Permeability For Stereo Matching – Cevahir Cigla and A.Aydın Alatan – Signal Processing: Image Communication, 2013 Radiometric.
Binaural Sonification of Disparity Maps Alfonso Alba, Carlos Zubieta, Edgar Arce Facultad de Ciencias Universidad Autónoma de San Luis Potosí.
Noise-Robust Spatial Preprocessing Prior to Endmember Extraction from Hyperspectral Data Gabriel Martín, Maciel Zortea and Antonio Plaza Hyperspectral.
Fast Approximate Energy Minimization via Graph Cuts
Mutual Information-based Stereo Matching Combined with SIFT Descriptor in Log-chromaticity Color Space Yong Seok Heo, Kyoung Mu Lee, and Sang Uk Lee.
ICPR/WDIA-2012 High Quality Novel View Synthesis Based on Low Resolution Depth Image and High Resolution Color Image Jui-Chiu Chiang, Zheng-Feng Liu, and.
Prakash Chockalingam Clemson University Non-Rigid Multi-Modal Object Tracking Using Gaussian Mixture Models Committee Members Dr Stan Birchfield (chair)
A Local Adaptive Approach for Dense Stereo Matching in Architectural Scene Reconstruction C. Stentoumis 1, L. Grammatikopoulos 2, I. Kalisperakis 2, E.
Joint Depth Map and Color Consistency Estimation for Stereo Images with Different Illuminations and Cameras Yong Seok Heo, Kyoung Mu Lee and Sang Uk Lee.
Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi
Object Stereo- Joint Stereo Matching and Object Segmentation Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on Michael Bleyer Vienna.
Cross-Based Local Multipoint Filtering
A Non-local Cost Aggregation Method for Stereo Matching
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 34, NO. 2, FEBRUARY Leonardo De-Maeztu, Arantxa Villanueva, Member, IEEE, and.
Young Ki Baik, Computer Vision Lab.
Object Detection with Discriminatively Trained Part Based Models
Stereo Many slides adapted from Steve Seitz.
Feature-Based Stereo Matching Using Graph Cuts Gorkem Saygili, Laurens van der Maaten, Emile A. Hendriks ASCI Conference 2011.
A New Fingertip Detection and Tracking Algorithm and Its Application on Writing-in-the-air System The th International Congress on Image and Signal.
BING: Binarized Normed Gradients for Objectness Estimation at 300fps
A Region Based Stereo Matching Algorithm Using Cooperative Optimization Zeng-Fu Wang, Zhi-Gang Zheng University of Science and Technology of China Computer.
1 Real-Time Stereo-Matching for Micro Air Vehicles Pascal Dufour Master Thesis Presentation.
Window-based Approach For Fast Stereo Correspondence Raj Kumar Gupta, Siu-Yeung Cho IET Computer Vision,
DSP final project proosal From Bilateral-filter to Trilateral-filter : A better improvement on denoising of images R 張錦文.
Fingertip Detection with Morphology and Geometric Calculation Dung Duc Nguyen ; Thien Cong Pham ; Jae Wook Jeon Intelligent Robots and Systems, IEEE/RSJ.
Fast Cost-volume Filtering For Visual Correspondence and Beyond Asmaa Hosni, Member, IEEE, Christoph Rhemann, Michael Bleyer, Member, IEEE, Carsten Rother,
Improved Census Transforms for Resource-Optimized Stereo Vision
Jeong Kanghun CRV (Computer & Robot Vision) Lab..
Efficient Stereo Matching Based on a New Confidence Metric
Predicting the Location and Time of Mobile Phone Users by Using Sequential Pattern Mining Techniques Mert Özer, Ilkcan Keles, Ismail Hakki Toroslu, Pinar.
A global approach Finding correspondence between a pair of epipolar lines for all pixels simultaneously Local method: no guarantee we will have one to.
Stereo Video 1. Temporally Consistent Disparity Maps from Uncalibrated Stereo Videos 2. Real-time Spatiotemporal Stereo Matching Using the Dual-Cross-Bilateral.
Photoconsistency constraint C2 q C1 p l = 2 l = 3 Depth labels If this 3D point is visible in both cameras, pixels p and q should have similar intensities.
Energy minimization Another global approach to improve quality of correspondences Assumption: disparities vary (mostly) smoothly Minimize energy function:
Local Stereo Matching Using Motion Cue and Modified Census in Video Disparity Estimation Zucheul Lee, Ramsin Khoshabeh, Jason Juang and Truong Q. Nguyen.
Video object segmentation and its salient motion detection using adaptive background generation Kim, T.K.; Im, J.H.; Paik, J.K.;  Electronics Letters 
A. M. R. R. Bandara & L. Ranathunga
Summary of “Efficient Deep Learning for Stereo Matching”
Semi-Global Matching with self-adjusting penalties
Efficient Image Classification on Vertically Decomposed Data
SoC and FPGA Oriented High-quality Stereo Vision System
Efficient Image Classification on Vertically Decomposed Data
Image Segmentation Techniques
Housam Babiker, Randy Goebel and Irene Cheng
Presentation transcript:

M.S. Student, Hee-Jong Hong Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Dongbo Min, Member, IEEE, Jiangbo Lu, Member, IEEE, Minh N. Do, Senior Member, IEEE M.S. Student, Hee-Jong Hong Sep 24, 2013

Proposed Method : Improve Cost Aggregation Experimental Results Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Outline Introduction Related Works Proposed Method : Improve Cost Aggregation Experimental Results Conclusion

Introduction Goal:Perform efficient cost aggregation. Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Introduction Goal:Perform efficient cost aggregation. Solution : Joint histogram + reduce redundancy Advantage : Low complexity but keep high-quality.

Related Works Complexity of aggregation:O(NBL) Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Related Works N : all pixels (W*H) B : window size L : disparity level Complexity of aggregation:O(NBL) Reduce complexity approach Scale image : Multi Scale Approach D. Min and K. Sohn, “Cost aggregation and occlusion handling with WLS in stereo matching,” IEEE Trans. on Image Processing, 2008. Bilateral filter : Bilateral Approximation C. Richardt, D. Orr, I. P. Davies, A. Criminisi, and N. A. Dodgson, “Real-time spatiotemporal stereo matching using the dual-cross- bilateral grid,” in European Conf. on Computer Vision, 2010 S. Paris and F. Durand, “A fast approximation of the bilateral filter using a signal processing approach,” International Journal of Computer Vision, 2009. Guided filter : Run in constant time => O(NL) C.Rhemann,A.Hosni,M.Bleyer,C.Rother,andM.Gelautz,“Fast cost-volume filtering for visual correspondence and beyond,” in IEEE Conf. on Computer Vision and Pattern Recognition, 2011

Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Proposed Method

Local Method Algorithm Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Local Method Algorithm Cost initialization : Truncated Absolute Difference => Cost aggregation : Weighted filter Disparity computation : Winner take all [4,8]

Improve Cost Aggregation Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Improve Cost Aggregation New formulation for aggregation Remove normalization Joint histogram representation Compact representation for search range Reduce disparity levels Spatial sampling of matching window Regularly sampled neighboring pixels Pixel-independent sampling

New formulation for aggregation Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 New formulation for aggregation Remove normalization => Joint histogram representation J. Ding, J. Liu, W. Zhou, H. Yu, Y. Wang, and X. Gong, “Realtime stereo vision system using adaptive weight cost aggregation approach,” EURASIP J. Image and Video Processing, vol. 2011, p. 20,

Compact Search Range Cost aggregation O( NBD ) => O( NBDc ) Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Compact Search Range Cost aggregation => MC(q):a subset of disparity levels whose size is Dc. N : all pixels (W*H) B : window size D : disparity level O( NBD ) O( NBDc )

Compact Search Range Non-occluded region of ‘Teddy’ image Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Compact Search Range Non-occluded region of ‘Teddy’ image Dc = 5 (Best) Final Accuracy = 94.2% Dc = 6 Final Accuracy = 94.1% Dc = 60 Final Accuracy = 93.7%

Spatial Sampling of Matching Window Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Spatial Sampling of Matching Window Reason A large matching window and a well-defined weighting function leads to high complexity. Pixels should aggregate in the same object, NOT in the window. Solution Color segmentation => Time consuming (Heavy) Spatial Sampling => Easy but powerful Regular Sampling => Independent from reference pixel => Reduce Complexity

Spatial Sampling of Matching Window Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Spatial Sampling of Matching Window Cost aggregation => S : sampling ratio O( NBDc ) O( NBDc / S2)

Parameter definition Pre-procseeing N : size of image Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Parameter definition N : size of image B : size of matching window N(p)=W×W MD : disparity levels size=D MC : The subset of disparity size=DC<<D S : Sampling ratio Pre-procseeing

Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Experimental Result

Experimental Results Pre-processing Post-processing Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Experimental Results Pre-processing 5*5 Box filter Post-processing Cross-checking technique Weighted median filter (WMF) Device:Intel Xeon 2.8-GHz CPU (using a single core only) and a 6-GB RAM Parameter setting ( ) = (1.5, 1.7, 31*31, 0.11, 13.5, 2.0)

Experimental Results (a) (b) (c) (d) Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Experimental Results (a) (b) (c) (d)

Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Experimental Results Using too large box windows (7×7, 9×9) deteriorates the quality, and incurs more computational overhead. Pre-filtering can be seen as the first cost aggregation step and serves the removal of noise.

Experimental Results The smaller S, the better Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Experimental Results The smaller S, the better Fig. 5. Performance evaluation: average percent (%) of bad matching pixels for ‘nonocc’, ‘all’ and ‘disc’ regions according to Dc and S. 2 better than 1

Experimental Results The smaller S, the longer Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Experimental Results The smaller S, the longer The bigger Dc, the longer

Experimental Results APBP : Average Percentage of Bad Pixels Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Experimental Results APBP : Average Percentage of Bad Pixels

Experimental Results Ground truth Error maps Results Original images Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Experimental Results Ground truth Error maps Results Original images

Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Experimental Results

Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Conclusion

Conclusion Contribution Future work Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 Conclusion Contribution Re-formulate the problem with the relaxed joint histogram. Reduce the complexity of the joint histogram-based aggregation. Achieved both accuracy and efficiency. Future work More elaborate algorithms for selecting the subset of label hypotheses. Estimate the optimal number Dc adaptively. Extend the method to an optical flow estimation.

Thank you!