Stereo Matching Segment-based Belief Propagation Iolanthe II racing in Waitemata Harbour.

Slides:



Advertisements
Similar presentations
Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard CVPR 2014 Yongho Shin.
Advertisements

November 12, 2013Computer Vision Lecture 12: Texture 1Signature Another popular method of representing shape is called the signature. In order to compute.
Presented By: Vennela Sunnam
Spatial-Temporal Consistency in Video Disparity Estimation ICASSP 2011 Ramsin Khoshabeh, Stanley H. Chan, Truong Q. Nguyen.
The SIFT (Scale Invariant Feature Transform) Detector and Descriptor
Computer Vision Lecture 16: Texture
1Ellen L. Walker Edges Humans easily understand “line drawings” as pictures.
Last Time Pinhole camera model, projection
Segmentation Divide the image into segments. Each segment:
Stereo & Iterative Graph-Cuts Alex Rav-Acha Vision Course Hebrew University.
Introduction to Computer Vision 3D Vision Topic 9 Stereo Vision (I) CMPSCI 591A/691A CMPSCI 570/670.
The plan for today Camera matrix
Optical flow and Tracking CISC 649/849 Spring 2009 University of Delaware.
Stereo Computation using Iterative Graph-Cuts
Lecture 11: Stereo and optical flow CS6670: Computer Vision Noah Snavely.
Lecture 4: Edge Based Vision Dr Carole Twining Thursday 18th March 2:00pm – 2:50pm.
Scale-Invariant Feature Transform (SIFT) Jinxiang Chai.
Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’04) /04 $20.00 c 2004 IEEE 1 Li Hong.
Michael Bleyer LVA Stereo Vision
A Rapid Stereo Matching Algorithm Based on Disparity Interpolation Gang Yao Yong Liu Bangjun Lei Dong Ren Institute of Intelligent Vision and Image Information.
Lecture 12 Stereo Reconstruction II Lecture 12 Stereo Reconstruction II Mata kuliah: T Computer Vision Tahun: 2010.
A Local Adaptive Approach for Dense Stereo Matching in Architectural Scene Reconstruction C. Stentoumis 1, L. Grammatikopoulos 2, I. Kalisperakis 2, E.
7.1. Mean Shift Segmentation Idea of mean shift:
From Pixels to Features: Review of Part 1 COMP 4900C Winter 2008.
Object Stereo- Joint Stereo Matching and Object Segmentation Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on Michael Bleyer Vienna.
CS654: Digital Image Analysis Lecture 8: Stereo Imaging.
Course 13 Curves and Surfaces. Course 13 Curves and Surface Surface Representation Representation Interpolation Approximation Surface Segmentation.
EDGE DETECTION IN COMPUTER VISION SYSTEMS PRESENTATION BY : ATUL CHOPRA JUNE EE-6358 COMPUTER VISION UNIVERSITY OF TEXAS AT ARLINGTON.
CS 4487/6587 Algorithms for Image Analysis
EECS 274 Computer Vision Segmentation by Clustering II.
Feature-Based Stereo Matching Using Graph Cuts Gorkem Saygili, Laurens van der Maaten, Emile A. Hendriks ASCI Conference 2011.
Course 9 Texture. Definition: Texture is repeating patterns of local variations in image intensity, which is too fine to be distinguished. Texture evokes.
December 9, 2014Computer Vision Lecture 23: Motion Analysis 1 Now we will talk about… Motion Analysis.
A Region Based Stereo Matching Algorithm Using Cooperative Optimization Zeng-Fu Wang, Zhi-Gang Zheng University of Science and Technology of China Computer.
1 University of Texas at Austin Machine Learning Group 图像与视频处理 计算机学院 Motion Detection and Estimation.
Effective Optical Flow Estimation
CS654: Digital Image Analysis Lecture 30: Clustering based Segmentation Slides are adapted from:
Computer Vision Lecture #10 Hossam Abdelmunim 1 & Aly A. Farag 2 1 Computer & Systems Engineering Department, Ain Shams University, Cairo, Egypt 2 Electerical.
Levels of Image Data Representation 4.2. Traditional Image Data Structures 4.3. Hierarchical Data Structures Chapter 4 – Data structures for.
Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.
Mean Shift ; Theory and Applications Presented by: Reza Hemati دی 89 December گروه بینایی ماشین و پردازش تصویر Machine Vision and Image Processing.
Course14 Dynamic Vision. Biological vision can cope with changing world Moving and changing objects Change illumination Change View-point.
Jeong Kanghun CRV (Computer & Robot Vision) Lab..
A global approach Finding correspondence between a pair of epipolar lines for all pixels simultaneously Local method: no guarantee we will have one to.
John Morris Stereo Vision (continued) Iolanthe returns to the Waitemata Harbour.
Advanced Computer Vision Chapter 11 Stereo Correspondence Presented by: 蘇唯誠 指導教授 : 傅楸善 博士.
Machine Vision Edge Detection Techniques ENT 273 Lecture 6 Hema C.R.
John Morris These slides were adapted from a set of lectures written by Mircea Nicolescu, University of Nevada at Reno Stereo Vision Iolanthe in the Bay.
Instructor: Mircea Nicolescu Lecture 7
Instructor: Mircea Nicolescu Lecture 5 CS 485 / 685 Computer Vision.
Image Features (I) Dr. Chang Shu COMP 4900C Winter 2008.
Photoconsistency constraint C2 q C1 p l = 2 l = 3 Depth labels If this 3D point is visible in both cameras, pixels p and q should have similar intensities.
Instructor: Mircea Nicolescu Lecture 10 CS 485 / 685 Computer Vision.
Energy minimization Another global approach to improve quality of correspondences Assumption: disparities vary (mostly) smoothly Minimize energy function:
Correspondence and Stereopsis. Introduction Disparity – Informally: difference between two pictures – Allows us to gain a strong sense of depth Stereopsis.
April 21, 2016Introduction to Artificial Intelligence Lecture 22: Computer Vision II 1 Canny Edge Detector The Canny edge detector is a good approximation.
CS 6501: 3D Reconstruction and Understanding Stereo Cameras
A Plane-Based Approach to Mondrian Stereo Matching
CS262: Computer Vision Lect 09: SIFT Descriptors
Summary of “Efficient Deep Learning for Stereo Matching”
Image Pre-Processing in the Spatial and Frequent Domain
Semi-Global Stereo Matching with Surface Orientation Priors
Image gradients and edges
Presented by: Cindy Yan EE6358 Computer Vision
Multiway Cut for Stereo and Motion with Slanted Surfaces
Dr. Chang Shu COMP 4900C Winter 2008
Binocular Stereo Vision
Image and Video Processing
Detection of salient points
Detecting and analysing motion
Presentation transcript:

Stereo Matching Segment-based Belief Propagation Iolanthe II racing in Waitemata Harbour

Segmentation #1 on the Middlebury rankings Klaus, Sormann and Karner, “Segment-based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure”, ICPR (3) 2006: ICPR (3) 2006 Basic idea Depth (disparity) changes should occur at region boundaries in the image so Segment the image and match the patches

Algorithm Pair of rectified images Extract homogenous regions in reference image Apply local window based matching Extract set of disparity planes Approximate optimal disparity plane assignment Disparity map

Step 1 - Segmentation Decompose reference image  Regions of homogenous colour | grey-scale Assume Disparity values vary smoothly within these regions Note: Possibly sloping planes Fronto-planar assumption not required! Mean-shift colour segmentation used Gradient descent search for maxima in a density function over a high dimensional feature space Feature space: spatial coordinates + associated attributes (including edge information)

Step 2 – Local matching Local matching in pixel domain Window based correlation (SSD or SAD) Gradient or non-parametric matching algorithms possible  Better tolerate gain and offset changes KSK use self-adapting combination score C(x,y,d) = (1 -  ) C SAD (x,y,d) +  C grad (x,y,d) where C SAD (x,y,d) =  i,j  N | I L (i,j) – I R (i+d,j) | C grad (x,y,d) =  i,j  N |  x I L (i,j) -  x I R (i+d,j) | +  i,j  N |  y I L (i,j) -  y I R (i+d,j) | L  R Difference in Intensity L  R Difference in Horizontal Gradient L  R Difference in Vertical Gradient

Step 2 – Local matching KSK use self-adapting combination score C(x,y,d) = (1 -  ) C SAD (x,y,d) +  C grad (x,y,d) where C SAD (x,y,d) =  i,j  N | I L (i,j) – I R (i+d,j) | C grad (x,y,d) =  i,j  N |  x I L (i,j) -  x I R (i+d,j) | +  i,j  N |  y I L (i,j) -  y I R (i+d,j) | N(x,y) is a 3  3 window Nx(x,y) is a 3  2 window; Ny(x,y) is a 2  3 window Weighting,  chosen Winner-take-all – choose disparity with lowest cost + Maximising number of reliable correspondences filtered out by L  R crosscheck L  R Difference in Intensity L  R Difference in Horizontal Gradient L  R Difference in Vertical Gradient

Step 2 – Local matching Weighting,  chosen Winner-take-all – choose disparity with lowest cost + Maximising number of reliable correspondences filtered out by L  R crosscheck Normalized dissimilarity measure used Reliable correspondences used to estimate signal- noise ratio (SNR) Because of normalization, a fixed truncation threshhold is set just above the noise level  Robust matching score

Step 3 – Disparity plane estimation Allow sloping planes Abandon fronto-planar surface assumption! Disparity plane estimation Specify the plane with c 1, c 2, c 3 : d = c 1 x + c 2 y + c 3  Large number of possible planes  Use only reliable disparities Decompose fitting problem: Decompose fitting problem: Fit Horizontal and vertical slant separately Find  d/  x (  d/  y ) for each pixel Distribution of derivatives  d/  x (  d/  y ) is convolved with Gaussian kernel to determine mean  d/  x Slant in the centre of a segment then determined by finding distribution of centre disparities for each reliable point as before

Step 3 – Disparity plane estimation Allow sloping planes Abandon fronto-planar surface assumption! Estimate disparity planes Specify a plane with c 1, c 2, c 3 : d = c 1 x + c 2 y + c 3  Large number of possible planes  Use only reliable disparities Decompose fitting problem: Fit Horizontal and vertical slant separately Find  d/  x (  d/  y ) for each pixel Distribution of derivatives  d/  x (  d/  y ) is convolved with Gaussian kernel to determine mean  d/  x Slant in the centre of a segment then determined from distribution of centre disparities for each reliable point as before

Step 4 – Disparity plane refinement Increase the accuracy of the disparity plane Group regions that belong to the same disparity plane Calculate a matching cost For the plane, P, fitted to segment, S : C SEG (S,P) =  (x,y)  S C(x,y,d) Disparity plane with the minimum matching cost is assigned to each segment Segments assigned to that disparity plane are grouped Repeat for all grouped segments For each pixel in segment, S Mismatch cost for disparity, d

Step 5 – Disparity plane assignment Search for optimal segment  disparity place assignment Formulate as an energy minimization problem Find a labelling, f, which matches a segment, s  R, to a plane, f(s) in D ‘Energy’ for labelling f is E(f) = E data (f) + E smooth (f) where E data (f) =  s  R C SEG (s,f(s)) and E smooth (f) =  (  (si,sj)  SN|f(si)  f(sj) ) disc (s i,s j )  disc (s i,s j ) – discontinuity penalty – includes common border lengths and colour similarity S N set of adjacent segments

Step 5 Use loopy belief propagation to find optimal labeling with minimum energy Results: Matching – state-of-the-art! Top ‘average’ rank on the Middlebury set Sloping planes of ‘Venus’ are well handled Still has problems with edges!! See results in fig 2 of the paper Pixelization not handled? Computation cost? Not mentioned in paper! Algorithm contains many repeated steps Analysis of benefit of each one useful