Computer Vision : CISC 4/689 Adaptation from: Prof. James M. Rehg, G.Tech.

Slides:



Advertisements
Similar presentations
Stereo Vision Reading: Chapter 11
Advertisements

Public Library, Stereoscopic Looking Room, Chicago, by Phillips, 1923.
Gratuitous Picture US Naval Artillery Rangefinder from World War I (1918)!!
Stereo Many slides adapted from Steve Seitz. Binocular stereo Given a calibrated binocular stereo pair, fuse it to produce a depth image Where does the.
Recap from Previous Lecture Tone Mapping – Preserve local contrast or detail at the expense of large scale contrast. – Changing the brightness within.
Lecture 8: Stereo.
Stereo.
Last Time Pinhole camera model, projection
Stanford CS223B Computer Vision, Winter 2005 Lecture 6: Stereo 2 Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.
CS6670: Computer Vision Noah Snavely Lecture 17: Stereo
Multiple View Geometry : Computational Photography Alexei Efros, CMU, Fall 2005 © Martin Quinn …with a lot of slides stolen from Steve Seitz and.
The Fundamental Matrix F
Stereo & Iterative Graph-Cuts Alex Rav-Acha Vision Course Hebrew University.
Stanford CS223B Computer Vision, Winter 2006 Lecture 6 Stereo II Professor Sebastian Thrun CAs: Dan Maynes-Aminzade, Mitul Saha, Greg Corrado Stereo.
Stereo Binocular Stereo Calibration (finish up) Next Time Motivation
Stereopsis Mark Twain at Pool Table", no date, UCR Museum of Photography.
The plan for today Camera matrix
CS 223b 1 More on stereo and correspondence. CS 223b 2 =?f g Mostpopular For each window, match to closest window on epipolar line in other image. (slides.
3D from multiple views : Rendering and Image Processing Alexei Efros …with a lot of slides stolen from Steve Seitz and Jianbo Shi.
Stereo Computation using Iterative Graph-Cuts
CSE473/573 – Stereo Correspondence
Announcements PS3 Due Thursday PS4 Available today, due 4/17. Quiz 2 4/24.
Stereo Guest Lecture by Li Zhang
Project 1 artifact winners Project 2 questions Project 2 extra signup slots –Can take a second slot if you’d like Announcements.
Midterm went out on Tuesday (due next Tuesday) Project 3 out today Announcements.
Stereo Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with slides by James Rehg and.
Multiple View Geometry : Computational Photography Alexei Efros, CMU, Fall 2006 © Martin Quinn …with a lot of slides stolen from Steve Seitz and.
VZvZ r t b image cross ratio Measuring height B (bottom of object) T (top of object) R (reference point) ground plane H C scene cross ratio  scene points.
Stereo matching Class 10 Read Chapter 7 Tsukuba dataset.
Computer Vision Spring ,-685 Instructor: S. Narasimhan WH 5409 T-R 10:30am – 11:50am Lecture #15.
Stereo Readings Trucco & Verri, Chapter 7 –Read through 7.1, 7.2.1, 7.2.2, 7.3.1, 7.3.2, and 7.4, –The rest is optional. Single image stereogram,
Announcements Project 1 artifact winners Project 2 questions
Structure from images. Calibration Review: Pinhole Camera.
Lecture 12 Stereo Reconstruction II Lecture 12 Stereo Reconstruction II Mata kuliah: T Computer Vision Tahun: 2010.
Recap from Monday Image Warping – Coordinate transforms – Linear transforms expressed in matrix form – Inverse transforms useful when synthesizing images.
Stereo Vision Reading: Chapter 11 Stereo matching computes depth from two or more images Subproblems: –Calibrating camera positions. –Finding all corresponding.
Stereo Dan Kong.
Geometry 3: Stereo Reconstruction Introduction to Computer Vision Ronen Basri Weizmann Institute of Science.
Stereo Readings Szeliski, Chapter 11 (through 11.5) Single image stereogram, by Niklas EenNiklas Een.
Stereo Many slides adapted from Steve Seitz.
CS 4487/6587 Algorithms for Image Analysis
Project 2 code & artifact due Friday Midterm out tomorrow (check your ), due next Fri Announcements TexPoint fonts used in EMF. Read the TexPoint.
Stereo Many slides adapted from Steve Seitz. Binocular stereo Given a calibrated binocular stereo pair, fuse it to produce a depth image image 1image.
Computer Vision, Robert Pless
A Region Based Stereo Matching Algorithm Using Cooperative Optimization Zeng-Fu Wang, Zhi-Gang Zheng University of Science and Technology of China Computer.
Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.
Computer Vision Stereo Vision. Bahadir K. Gunturk2 Pinhole Camera.
Bahadir K. Gunturk1 Phase Correlation Bahadir K. Gunturk2 Phase Correlation Take cross correlation Take inverse Fourier transform  Location of the impulse.
Lecture 16: Stereo CS4670 / 5670: Computer Vision Noah Snavely Single image stereogram, by Niklas EenNiklas Een.
Jochen Triesch, UC San Diego, 1 Stereo Outline: parallel camera axes convergent axes, epipolar geometry correspondence.
Digital Image Processing
Solving for Stereo Correspondence Many slides drawn from Lana Lazebnik, UIUC.
A global approach Finding correspondence between a pair of epipolar lines for all pixels simultaneously Local method: no guarantee we will have one to.
Project 2 due today Project 3 out today Announcements TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAA.
Advanced Computer Vision Chapter 11 Stereo Correspondence Presented by: 蘇唯誠 指導教授 : 傅楸善 博士.
Project 2 artifacts—vote now!! Project 3 questions? Start thinking about final project ideas, partners Announcements.
Stereo Matching Using Dynamic Programming
Project 3 code & artifact due Tuesday Final project proposals due noon Wed (by ) –One-page writeup (from project web page), specifying: »Your team.
Computational Vision CSCI 363, Fall 2012 Lecture 17 Stereopsis II
數位三維視訊 楊 家 輝 Jar-Ferr Yang 電腦與通信工程研究所 電機工程學系 國立成功大學 Institute of Computer and Communication Engineering Department of Electrical Engineering National Cheng.
Energy minimization Another global approach to improve quality of correspondences Assumption: disparities vary (mostly) smoothly Minimize energy function:
CSE 185 Introduction to Computer Vision Stereo 2.
Stereo CS4670 / 5670: Computer Vision Noah Snavely Single image stereogram, by Niklas EenNiklas Een.
CS4670 / 5670: Computer Vision Kavita Bala Lec 27: Stereo.
Geometry 3: Stereo Reconstruction
Announcements Midterms graded (handed back at end of lecture)
Thanks to Richard Szeliski and George Bebis for the use of some slides
What have we learned so far?
Computer Vision Stereo Vision.
Stereo vision Many slides adapted from Steve Seitz.
Presentation transcript:

Computer Vision : CISC 4/689 Adaptation from: Prof. James M. Rehg, G.Tech

Computer Vision : CISC 4/689 Public Library, Stereoscopic Looking Room, Chicago, by Phillips, 1923

Computer Vision : CISC 4/689 Teesta suspension bridge-Darjeeling, India

Computer Vision : CISC 4/689 Mark Twain at Pool Table", no date, UCR Museum of Photography

Computer Vision : CISC 4/689 Woman getting eye exam during immigration procedure at Ellis Island, c , UCR Museum of Phography

Computer Vision : CISC 4/689 Stereo results Ground truthScene –Data from University of Tsukuba (Seitz)

Computer Vision : CISC 4/689 Results with window correlation Window-based matching (best window size) Ground truth (Seitz)

Computer Vision : CISC 4/689 Results with better method State of the art method Boykov et al., Fast Approximate Energy Minimization via Graph Cuts,Fast Approximate Energy Minimization via Graph Cuts International Conference on Computer Vision, September Ground truth (Seitz)

Computer Vision : CISC 4/689 Stereo Constraint Color constancy –The color of any world points remains constant from image to image –This assumption is true under Lambertian Model –In practice, given photometric camera calibration and typical scenes, color constancy holds well enough for most stereo algorithms.

Computer Vision : CISC 4/689 Stereo Constraint Epipolar geometry –The epipolar geometry is the fundamental constraint in stereo. –Rectification aligns epipolar lines with scanlines Epipolar plane Epipolar line for p Epipolar line for p’

Computer Vision : CISC 4/689 Stereo Constraint Uniqueness and Continuity –Proposed by Marr&Poggio. –Each item from each image may be assigned at most one disparity value, ” and the “ disparity ” varies smoothly almost everywhere.

Computer Vision : CISC 4/689 Problems with Window-based matching Disparity within the window must be constant. Bias the results towards frontal-parallel surfaces. Blur across depth discontinuities. Perform poorly in textureless regions. Erroneous results in occluded regions

Computer Vision : CISC 4/689 Cooperative Stereo Algorithm Based on two basic assumption by Marr and Poggio: –Uniqueness: at most a single unique match exists for each pixel. –Continuous: disparity values are generally continuous, i.e., smooth within a local neighborhood.

Computer Vision : CISC 4/689 Disparity Space Image (DSI) The 3D disparity space has dimensions row r column c and disparity d. Each element (r, c, d) of the disparity space projects to the pixel (r, c) in the left image and to the (r, c + d) in the right image DSI represents the confidence or likelihood of a particular match.

Computer Vision : CISC 4/689 Illustration of DSI (r, c) slices for different d (c, d) slice for r = 151

Computer Vision : CISC 4/689 Definition Match value assigned to element (r, c, d) at iteration n Initial values computed from SSD or NCC Inhibition area for element (r, c, d) Local support area for element (r, c, d)

Computer Vision : CISC 4/689 Illustration of Inhibitory and Support Regions If d is disparity for (r,c), d -1 is for (r,c-1) And d+1 for (r,c+1), etc. Vertical inhibition is self explanatory. Diagonal basically means, r,c+1in left image cannot map to r,c+d in right image, i.e, it cannot have disparity of d-1. Similarly, r,c-1 in left image cannot map to r,c+d so it cannot have disparity of d+1. This satisfies uniqueness constraint.

Computer Vision : CISC 4/689 Iterative Updating DSI Questions to ponder: i)What is typical alpha? ii)Do we include inhibition region in numerator of Rn?

Computer Vision : CISC 4/689 Explicit Detection of Occlusion Identify occlusions by examining the magnitude of the converged values in conjunction with the uniqueness constrain

Computer Vision : CISC 4/689 Summary of Cooperative Stereo Prepare a 3D array, (r, c, d): (r, c) for each pixel in the reference image and d for the range of disparity. Set initial match values using a function of image intensities, such as normalized correlation or SSD. Iteratively update match values using (4) until the match values converge. For each pixel (r, c), find the element (r, c, d) with the maximum match value. If the maximum match value is higher than a threshold, output the disparity d, otherwise, declare a occlusion.

Computer Vision : CISC 4/689 Ordering Constraint If an object a is left on an object b in the left image then object a will also appear to the left of object b in the right image Ordering constraint……and its failure

Computer Vision : CISC 4/689 Stereo Correspondences …… Left scanlineRight scanline Match intensities sequentially between two scanlines

Computer Vision : CISC 4/689 Stereo Correspondences …… Left scanlineRight scanline Match Left occlusionRight occlusion

Computer Vision : CISC 4/689 Search Over Correspondences Three cases: –Sequential – cost of match –Left occluded – cost of no match –Right occluded – cost of no match Left scanline Right scanline Left Occluded Pixels Right occluded Pixels

Computer Vision : CISC 4/689 Standard 3-move Dynamic Programming for Stereo Dynamic programming yields the optimal path through grid. This is the best set of matches that satisfy the ordering constraint Left Occluded Pixels Left scanline Right occluded Pixels Right scanline Start End

Computer Vision : CISC 4/689 Dynamic Programming Efficient algorithm for solving sequential decision (optimal path) problems … How many paths through this trellis?

Computer Vision : CISC 4/689 Dynamic Programming Suppose cost can be decomposed into stages: States:

Computer Vision : CISC 4/689 Dynamic Programming Principle of Optimality for an n-stage assignment problem

Computer Vision : CISC 4/689 Dynamic Programming

Computer Vision : CISC 4/689 Stereo Matching with Dynamic Programming Scan across grid computing optimal cost for each node given its upper-left neighbors. Backtrack from the terminal to get the optimal path. Occluded Pixels Left scanline Dis-occluded Pixels Right scanline Terminal

Computer Vision : CISC 4/689 Stereo Matching with Dynamic Programming Scan across grid computing optimal cost for each node given its upper-left neighbors. Backtrack from the terminal to get the optimal path. Occluded Pixels Left scanline Dis-occluded Pixels Right scanline Terminal

Computer Vision : CISC 4/689 Stereo Matching with Dynamic Programming Scan across grid computing optimal cost for each node given its upper-left neighbors. Backtrack from the terminal to get the optimal path. Occluded Pixels Left scanline Dis-occluded Pixels Right scanline Terminal

Computer Vision : CISC 4/689 Stereo Matching with Dynamic Programming Scan across grid computing optimal cost for each node given its upper-left neighbors. Backtrack from the terminal to get the optimal path. Occluded Pixels Left scanline Dis-occluded Pixels Right scanline Terminal

Computer Vision : CISC 4/689 Stereo Matching with Dynamic Programming Scan across grid computing optimal cost for each node given its upper-left neighbors. Backtrack from the terminal to get the optimal path. Occluded Pixels Left scanline Dis-occluded Pixels Right scanline Terminal

Computer Vision : CISC 4/689 Stereo Matching with Dynamic Programming Scan across grid computing optimal cost for each node given its upper-left neighbors. Backtrack from the terminal to get the optimal path. Occluded Pixels Left scanline Dis-occluded Pixels Right scanline Terminal

Computer Vision : CISC 4/689 Stereo Matching with Dynamic Programming Scan across grid computing optimal cost for each node given its upper-left neighbors. Backtrack from the terminal to get the optimal path. For each (i,j), look for what is Optimal to get to: would it be From (i-1,j-1), or (i,j-1), or (i-1,j-1)? Assign large values for left occlusion and right occlusion. Occluded Pixels Left scanline Dis-occluded Pixels Right scanline Terminal

Computer Vision : CISC 4/689 Stereo Matching with Dynamic Programming Pseudo-code describing how to calculate the optimal match

Computer Vision : CISC 4/689 Stereo Matching with Dynamic Programming Pseudo-code describing how to reconstruct the optimal path

Computer Vision : CISC 4/689 Results Local errors may be propagated along a scan-line and no inter scan-line consistency is enforced.

Computer Vision : CISC 4/689 Assumption Behind Segmentation-based Stereo Depth discontinuity tend to correlate well with color edges Disparity variation within a segment is small Approximating the scene with piece-wise planar surfaces

Computer Vision : CISC 4/689 Segmentation-based stereo Plane equation is fitted in each segment based on initial disparity estimation obtained SSD or Correlation Global matching criteria: if a depth map is good, warping the reference image to the other view according to this depth will render an image that matches the real view Optimization by iterative neighborhood depth hypothesizing

Computer Vision : CISC 4/689 Result

Computer Vision : CISC 4/689 Another Result