Stereo. STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms.

Slides:



Advertisements
Similar presentations
Lecture 11: Two-view geometry
Advertisements

Stereo matching Class 7 Read Chapter 7 of tutorial Tsukuba dataset.
Stereo Vision Reading: Chapter 11
Gratuitous Picture US Naval Artillery Rangefinder from World War I (1918)!!
Stereo Many slides adapted from Steve Seitz. Binocular stereo Given a calibrated binocular stereo pair, fuse it to produce a depth image Where does the.
MASKS © 2004 Invitation to 3D vision Lecture 7 Step-by-Step Model Buidling.
Stereo.
Last Time Pinhole camera model, projection
Scene Planes and Homographies class 16 Multiple View Geometry Comp Marc Pollefeys.
Stereo and Epipolar geometry
Multiple View Geometry : Computational Photography Alexei Efros, CMU, Fall 2005 © Martin Quinn …with a lot of slides stolen from Steve Seitz and.
Computer Vision cmput 613 Sequential 3D Modeling from images using epipolar geometry and F 3D Modeling from images using epipolar geometry and F Martin.
Multiple View Geometry & Stereo
Computing F and rectification class 14 Multiple View Geometry Comp Marc Pollefeys.
Multi-view stereo Many slides adapted from S. Seitz.
Stereopsis Mark Twain at Pool Table", no date, UCR Museum of Photography.
Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai.
The plan for today Camera matrix
Computer Vision Optical Flow Marc Pollefeys COMP 256 Some slides and illustrations from L. Van Gool, T. Darell, B. Horn, Y. Weiss, P. Anandan, M. Black,
CS 223b 1 More on stereo and correspondence. CS 223b 2 =?f g Mostpopular For each window, match to closest window on epipolar line in other image. (slides.
3D from multiple views : Rendering and Image Processing Alexei Efros …with a lot of slides stolen from Steve Seitz and Jianbo Shi.
Assignment 2 Compute F automatically from image pair (putative matches, 8-point, 7-point, iterative, RANSAC, guided matching) (due by Wednesday 19/03/03)
Lecture 20: Two-view geometry CS6670: Computer Vision Noah Snavely.
May 2004Stereo1 Introduction to Computer Vision CS / ECE 181B Tuesday, May 11, 2004  Multiple view geometry and stereo  Handout #6 available (check with.
3D Vision and Graphics. The Geometry of Multiple Views Motion Field Stereo Epipolar Geometry The Essential Matrix The Fundamental Matrix Structure from.
Lec 21: Fundamental Matrix
CSE473/573 – Stereo Correspondence
Computer Vision Multiple View Geometry & Stereo Marc Pollefeys COMP 256.
Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai.
Multiple View Geometry : Computational Photography Alexei Efros, CMU, Fall 2006 © Martin Quinn …with a lot of slides stolen from Steve Seitz and.
Stereo matching “Stereo matching” is the correspondence problem –For a point in Image #1, where is the corresponding point in Image #2? C1C1 C2C2 ? ? C1C1.
Stereo matching Class 10 Read Chapter 7 Tsukuba dataset.
Review: Binocular stereo If necessary, rectify the two stereo images to transform epipolar lines into scanlines For each pixel x in the first image Find.
3-D Scene u u’u’ Study the mathematical relations between corresponding image points. “Corresponding” means originated from the same 3D point. Objective.
CSCE 641 Computer Graphics: Image-based Modeling Jinxiang Chai.
Computer Vision Spring ,-685 Instructor: S. Narasimhan WH 5409 T-R 10:30am – 11:50am Lecture #15.
Camera Calibration & Stereo Reconstruction Jinxiang Chai.
Structure from images. Calibration Review: Pinhole Camera.
Recap from Monday Image Warping – Coordinate transforms – Linear transforms expressed in matrix form – Inverse transforms useful when synthesizing images.
Stereo Class 7 Read Chapter 7 of tutorial Tsukuba dataset.
Stereo Vision Reading: Chapter 11 Stereo matching computes depth from two or more images Subproblems: –Calibrating camera positions. –Finding all corresponding.
Stereo Vision ECE 847: Digital Image Processing Stan Birchfield Clemson University.
Stereo Many slides adapted from Steve Seitz.
Binocular Stereo #1. Topics 1. Principle 2. binocular stereo basic equation 3. epipolar line 4. features and strategies for matching.
Stereo Many slides adapted from Steve Seitz. Binocular stereo Given a calibrated binocular stereo pair, fuse it to produce a depth image image 1image.
Computer Vision, Robert Pless
EECS 274 Computer Vision Stereopsis.
Computer Vision Lecture #10 Hossam Abdelmunim 1 & Aly A. Farag 2 1 Computer & Systems Engineering Department, Ain Shams University, Cairo, Egypt 2 Electerical.
CSE 185 Introduction to Computer Vision Stereo. Taken at the same time or sequential in time stereo vision structure from motion optical flow Multiple.
Lecture 16: Stereo CS4670 / 5670: Computer Vision Noah Snavely Single image stereogram, by Niklas EenNiklas Een.
stereo Outline : Remind class of 3d geometry Introduction
776 Computer Vision Jan-Michael Frahm Spring 2012.
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography.
Project 2 due today Project 3 out today Announcements TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAA.
Correspondence and Stereopsis Original notes by W. Correa. Figures from [Forsyth & Ponce] and [Trucco & Verri]
John Morris Stereo Vision (continued) Iolanthe returns to the Waitemata Harbour.
Advanced Computer Vision Chapter 11 Stereo Correspondence Presented by: 蘇唯誠 指導教授 : 傅楸善 博士.
Correspondence and Stereopsis. Introduction Disparity – Informally: difference between two pictures – Allows us to gain a strong sense of depth Stereopsis.
Introduction à la vision artificielle VIII Jean Ponce Web:
CSE 185 Introduction to Computer Vision Stereo 2.
Stereo CS4670 / 5670: Computer Vision Noah Snavely Single image stereogram, by Niklas EenNiklas Een.
제 5 장 스테레오.
CS4670 / 5670: Computer Vision Kavita Bala Lec 27: Stereo.
STEREOPSIS The Stereopsis Problem: Fusion and Reconstruction
STEREOPSIS The Stereopsis Problem: Fusion and Reconstruction
EECS 274 Computer Vision Stereopsis.
Thanks to Richard Szeliski and George Bebis for the use of some slides
What have we learned so far?
Chapter 11: Stereopsis Stereopsis: Fusing the pictures taken by two cameras and exploiting the difference (or disparity) between them to obtain the depth.
Stereo vision Many slides adapted from Steve Seitz.
Presentation transcript:

Stereo

STEREOPSIS Reading: Chapter 11. The Stereopsis Problem: Fusion and Reconstruction Human Stereopsis and Random Dot Stereograms Cooperative Algorithms Correlation-Based Fusion Multi-Scale Edge Matching Dynamic Programming Using Three or More Cameras

An Application: Mobile Robot Navigation The Stanford Cart, H. Moravec, The INRIA Mobile Robot, Courtesy O. Faugeras and H. Moravec.

Reconstruction / Triangulation

(Binocular) Fusion

Reconstruction Linear Method: find P such that Non-Linear Method: find Q minimizing

Rectification All epipolar lines are parallel in the rectified image plane.

Image pair rectification simplify stereo matching by warping the images Apply projective transformation so that epipolar lines correspond to horizontal scanlines e e map epipole e to (1,0,0) try to minimize image distortion problem when epipole in (or close to) the image

Planar rectification Bring two views to standard stereo setup (moves epipole to  ) (not possible when in/close to image) ~ image size (calibrated) Distortion minimization (uncalibrated) (standard approach)

Polar re-parameterization around epipoles Requires only (oriented) epipolar geometry Preserve length of epipolar lines Choose  so that no pixels are compressed original image rectified image Polar rectification (Pollefeys et al. ICCV’99) Works for all relative motions Guarantees minimal image size

polar rectification: example

Reconstruction from Rectified Images Disparity: d=u’-u.Depth: z = -B/d.

Human Stereopsis: Binocular Fusion How are the correspondences established? Julesz (1971): Is the mechanism for binocular fusion a monocular process or a binocular one?? There is anecdotal evidence for the latter (camouflage). Random dot stereograms provide an objective answer BP!

A Cooperative Model (Marr and Poggio, 1976) Excitory connections: continuity Inhibitory connections: uniqueness Iterate: C =  C - w  C + C. ei0 Reprinted from Vision: A Computational Investigation into the Human Representation and Processing of Visual Information by David Marr.  1982 by David Marr. Reprinted by permission of Henry Holt and Company, LLC.

Correlation Methods (1970--) Normalized Correlation: minimize  instead. Slide the window along the epipolar line until w.w’ is maximized. 2 Minimize |w-w’|.

Correlation Methods: Foreshortening Problems Solution: add a second pass using disparity estimates to warp the correlation windows, e.g. Devernay and Faugeras (1994). Reprinted from “Computing Differential Properties of 3D Shapes from Stereopsis without 3D Models,” by F. Devernay and O. Faugeras, Proc. IEEE Conf. on Computer Vision and Pattern Recognition (1994).  1994 IEEE.

Multi-Scale Edge Matching (Marr, Poggio and Grimson, ) Edges are found by repeatedly smoothing the image and detecting the zero crossings of the second derivative (Laplacian). Matches at coarse scales are used to offset the search for matches at fine scales (equivalent to eye movements).

Multi-Scale Edge Matching (Marr, Poggio and Grimson, ) One of the two input images Image Laplacian Zeros of the Laplacian Reprinted from Vision: A Computational Investigation into the Human Representation and Processing of Visual Information by David Marr.  1982 by David Marr. Reprinted by permission of Henry Holt and Company, LLC.

Multi-Scale Edge Matching (Marr, Poggio and Grimson, ) Reprinted from Vision: A Computational Investigation into the Human Representation and Processing of Visual Information by David Marr.  1982 by David Marr. Reprinted by permission of Henry Holt and Company, LLC.

The Ordering Constraint But it is not always the case.. In general the points are in the same order on both epipolar lines.

Dynamic Programming (Baker and Binford, 1981) Find the minimum-cost path going monotonically down and right from the top-left corner of the graph to its bottom-right corner. Nodes = matched feature points (e.g., edge points). Arcs = matched intervals along the epipolar lines. Arc cost = discrepancy between intervals.

Dynamic Programming (Ohta and Kanade, 1985) Reprinted from “Stereo by Intra- and Intet-Scanline Search,” by Y. Ohta and T. Kanade, IEEE Trans. on Pattern Analysis and Machine Intelligence, 7(2): (1985).  1985 IEEE.

Three Views The third eye can be used for verification..

More Views (Okutami and Kanade, 1993) Pick a reference image, and slide the corresponding window along the corresponding epipolar lines of all other images, using inverse depth relative to the first image as the search parameter. Use the sum of correlation scores to rank matches. Reprinted from “A Multiple-Baseline Stereo System,” by M. Okutami and T. Kanade, IEEE Trans. on Pattern Analysis and Machine Intelligence, 15(4): (1993). \copyright 1993 IEEE.

Stereo matching Optimal path (dynamic programming ) Similarity measure (SSD or NCC) Constraints epipolar ordering uniqueness disparity limit disparity gradient limit Trade-off Matching cost (data) Discontinuities (prior) (Cox et al. CVGIP’96; Koch’96; Falkenhagen´97; Van Meerbergen,Vergauwen,Pollefeys,VanGool IJCV‘02)

Hierarchical stereo matching Downsampling (Gaussian pyramid) Disparity propagation Allows faster computation Deals with large disparity ranges ( Falkenhagen´97;Van Meerbergen,Vergauwen,Pollefeys,VanGool IJCV‘02)

Disparity map image I(x,y) image I´(x´,y´) Disparity map D(x,y) (x´,y´)=(x+D(x,y),y)

Example: reconstruct image from neighboring images

I1I2I10 Reprinted from “A Multiple-Baseline Stereo System,” by M. Okutami and T. Kanade, IEEE Trans. on Pattern Analysis and Machine Intelligence, 15(4): (1993). \copyright 1993 IEEE.

Real-time stereo on graphics hardware Computes Sum-of-Square-Differences Hardware mip-map generation used to aggregate results over support region Trade-off between small and large support window Ruigang Yang and Marc Pollefeys 140M disparity hypothesis/sec on Radeon 9700pro e.g. 512x512x20disparities at 30Hz Shape of a kernel for summing up 6 levels

(1x1) (1x1+2x2) (1x1+2x2 +4x4+8x8) (1x1+2x2 +4x4+8x8 +16x16) Combine multiple aggregation windows using hardware mipmap and multiple texture units in single pass

Live stereo demo PC with ATI Radeon 9700pro Bumblebee (Point Grey) OpenGL 1.4 (Fragment programs) Code available at:

More on stereo … The Middleburry Stereo Vision Research Page Recommended reading D. Scharstein and R. Szeliski. A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms. IJCV 47(1/2/3):7-42, April-June PDF file (1.15 MB) - includes current evaluation. Microsoft Research Technical Report MSR-TR , November PDF file (1.27 MB).PDF file