Lecture 22: Structure from motion CS6670: Computer Vision Noah Snavely.

Slides:



Advertisements
Similar presentations
Structure from motion.
Advertisements

Assignment 4 Cloning Yourself
The fundamental matrix F
Lecture 11: Two-view geometry
Building Rome in a Day Sameer Agarwal1 Noah Snavely2 Ian Simon1 Steven M. Seitz1 Richard Szeliski3 1University of Washington 2Cornell University 3Microsoft.
Stereo and Projective Structure from Motion
MASKS © 2004 Invitation to 3D vision Lecture 7 Step-by-Step Model Buidling.
Two-View Geometry CS Sastry and Yang
Two-view geometry.
Discrete-Continuous Optimization for Large-scale Structure from Motion David Crandall, Andrew Owens, Noah Snavely, Dan Huttenlocher Presented by: Rahul.
Lecture 11: Structure from Motion
Camera calibration and epipolar geometry
Structure from motion.
Mosaics con’t CSE 455, Winter 2010 February 10, 2010.
Projective structure from motion
Lecture 23: Structure from motion and multi-view stereo
Lecture 11: Structure from motion, part 2 CS6670: Computer Vision Noah Snavely.
Global Alignment and Structure from Motion
Epipolar geometry. (i)Correspondence geometry: Given an image point x in the first view, how does this constrain the position of the corresponding point.
Structure from motion. Multiple-view geometry questions Scene geometry (structure): Given 2D point matches in two or more images, where are the corresponding.
Lecture 21: Multiple-view geometry and structure from motion
3D reconstruction of cameras and structure x i = PX i x’ i = P’X i.
Many slides and illustrations from J. Ponce
Lecture 11: Structure from motion CS6670: Computer Vision Noah Snavely.
Lecture 22: Structure from motion CS6670: Computer Vision Noah Snavely.
Lecture 16: Single-view modeling, Part 2 CS6670: Computer Vision Noah Snavely.
CS664 Lecture #19: Layers, RANSAC, panoramas, epipolar geometry Some material taken from:  David Lowe, UBC  Jiri Matas, CMP Prague
Lecture 20: Two-view geometry CS6670: Computer Vision Noah Snavely.
Advanced Computer Vision Structure from Motion. Geometric structure-from-motion problem: using image matches to estimate: The 3D positions of the corresponding.
Global Alignment and Structure from Motion Computer Vision CSE455, Winter 2008 Noah Snavely.
CSCE 641 Computer Graphics: Image-based Modeling (Cont.) Jinxiang Chai.
Lecture 12: Structure from motion CS6670: Computer Vision Noah Snavely.
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography.
Mosaics CSE 455, Winter 2010 February 8, 2010 Neel Joshi, CSE 455, Winter Announcements  The Midterm went out Friday  See to the class.
Multi-view geometry. Multi-view geometry problems Structure: Given projections of the same 3D point in two or more images, compute the 3D coordinates.
Automatic Camera Calibration
Computer vision: models, learning and inference
Multi-view geometry.
Structure from Motion Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 03/06/12 Many slides adapted from Lana Lazebnik, Silvio Saverese,
Example: line fitting. n=2 Model fitting Measure distances.
CSCE 643 Computer Vision: Structure from Motion
Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.
Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning.
Lecture 03 15/11/2011 Shai Avidan הבהרה : החומר המחייב הוא החומר הנלמד בכיתה ולא זה המופיע / לא מופיע במצגת.
Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( )
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography.
COS429 Computer Vision =++ Assignment 4 Cloning Yourself.
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography.
776 Computer Vision Jan-Michael Frahm & Enrique Dunn Spring 2013.
Announcements No midterm Project 3 will be done in pairs same partners as for project 2.
EECS 274 Computer Vision Projective Structure from Motion.
CSE 185 Introduction to Computer Vision Stereo 2.
Lecture 10: Image alignment CS4670/5760: Computer Vision Noah Snavely
Multi-view geometry. Multi-view geometry problems Structure: Given projections of the same 3D point in two or more images, compute the 3D coordinates.
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
CS4670 / 5670: Computer Vision Kavita Bala Lec 27: Stereo.
Lecture 7: Image alignment
Epipolar geometry.
Modeling the world with photos
Structure from motion Input: Output: (Tomasi and Kanade)
Noah Snavely.
Lecture 23: Structure from motion 2
Multi-view geometry.
Lecture 8: Image alignment
Structure from motion.
Calibration and homographies
Structure from motion Input: Output: (Tomasi and Kanade)
Lecture 15: Structure from motion
CS5760: Computer Vision Lecture 9: RANSAC Noah Snavely
CS5760: Computer Vision Lecture 9: RANSAC Noah Snavely
Presentation transcript:

Lecture 22: Structure from motion CS6670: Computer Vision Noah Snavely

Readings Szeliski, Chapter 7.1 – 7.4

Road map What we’ve seen so far: – Low-level image processing: filtering, edge detecting, feature detection – Geometry: image transformations, panoramas, single- view modeling Fundamental matrices What’s next: – Finishing up geometry – Recognition – Image formation

Back to image filtering Here’s an image with many high frequencies

Large-scale structure from motion Dubrovnik, Croatia. 4,619 images (out of an initial 57,845). Total reconstruction time: 23 hours Number of cores: 352

Structure from motion Given many images, how can we a) figure out where they were all taken from? b) build a 3D model of the scene? This is (roughly) the structure from motion problem

Structure from motion Input: images with points in correspondence p i,j = (u i,j,v i,j ) Output structure: 3D location x i for each point p i motion: camera parameters R j, t j possibly K j Objective function: minimize reprojection error Reconstruction (side) (top)

Also doable from video

What we’ve seen so far… 2D transformations between images – Translations, affine transformations, homographies… Fundamental matrices – Still represent relationships between 2D images What’s new: Explicitly representing 3D geometry of cameras and points

Input

Camera calibration and triangulation Suppose we know 3D points – And have matches between these points and an image – How can we compute the camera parameters? Suppose we have know camera parameters, each of which observes a point – How can we compute the 3D location of that point?

Structure from motion SfM solves both of these problems at once A kind of chicken-and-egg problem – (but solvable)

Photo Tourism

First step: how to get correspondence? Feature detection and matching

Feature detection Detect features using SIFT [Lowe, IJCV 2004]

Feature detection Detect features using SIFT [Lowe, IJCV 2004]

Feature matching Match features between each pair of images

Feature matching Refine matching using RANSAC to estimate fundamental matrix between each pair

Image connectivity graph (graph layout produced using the Graphviz toolkit:

Structure from motion Camera 1 Camera 2 Camera 3 R 1,t 1 R 2,t 2 R 3,t 3 X1X1 X4X4 X3X3 X2X2 X5X5 X6X6 X7X7 minimize g(R, T, X) p 1,1 p 1,2 p 1,3 non-linear least squares

Problem size What are the variables? How many variables per camera? How many variables per point? Trevi Fountain collection 466 input photos + > 100,000 3D points = very large optimization problem

Structure from motion Minimize sum of squared reprojection errors: Minimizing this function is called bundle adjustment – Optimized using non-linear least squares, e.g. Levenberg-Marquardt predicted image location observed image location indicator variable: is point i visible in image j ?

Is SfM always uniquely solvable?

No…