Structure from Motion with Non-linear Least Squares

Slides:

Advertisements

Similar presentations

Real-Time Template Tracking

Advertisements

Instabilities of SVD Small eigenvalues -> m+ sensitive to small amounts of noise Small eigenvalues maybe indistinguishable from 0 Possible to remove small.

The fundamental matrix F

Lecture 11: Two-view geometry

L1 sparse reconstruction of sharp point set surfaces

Some problems... Lens distortion  Uncalibrated structure and motion recovery assumes pinhole cameras  Real cameras have real lenses  How can we.

Inversion Transforming the apparent to « real » resistivity. Find a numerical model that explains the field measurment.

Siddharth Choudhary.  Refines a visual reconstruction to produce jointly optimal 3D structure and viewing parameters  ‘bundle’ refers to the bundle.

Computer vision: models, learning and inference

Forward and Inverse Kinematics CSE 3541 Matt Boggus.

Some useful linear algebra. Linearly independent vectors span(V): span of vector space V is all linear combinations of vectors v i, i.e.

Methods For Nonlinear Least-Square Problems

Stanford CS223B Computer Vision, Winter 2007 Lecture 8 Structure From Motion Professors Sebastian Thrun and Jana Košecká CAs: Vaibhav Vaish and David Stavens.

Direct Methods for Visual Scene Reconstruction Paper by Richard Szeliski & Sing Bing Kang Presented by Kristin Branson November 7, 2002.

Epipolar geometry. (i)Correspondence geometry: Given an image point x in the first view, how does this constrain the position of the corresponding point.

Stanford CS223B Computer Vision, Winter 2006 Lecture 8 Structure From Motion Professor Sebastian Thrun CAs: Dan Maynes-Aminzade, Mitul Saha, Greg Corrado.

CSCE 641 Computer Graphics: Image-based Modeling (Cont.) Jinxiang Chai.

12 1 Variations on Backpropagation Variations Heuristic Modifications –Momentum –Variable Learning Rate Standard Numerical Optimization –Conjugate.

Global Alignment and Structure from Motion Computer Vision CSE455, Winter 2008 Noah Snavely.

CSCE 641 Computer Graphics: Image-based Modeling (Cont.) Jinxiang Chai.

Lecture 12: Structure from motion CS6670: Computer Vision Noah Snavely.

Structure Computation. How to compute the position of a point in 3- space given its image in two views and the camera matrices of those two views Use.

A plane-plus-parallax algorithm Basic Model: When FOV is not very large and the camera motion has a small rotation, the 2D displacement (u,v) of an image.

CSC 589 Lecture 22 Image Alignment and least square methods Bei Xiao American University April 13.

Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.

CSCE 643 Computer Vision: Structure from Motion

Computer Animation Rick Parent Computer Animation Algorithms and Techniques Optimization & Constraints Add mention of global techiques Add mention of calculus.

Announcements Project 3 due Thursday by 11:59pm Demos on Friday; signup on CMS Prelim to be distributed in class Friday, due Wednesday by the beginning.

Data Modeling Patrice Koehl Department of Biological Sciences National University of Singapore

Variations on Backpropagation.

Distinctive Image Features from Scale-Invariant Keypoints

Chapter 2-OPTIMIZATION G.Anuradha. Contents Derivative-based Optimization –Descent Methods –The Method of Steepest Descent –Classical Newton’s Method.

Announcements No midterm Project 3 will be done in pairs same partners as for project 2.

Lucas-Kanade Image Alignment Iain Matthews. Paper Reading Simon Baker and Iain Matthews, Lucas-Kanade 20 years on: A Unifying Framework, Part 1

STAR SVT Self Alignment V. Perevoztchikov Brookhaven National Laboratory,USA.

Tracking Hands with Distance Transforms Dave Bargeron Noah Snavely.

Lecture 10: Image alignment CS4670/5760: Computer Vision Noah Snavely

Lecture 16: Image alignment

CSCE 441: Computer Graphics Forward/Inverse kinematics

Character Animation Forward and Inverse Kinematics

Answering ‘Where am I?’ by Nonlinear Least Squares

Digital Visual Effects, Spring 2007 Yung-Yu Chuang 2007/4/17

Digital Visual Effects, Spring 2006 Yung-Yu Chuang 2005/4/19

Digital Visual Effects, Spring 2008 Yung-Yu Chuang 2008/4/15

Epipolar geometry.

Structure from motion Input: Output: (Tomasi and Kanade)

CS5321 Numerical Optimization

The Brightness Constraint

Non-linear Least-Squares

Some useful linear algebra

Collaborative Filtering Matrix Factorization Approach

CSCE 441: Computer Graphics Forward/Inverse kinematics

Digital Visual Effects Yung-Yu Chuang

Variations on Backpropagation.

Outline Single neuron case: Nonlinear error correcting learning

Structure from Motion with Non-linear Least Squares

Lecture 7. Learning (IV): Error correcting Learning and LMS Algorithm

Introduction to Scientific Computing II

Introduction to Scientific Computing II

Introduction to Scientific Computing II

Course 7 Motion.

The loss function, the normal equation,

Multi-view geometry.

Mathematical Foundations of BME Reza Shadmehr

Introduction to Scientific Computing II

Variations on Backpropagation.

Performance Optimization

Structure from motion Input: Output: (Tomasi and Kanade)

Lecture 15: Structure from motion

Presentation transcript:

Structure from Motion with Non-linear Least Squares David Bargeron Noah Snavely

Structure from Motion Input: Output Applications Projection of a set of 3D points onto a set of camera projection planes Output 3D point locations Camera motion Applications Object tracking “Match Moves” in TV, movies

Structure from Motion Solution: minimize the residual error of the projections of reconstructed 3D points Algorithm: Levenberg-Marquardt (with Conjugate Gradient under the hood)

Levenberg-Marquardt Function to minimize: Jacobian: Hessian:

Levenberg-Marquardt Inverse Hessian: Steepest Descent: Choose c: Modified Hessian: Levenberg-Marquardt: , so

Synthetic Example #1 Generated test data from a rotating sphere Sphere shape Vertex projections

Synthetic Example #1 Solved for 3D vertex positions Parameters of transformation R(Rj p) + tj (Seven global parameters, four parameters per frame) Ran Levenberg-Marquardt for 35 iterations

Synthetic Example #1 Iteration 0

Synthetic Example #1 Iteration 1

Synthetic Example #1 Iteration 2

Synthetic Example #1 Iteration 3

Synthetic Example #1 Iteration 6

Synthetic Example #1 Iteration 7

Synthetic Example #1 Iteration 8

Synthetic Example #1 Iteration 10

Synthetic Example #1 RMS Projection Error vs. Iteration RMS 3D Error vs. Iteration

Synthetic Example #2 Pig shape Vertex projections

Synthetic Example #2 True pig Reconstructed pig

Real Application – Match Move First tracked points in an input video Video Tracked points (Thanks to Li Zhang for the video)

Real Application – Match Move Solved for 3D points, camera motion Reconstructed points

Real Application – Match Move Used camera motion to insert synthetic object Results!