Affine Structure-from-Motion: A lot of frames (1) ISP.

Slides:



Advertisements
Similar presentations
Announcements. Structure-from-Motion Determining the 3-D structure of the world, and/or the motion of a camera using a sequence of images taken by a moving.
Advertisements

Invariants (continued).
3D reconstruction.
3D Reconstruction – Factorization Method Seong-Wook Joo KG-VISA 3/10/2004.
Two-View Geometry CS Sastry and Yang
Mapping: Scaling Rotation Translation Warp
Geometry (Many slides adapted from Octavia Camps and Amitabh Varshney)
Camera calibration and epipolar geometry
Dan Witzner Hansen  Groups?  Improvements – what is missing?
Structure from motion.
Linear Algebra and SVD (Some slides adapted from Octavia Camps)
Structure-from-Motion Determining the 3-D structure of the world, and/or the motion of a camera using a sequence of images taken by a moving camera. –Equivalently,
Epipolar geometry. (i)Correspondence geometry: Given an image point x in the first view, how does this constrain the position of the corresponding point.
Structure from motion. Multiple-view geometry questions Scene geometry (structure): Given 2D point matches in two or more images, where are the corresponding.
Uncalibrated Geometry & Stratification Sastry and Yang
Outline for Today More math… Finish linear algebra: Matrix composition
Multiple-view Reconstruction from Points and Lines
Uncalibrated Epipolar - Calibration
CSci 6971: Image Registration Lecture 2: Vectors and Matrices January 16, 2004 Prof. Chuck Stewart, RPI Dr. Luis Ibanez, Kitware Prof. Chuck Stewart, RPI.
© 2003 by Davi GeigerComputer Vision October 2003 L1.1 Structure-from-EgoMotion (based on notes from David Jacobs, CS-Maryland) Determining the 3-D structure.
Previously Two view geometry: epipolar geometry Stereo vision: 3D reconstruction epipolar lines Baseline O O’ epipolar plane.
Single-view geometry Odilon Redon, Cyclops, 1914.
The Pinhole Camera Model
Boot Camp in Linear Algebra Joel Barajas Karla L Caballero University of California Silicon Valley Center October 8th, 2008.
Stockman MSU/CSE Math models 3D to 2D Affine transformations in 3D; Projections 3D to 2D; Derivation of camera matrix form.
CSCE 641 Computer Graphics: Image-based Modeling (Cont.) Jinxiang Chai.
University of Texas at Austin CS 378 – Game Technology Don Fussell CS 378: Computer Game Technology 3D Engines and Scene Graphs Spring 2012.
Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka Virginia de Sa (UCSD) Cogsci 108F Linear.
Multi-view geometry. Multi-view geometry problems Structure: Given projections of the same 3D point in two or more images, compute the 3D coordinates.
ME451 Kinematics and Dynamics of Machine Systems Review of Matrix Algebra – 2.2 Review of Elements of Calculus – 2.5 Vel. and Acc. of a point fixed in.
Linear Equations in Linear Algebra
6.837 Linear Algebra Review Patrick Nichols Thursday, September 18, 2003.
Patrick Nichols Thursday, September 18, Linear Algebra Review.
Euclidean cameras and strong (Euclidean) calibration Intrinsic and extrinsic parameters Linear least-squares methods Linear calibration Degenerate point.
Camera Geometry and Calibration Thanks to Martial Hebert.
Epipolar geometry The fundamental matrix and the tensor
1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained.
Projective cameras Motivation Elements of Projective Geometry Projective structure from motion Planches : –
© 2005 Yusuf Akgul Gebze Institute of Technology Department of Computer Engineering Computer Vision Geometric Camera Calibration.
Course 12 Calibration. 1.Introduction In theoretic discussions, we have assumed: Camera is located at the origin of coordinate system of scene.
Geometric Models & Camera Calibration
6.837 Linear Algebra Review Patrick Nichols Thursday, September 18, 2003.
Transformations Jehee Lee Seoul National University.
1 1.5 © 2016 Pearson Education, Inc. Linear Equations in Linear Algebra SOLUTION SETS OF LINEAR SYSTEMS.
Affine Structure from Motion
2D Transformation. Transformation changes an object’s: – Position(Translation) – Size(Scaling) – Orientation(Rotation) – Shape(Deformation) Transformation.
EECS 274 Computer Vision Affine Structure from Motion.
1 Chapter 2: Geometric Camera Models Objective: Formulate the geometrical relationships between image and scene measurements Scene: a 3-D function, g(x,y,z)
Affine Geometry.
Auto-calibration we have just calibrated using a calibration object –another calibration object is the Tsai grid of Figure 7.1 on HZ182, which can be used.
Determining 3D Structure and Motion of Man-made Objects from Corners.
Structure from Motion Paul Heckbert, Nov , Image-Based Modeling and Rendering.
Structure from motion Multi-view geometry Affine structure from motion Projective structure from motion Planches : –
EECS 274 Computer Vision Projective Structure from Motion.
Camera Calibration Course web page: vision.cis.udel.edu/cv March 24, 2003  Lecture 17.
Boot Camp in Linear Algebra TIM 209 Prof. Ram Akella.
CS246 Linear Algebra Review. A Brief Review of Linear Algebra Vector and a list of numbers Addition Scalar multiplication Dot product Dot product as a.
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
Review: Transformations
Depth from disparity (x´,y´)=(x+D(x,y), y)
René Vidal and Xiaodong Fan Center for Imaging Science
Review: Transformations
Linear Algebra Review.
Epipolar geometry.
Structure from motion Input: Output: (Tomasi and Kanade)
Uncalibrated Geometry & Stratification
Reconstruction.
The Pinhole Camera Model
Structure from motion Input: Output: (Tomasi and Kanade)
Image Stitching Linda Shapiro ECE P 596.
Presentation transcript:

Affine Structure-from-Motion: A lot of frames (1) ISP

First Step: Solve for Translation (1) This is trivial, because we can pick a simple origin. –World origin is arbitrary. –Example: We can assume first point is at origin. Rotation then doesn’t effect that point. All its motion is translation. –Better to pick center of mass as origin. Average of all points. This also averages all noise.

Specifically, we can never tell where the world points were to begin with. Adding one to every x coordinate in P and then subtracting 1 in every tx is undetectable. So, wlog we can assume that sum(P(k,:)) = 0 for k from 1 to 3, ie., sum(x1 … xn) = 0, sum(y1…yn) = 0, sum(z1 … zn) = 0. Rotation doesn’t move the origin, which is now the center of mass. Neither does scaled orthographic projection. So, this only moves from translation. Explicitly, we assume sum(p) = (0,0,0)^T. Then: sum(s*R(p)) = s*R(sum(p)) = s*R(0,0,0)^T = (0,0,0)^T. (^T means transpose).

More explicitly, suppose sum(p) = (0,0,0,n)^T. Then, sum(R*P) = R*(sum(P)) = R*(0,0,0,n)^T = (0,0,0,n)^T. Sum(T*R*P) = T*(0,0,0,n)^T = (ntx,nty,ntz,n)^T. (Or just look at the 2x4 projection matrix). If we subtract tx or ty from every row, then the residual is (s11,s12,s13;s21,s22,s23)*P. I = s part of matrix + t part of matrix.

Even more explicitly. Consider the first row of the image matrix I. Average together all the entries in this row. This gives us: sum( (s{1,1},s{1,2},s{1,3})*(x_i,y_i,z_i) + tx)/n = (s{1,1},s{1,2},s{1,3})*sum(x_i,y_i,z_i)/n + tx = (s{1,1},s{1,2},s{1,3})*(0,0,0) + tx = tx. So we’ve solved for tx. If we subtract tx from every element in the first row of I, we remove the effects of translation.

First Step: Solve for Translation (2)

First Step: Solve for Translation (3) As if by magic, there’s no translation.

Rank Theorem has rank 3. This means there are 3 vectors such that every row of is a linear combination of these vectors. These vectors are the rows of P. S P

Solve for S SVD is made to do this. D is diagonal with non-increasing values. U and V have orthonormal rows. Ignoring values that get set to 0, we have U(:,1:3) for S, and D(1:3,1:3)*V(1:3,:) for P.

Linear Ambiguity (as before) = U(:,1:3) * D(1:3,1:3) * V(1:3,:) = (U(:,1:3) * A) * (inv(A) *D(1:3,1:3) * V(1:3,:))

Noise has full rank. Best solution is to estimate I that’s as near to as possible, with estimate of I having rank 3. Our current method does this.

Weak Perspective Motion S P Row 2k and 2k+1 of S should be orthogonal. All rows should be unit vectors. (Push all scale into P). =(U(:,1:3)*A)*(inv(A) *D(1:3,1:3)*V(1:3,:)) Choose A so (U(:,1:3) * A) satisfies these conditions.

Related problems we won’t cover Missing data. Points with different, known noise. Multiple moving objects.

Final Messages Structure-from-motion for points can be reduced to linear algebra. Epipolar constraint reemerges. SVD important. Rank Theorem says the images a scene produces aren’t complicated (also important for recognition).