Multi-linear Systems and Invariant Theory in the Context of Computer Vision and Graphics Class 3: Infinitesimal Motion CS329 Stanford University Amnon Shashua Class 3 Class 2: Homography Tensors

Material We Will Cover Today Infinitesimal Motion Model Infinitesimal Planar Homography (8-parameter flow) Factorization Principle for Motion/Structure Recovery Direct Estimation Class 3

Infinitesimal Motion Model Rodriguez Formula: Class 3

Infinitesimal Motion Model Class 3

Reminder: Assume: Class 3

Infinitesimal Motion Model Let Class 3

Infinitesimal Motion Model Class 3

Infinitesimal Planar Motion (the 8-parameter flow) Class 3

Infinitesimal Planar Motion (the 8-parameter flow) Class 3

Infinitesimal Planar Motion (the 8-parameter flow) Note: unlike the discrete case, there is no scale factor Class 3

Reconstruction of Structure/Motion (factorization principle) Note: 2 interchanges 1 interchanges Class 3

Reconstruction of Structure/Motion (factorization principle) Class 3

Reconstruction of Structure/Motion (factorization principle) Let be the “flow” of point i at image j (image 0 is ref frame) Class 3

Reconstruction of Structure/Motion (factorization principle) Given W, find S,M (using SVD) Let for some Goal: find such that using the “structural” constraints on S Class 3

Reconstruction of Structure/Motion (factorization principle) Goal: find such that using the “structural” constraints on S Columns 1-3 of S are known, thus columns 1-3 of A can be determined. Columns 4-6 of A contain 18 unknowns: eliminate Z and one obtains 5 constraints Class 3

Reconstruction of Structure/Motion (factorization principle) Goal: find such that using the “structural” constraints on S Let because Class 3

Reconstruction of Structure/Motion (factorization principle) because Each point provides 5 constraints, thus we need 4 points and 7 views Class 3

Direct Estimation The grey values of images 1,2 Goal: find u,v per pixel Class 3

Direct Estimation Assume: We are assuming that (u,v) can be found by correlation principle (minimizing the sum of square differences). Class 3

Direct Estimation Taylor expansion: Class 3

Direct Estimation gradient of image 2 image 1 minus image 2 Class 3

Direct Estimation “aperture problem” Class 3

Direct Estimation Estimating parametric flow: Every pixel contributes one linear equation for the 8 unknowns Class 3

Direct Estimation Estimating 3-frame Motion: Combine with: Class 3

Direct Estimation Let Class 3

Direct Estimation image 1 to image 2 image 1 to image 3 Each pixel contributes a linear equation to the 15 unknown parameters Class 3

Direct Estimation: Factorization Let be the “flow” of point i at image j (image 0 is ref frame) Class 3

Direct Estimation: Factorization Class 3

Direct Estimation: Factorization Recall: Class 3

Direct Estimation: Factorization Class 3

Direct Estimation: Factorization Rank=6 Rank=6 Enforcing rank=6 constraint on the measurement matrix removes errors in a least-squares sense. Class 3

Direct Estimation: Factorization Once U,V are recovered, one can solve for S,M as before. Class 3