1. Multi-linear Systems and Invariant Theory
in the Context of Computer Vision and Graphics
Class 3: Infinitesimal Motion
CS329
Stanford University
Amnon Shashua
Class 3
Class 2: Homography Tensors

2. Material We Will Cover Today
Infinitesimal Motion Model
Infinitesimal Planar Homography (8-parameter flow)
Factorization Principle for Motion/Structure Recovery
Direct Estimation
Class 3

3. Infinitesimal Motion Model
Rodriguez Formula:
Class 3

4. Infinitesimal Motion Model
Class 3

5. Reminder:
Assume:
Class 3

6. Infinitesimal Motion Model
Let
Class 3

7. Infinitesimal Motion Model
Class 3

8. Infinitesimal Planar Motion
(the 8-parameter flow)
Class 3

9. Infinitesimal Planar Motion
(the 8-parameter flow)
Class 3

10. Infinitesimal Planar Motion
(the 8-parameter flow)
Note: unlike the discrete case, there is no scale factor
Class 3

11. Reconstruction of Structure/Motion (factorization principle)
Note:
2 interchanges
1 interchanges
Class 3

12. Reconstruction of Structure/Motion (factorization principle)
Class 3

13. Reconstruction of Structure/Motion (factorization principle)
Let
be the “flow” of point i at image j (image 0 is ref frame)
Class 3

14. Reconstruction of Structure/Motion (factorization principle)
Given W, find S,M
(using SVD)
Let
for some
Goal: find
such that
using the “structural” constraints on S
Class 3

15. Reconstruction of Structure/Motion (factorization principle)
Goal: find
such that
using the “structural” constraints on S
Columns 1-3 of S are known, thus columns 1-3 of A can be determined.
Columns 4-6 of A contain 18 unknowns:
eliminate Z and one obtains 5 constraints
Class 3

16. Reconstruction of Structure/Motion (factorization principle)
Goal: find
such that
using the “structural” constraints on S
Let
because
Class 3

17. Reconstruction of Structure/Motion (factorization principle)
because
Each point provides 5 constraints,
thus we need 4 points and 7 views
Class 3

18. Direct Estimation The grey values of images 1,2
Goal: find u,v per pixel
Class 3

19. Direct Estimation Assume: We are assuming that (u,v) can be
found by correlation principle (minimizing
the sum of square differences).
Class 3

20. Direct Estimation
Taylor expansion:
Class 3

21. Direct Estimation
gradient of image 2
image 1 minus image 2
Class 3

22. Direct Estimation
“aperture problem”
Class 3

23. Direct Estimation Estimating parametric flow:
Every pixel contributes one linear equation for the 8 unknowns
Class 3

24. Direct Estimation
Estimating 3-frame Motion:
Combine with:
Class 3

25. Direct Estimation
Let
Class 3

26. Direct Estimation image 1 to image 2 image 1 to image 3
Each pixel contributes a linear equation to the 15 unknown parameters
Class 3

27. Direct Estimation: Factorization
Let
be the “flow” of point i at image j (image 0 is ref frame)
Class 3

28. Direct Estimation: Factorization
Class 3

29. Direct Estimation: Factorization
Recall:
Class 3

30. Direct Estimation: Factorization
Class 3

31. Direct Estimation: Factorization
Rank=6
Rank=6
Enforcing rank=6 constraint on the measurement matrix
removes errors in a least-squares sense.
Class 3

32. Direct Estimation: Factorization
Once U,V are recovered, one can solve for S,M as before.
Class 3