1. Structure From Motion Sebastian Thrun, Gary Bradski, Daniel Russakoff
Stanford CS223B Computer Vision

2. Structure From Motion (1)
[Tomasi & Kanade 92]

3. Structure From Motion (2)
[Tomasi & Kanade 92]

4. Structure From Motion (3)
[Tomasi & Kanade 92]

5. Structure From Motion Problem 1: Problem 2:
Given n points pij =(xij, yij) in m images
Reconstruct structure: 3-D locations Pj =(xj, yj, zj)
Reconstruct camera positions (extrinsics) Mi=(Aj, bj)
Problem 2:
Establish correspondence: c(pij)

6. Orthographic Camera Model
Limit of Pinhole Model:
Extrinsic Parameters
Rotation
Orthographic Projection

7. Orthographic Projection
Limit of Pinhole Model:
Orthographic Projection

8. The Affine SFM Problem

9. Count # Constraints vs #Unknowns
m camera poses
n points
2mn point constraints
8m+3n unknowns
Suggests: need 2mn  8m + 3n
But: Can we really recover all parameters???

10. How Many Parameters Can’t We Recover?
We can recover all but… 3 6 8 9 10 12 n m nm
Place Your Bet!

11. The Answer is (at least): 12

12. Points for Solving Affine SFM Problem
m camera poses
n points
Need to have: 2mn  8m + 3n-12

13. Affine SFM Fix coordinate system by making p0=origin Rank Theorem:
Proof:
Rank Theorem:
Q has rank 3

14. The Rank Theorem
2m elements
n elements

15. Tomasi/Kanade 1992
Singular Value Decomposition

16. Tomasi/Kanade 1992
Gives also the optimal affine reconstruction under noise

17. Back To Orthographic Projection
Find C and d for which constraints are met

18. Back To Projective Geometry
Orthographic (in the limit)
Projective

19. Projective Camera:
Non-Linear Optimization Problem: Bundle Adjustment!

20. Structure From Motion Problem 1: Problem 2:
Given n points pij =(xij, yij) in m images
Reconstruct structure: 3-D locations Pj =(xj, yj, zj)
Reconstruct camera positions (extrinsics) Mi=(Aj, bj)
Problem 2:
Establish correspondence: c(pij)

21. The Correspondence Problem
View 1
View 2
View 3

22. Correspondence: Solution 1
Track features (e.g., optical flow)
…but fails when images taken from widely different poses

23. Correspondence: Solution 2
Start with random solution A, b, P
Compute soft correspondence: p(c|A,b,P)
Plug soft correspondence into SFM
Reiterate
See Dellaert/Seitz/Thorpe/Thrun 2003

24. Example

25. Results: Cube

26. Animation

27. Tomasi’s Benchmark Problem

28. Reconstruction with EM

29. 3-D Structure