1. Structure from Motion Course web page: vision.cis.udel.edu/~cv April 25, 2003  Lecture 26

2. Announcements Read Hartley & Zisserman Chapter 17.2 (skip 17.2.1) and Forsyth & Ponce Chapter 12.3 on affine structure from motion for Monday Homework 4 due on Monday

3. Outline Triangulation Stratified reconstruction

4. Computing Structure Recall that canonical camera matrices P, P’ can be computed from fundamental matrix F –E.g. P = [Id j 0] and P’ = [[e’] £ F j e’], Triangulation: Back-projection of rays from image points x, x’ to 3-D point of intersection X such that x = PX and x’ = P’X from Hartley & Zisserman

5. Triangulation: Issues Errors in points x, x’ ) @ F such that x’ T F x = 0 or X such that x = PX and x’ = P’X This means that rays are skew — they don’t intersect from Hartley & Zisserman

6. Triangulation with Non-Intersecting Rays Define some heuristic for best estimate of X –Idea: Find midpoint of common perpendicular to the two rays But this is not invariant to projective transformations –Recall that without calibration the camera matrices are only known up to projection—i.e., PH, P’H are the “true” camera matrices for some non- singular H —so we will get different answers for X X from Hartley & Zisserman

7. Definition of Projectively Invariant Triangulation Suppose we compute a 3-D point X from the image points x, x’ and camera matrices P, P’ by some triangulation method ¿ We say that ¿ is projective-invariant if for any projective transformation H : X= ¿ (x, x’, P, P’) = H -1 ¿ (x, x’, PH -1, P’H -1 )

8. Optimal Projective-Invariant Triangulation: Reprojection Error Pick that exactly satisfies camera geometry so that and, and which minimizes Can use as error function for nonlinear minimization on two views –Polynomial solution exists from Hartley & Zisserman

9. DLT Triangulation There is a Direct Linear Transformation method for triangulation (see Hartley & Zisserman Chapter 11.2) –Not projectively invariant –Easily extends to > 2 views (whereas nonlinear method does not)

10. Covariance of Structure Recovery Bigger angle between rays ) Less uncertainty Can’t triangulate points on baseline (epipoles) because rays intersect along entire length from Hartley & Zisserman

11. Projective Reconstruction Theorem With uncalibrated cameras alone, we can reconstruct a scene (e.g., via triangulation) up to a projective ambiguity Calibrated cameras give metric reconstruction

12. Example: Projective Reconstruction Ambiguity from Hartley & Zisserman Reconstructions related by a 4 x 4 projection H Two views from which F and hence P, P’ are computed

13. Hierarchy of Transformations Less ambiguity Properties of transformations (2-D) from Hartley & Zisserman

14. Stratified Reconstruction Idea: Try to upgrade reconstruction to differ from the truth by a less ambiguous transformation Use additional constraints imposed by: –Scene –Motion –Camera calibration “Cheats” –Again: Cameras with known K, K’ ! Metric reconstruction –¸ 5 known 3-D points (no 4 coplanar) ! Euclidean reconstruction from Hartley & Zisserman

15. Projective ! Affine Upgrade Identify plane at infinity ¼ 1 ( in the “true” coord- inate frame, ¼ 1 = (0, 0, 0, 1) T ) –E.g., intersection points of three sets of parallel lines define a plane –E.g., if one camera is known to be affine from Hartley & Zisserman

16. Projective ! Affine Upgrade Then apply 4 x 4 transformation: This is the 3-D analog of affine image rectification via the line at infinity l 1 Things that can be computed/constructed with only affine ambiguity: –Midpoint of two points –Centroid of group of points –Lines parallel to other lines, planes

17. Example: Affine Reconstruction Ambiguity Affine reconstructions from Hartley & Zisserman

18. Affine ! Metric Upgrade Identify absolute conic ­ 1 on ¼ 1 via image of absolute conic (IAC) ! –From scene E.g., orthogonal lines –From known camera calibration Completely constrained: ! = K -T K -1 Partially constrained: –Zero skew –Square pixels –Same camera took all images ­ 1 and ! are beyond scope of this class—they won’t be on the final

19. Example: Metric Reconstruction with Texture Mapping Only overall scale ambiguity remains—i.e., what are units of length? from Hartley & Zisserman Original viewsSynthesized views of reconstruction