1. Multi-linear Systems and Invariant Theory in the Context of Computer Vision and Graphics CS329 Amnon Shashua

2. Material We Will Cover Today The structure of 3D->2D projection matrix The homography matrix A primer on projective geometry of the plane

3. The Structure of a Projection Matrix

6. Generally, is called the “ principle point ” is called the “ skew ” is “ aspect ratio ”

7. The Camera Center has rank=3, thus such that Why isthe camera center?

8. The Camera Center Why isthe camera center? Consider the “ optical ray ” All points along the line are mapped to the same point is a ray through the camera center

9. The Epipolar Points

10. Choice of Canonical Frame is the new world coordinate frame Choose W such that We have 15 degrees of freedom (16 upto scale)

11. Choice of Canonical Frame Let We are left with 4 degrees of freedom (upto scale):

12. Choice of Canonical Frame

13. whereare free variables

14. Projection Matrices Let be the image of pointat frame number j whereare free variables are known

15. Family of Homography Matrices Stands for the family of 2D projective transformations between two fixed images induced by a plane in space The remainder of this class is about making the above statement intelligible

16. Family of Homography Matrices Recall, 3D->2D from Euclidean world frame to image Let K,K ’ be the internal parameters of camera 1,2 and choose canonical frame in which R=I and T=0 for first camera. world frame to first camera frame

17. Family of Homography Matrices Recall that 3 rd row of K is

18. Family of Homography Matrices Assumeare on a planar surface

19. Family of Homography Matrices Let and Image-to-image mapping where the matching pair are induced by a planar surface.

20. Family of Homography Matrices first camera frame when is the homography matrix induced by the plane at infinity

21. Family of Homography Matrices is the epipole in the second image

22. Relationship Between two Homography Matrices is the projection of onto the first image

23. p p’  Estimating the Homography Matrix How many matching points? 4 points make a basis for the projective plane

24. Projective Geometry of the Plane Equation of a line in the 2D plane The line is represented by the vector and Correspondence between lines and vectors are not 1-1 because represents the same line Two vectors differing by a scale factor are equivalent. This equivalence class is called homogenous vector. Any vector is a representation of the equivalence class. The vectordoes represent any line.

25. Projective Geometry of the Plane A pointlies on the line (coincident with) which is represented byiff But also represents the point The vectordoes represent any point. Points and lines are dual to each other (only in the 2D case!).

26. Projective Geometry of the Plane note:

27. Lines and Points at Infinity Consider lines which represents the point with infinitely large coordinates All meet at the same point

28. Lines and Points at Infinity The points lie on a line The line is called the line at infinity The pointsare called ideal points. A linemeetsat (which is the direction of the line)

29. A Model of the Projective Plane Points are represented as lines (rays) through the origin Lines are represented as planes through the origin ideal point is the plane

30. A Model of the Projective Plane ={lines through the origin in} ={1-dim subspaces of }

31. Projective Transformations in The study of properties of the projective plane that are invariant under a group of transformations. Projectivity: that maps lines to lines (i.e. preserves colinearity) Any invertible 3x3 matrix is a Projectivity: LetColinear points, i.e. the pointslie on the line is the dual. is called homography, colineation

32. Projective Transformations in perspectivity A composition of perspectivities from a planeto other planes and back tois a projectivity. Every projectivity can be represented in this way.

33. Projective Transformations in Example, a prespectivity in 1D: Lines adjoining matching points are concurrent Lines adjoining matching points (a,a ’ ),(b,b ’ ),(c,c ’ ) are not concurrent

34. Projective Transformations in is not invariant under H: Points onare is not necessarily 0 Parallel lines do not remain parallel ! is mapped to

35. Projective Basis A Simplex inis a set of n+2 points such that no subset Of n+1 of them lie on a hyperplane (linearly dependent). In a Simplex is 4 points Theorem: there is a unique colineation between any two Simplexes

36. Why do we need 4 points Invariants are measurements that remain fixed under colineations # of independent invariants = # d.o.f of configuration - # d.o.f of trans. Ex: 1D case H has 3 d.o.f A point in 1D is represented by 1 parameter. 4 points we have: 4-3=1 invariant (cross ratio) 2D case: H has 8 d.o.f, a point has 2 d.o.f thus 5 points induce 2 invariants

37. Why do we need 4 points The cross-ratio of 4 points: 24 permutations of the 4 points forming 6 groups:

38. Why do we need 4 points 5 points gives us 10 d.o.f, thus 10-8=2 invariants which represent 2D are the 4 basis points (simplex) are determined uniquely by Point of intersection is preserved under projectivity (exercise) uniquely determined