1. CS 395/495-26: Spring 2003 IBMR: Week 2 B 2-D Projective Geometry Jack Tumblin jet@cs.northwestern.edu

2. Admin; Starter Code Website Ready: Plans evolving...Website Ready: Plans evolving... Starter Code Ready: also evolves...Starter Code Ready: also evolves... About Your Projects:About Your Projects: –Simple directory-based version control: –Save & Number everything that works: NEVER edit an unsaved working program –Always, Always: One Little Step at a Time! –Explain it as you write it (comments) “Writing IS thinking”

3. Admin; Starter Code Website Ready: Plans evolving...Website Ready: Plans evolving... Starter Code Ready: also evolves...Starter Code Ready: also evolves... About Your Projects:About Your Projects: –Simple directory-based version control: –What works well for me: One master directory for the projectOne master directory for the project All files of latest version in one directoryAll files of latest version in one directory Numbered previous versions: ‘snapshot’ copiesNumbered previous versions: ‘snapshot’ copies Copy master directory EVERY time something works.Copy master directory EVERY time something works. Program is ALWAYS working; one step at a time...Program is ALWAYS working; one step at a time... > 1 hour without compile/run? too big a step!> 1 hour without compile/run? too big a step!

4. l 1  l 2 = x x 1  x 2 = l 2D Homogeneous Coordinates 3 coords, but only 2 degrees of freedom (2DOF) (x 1 / x 3 ), (x 2 / x 3 )3 coords, but only 2 degrees of freedom (2DOF) (x 1 / x 3 ), (x 2 / x 3 ) –Ideal points == (x,y,0) –Line Intersections = point: (a 3D cross-product) (a 3D cross-product) –Point ‘Intersections’ = line: –Duality: P 2 theorem for lines  P 2 theorem for points! An ‘Ideal Point’ (0,0,0) at Center of Projection ‘Central Image Plane’ (x,y,1) -x 3 x2x2x2x2 x1x1x1x1 Ray

5. l 1  l 2 = x x 1  x 2 = l 2D Homogeneous Coordinates 3 coords, but only 2 degrees of freedom (2DOF) (x 1 / x 3 ), (x 2 / x 3 )3 coords, but only 2 degrees of freedom (2DOF) (x 1 / x 3 ), (x 2 / x 3 ) –Ideal points == (x,y,0) –Line Intersections = point: (a 3D cross-product) (a 3D cross-product) –Point ‘Intersections’ = line: –Duality: P 2 theorem for lines  P 2 theorem for points! An ‘Ideal Point’ (0,0,0) at Center of Projection ‘Central Image Plane’ (x,y,1) -x 3 x2x2x2x2 x1x1x1x1 Ray Left-Handed: use –x 3 or (x,y,-1) image ctr, or...

6. Every xy point describes a 3D ray in (x 1,x 2,x 3 )Every xy point describes a 3D ray in (x 1,x 2,x 3 ) ‘Phantom’ dimension x 3 is the ‘z-buffer’ value‘Phantom’ dimension x 3 is the ‘z-buffer’ value Homogeneous coords allows translation matrix:Homogeneous coords allows translation matrix: (BUT a 3x3 transform in R 3 Euclidean Space (x,y,z) can’t translate—only rotate,scale, skew!)(BUT a 3x3 transform in R 3 Euclidean Space (x,y,z) can’t translate—only rotate,scale, skew!) Useful 3D Graphics Ideas An ‘Ideal Point’ (0,0,0) at Center of Projection x y = 1 1 0 3 0 1 5 0 0 1 x+3y+51 ‘Central Image Plane’ (x,y,1) -x 3 x2x2x2x2 x1x1x1x1 x y = z 1 0 3 0 1 5 0 0 1 x+zy+zz Ray

7. Projective Transform: H Matrix H transforms Points:Matrix H transforms Points: –Given a point x: (x 1,x 2,x 3 ) (or in P 2 : (x 1 /x 3, x 2 /x 3 ) ) x’ = H x And H -T can also transform Lines:And H -T can also transform Lines: –Recall point x is on line l iff x T l =0 = –Lines transform ‘covariantly’ (points: ‘contravariantly’: x’ = Hx) l’ = H -T l H* ‘Line Homographies’ of many kinds are often denoted by a star: H*

8. x’ Apply the 3x3 matrix H Hx = x’ Apply the 3x3 matrix H Hx = x’ Projective Transform H x3x3x3x3 x2x2x2x2 x2x2x2x2 x y x’ y’ (x,y) x (x’,y’) Homog. coords x’ = [x’,y’,1] T  2D image (x’,y’) 2D image (x,y)  Homog. coords [x,y,1] T = x

9. Projective Transform: Hx = x’ x1x1x2x2x3x3x1x1x2x2x3x3 h 11 h 12 h 13 h 21 h 22 h 21 h 31 h 32 h 33 = x’ 1 x’ 2 x’ 3 Finding H from point pairs (correspondences) x’ = = y’ = = = xy1 Input (or output) image is on central plane x1x1x2x2x3x3x1x1x2x2x3x3 h 21 x + h 22 y + h 21 h 31 x + h 32 y + h 33 x’ 1 x’ 3 x’ 2 x’ 3 Write R 2 expressions; (for each x,x’ pair) h 11 x + h 12 y + h 13 h 31 x + h 32 y + h 33 x’ (h 31 x + h 32 y + h 33 ) = (h 11 x + h 12 y + h 13 ) y’ (h 31 x + h 32 y + h 33 ) = (h 11 x + h 12 y + h 13 ) Rearrange, solve as a matrix problem…

10. Projective Transform: Hx = x’ x1x1x2x2x3x3x1x1x2x2x3x3 h 11 h 12 h 13 h 21 h 22 h 21 h 31 h 32 h 33 = x’ 1 x’ 2 x’ 3 Finding H from point pairs (correspondences) x’ = = y’ = = = xy1 Input (or output) image is on central plane x1x1x2x2x3x3x1x1x2x2x3x3 h 21 x + h 22 y + h 21 h 31 x + h 32 y + h 33 x’ 1 x’ 3 x’ 2 x’ 3 Write R 2 expressions; (for each x,x’ pair) h 11 x + h 12 y + h 13 h 31 x + h 32 y + h 33 x’ (h 31 x + h 32 y + h 33 ) = (h 11 x + h 12 y + h 13 ) y’ (h 31 x + h 32 y + h 33 ) = (h 11 x + h 12 y + h 13 ) Rearrange, solve as a matrix problem… —!STOP! — You get Easier, Robust, but Partial results using l  or conics methods! Let’s try them first. Later we’ll return to this method (called ‘DLT’: direct linear xfm.)

11. Projective Transform: H Comp. Graphics ‘View interpolation’ notionComp. Graphics ‘View interpolation’ notion –Fixed, rigid 2D viewing point, viewing plane –Rigid 2D world plane positioned in 3D ( x 1 x 2 x 3 ) –only 6 DOF: world plane rotate & position. Comp. Vision ‘Projective Transform’ notionComp. Vision ‘Projective Transform’ notion –Fixed, rigid 2D viewing point, viewing plane –‘Stretchy’ 2D world plane: allow affine changes –result: (up to) 8 DOF

12. The bits and pieces of H The bits and pieces of H H has 8 independent variables (DOF) Computer Graphics method (3x3 matrix):Computer Graphics method (3x3 matrix): 2D Translation(t x, t y ) 3D Scale (s x,s y, s z ) 3D Rotation (  x,  y,  z ) 2D Translation(t x, t y ) 3D Scale (s x,s y, s z ) 3D Rotation (  x,  y,  z ) Computer Vision method(2D projective):Computer Vision method(2D projective): Isometry --3DOF (2D translate t x,t y ; 2Drotate  z ; ) Similarity --4DOF (add uniform scale s;) Affine --6DOF (add orientable scale s ,/s, s  /s) Projective--8DOF (changes x 3 ; 3D-rotation-like)

13. The bits and pieces of H The bits and pieces of H H has 8 independent variables (DOF) Computer Graphics method (3x3 matrix):Computer Graphics method (3x3 matrix): 2D Translation(t x, t y ) 3D Scale (s x,s y, s z ) 3D Rotation (  x,  y,  z ) 2D Translation(t x, t y ) 3D Scale (s x,s y, s z ) 3D Rotation (  x,  y,  z ) Computer Vision method(2D projective):Computer Vision method(2D projective): Isometry --3DOF (2D translate t x,t y ; 2Drotate  z ; ) Similarity --4DOF (add uniform scale s;) Affine --6DOF (add orientable scale s ,/s, s  /s) Projective--8DOF (changes x 3 ; 3D-rotation-like) Affects only x1,x2

14. The bits and pieces of H The bits and pieces of H H has 8 independent variables (DOF), not 9! Decomposable into 3 useful parts: (pg 22)Decomposable into 3 useful parts: (pg 22) H = H S H A H P = H = H S H A H P = Similarity H S :Similarity H S : 2D translate, rotate, uniform scale only (4DOF)2D translate, rotate, uniform scale only (4DOF) Affine H A :Affine H A : non-uniform scale (2DOF)non-uniform scale (2DOF) Projective H P :Projective H P : Projective coupling for x 3 (2DOF)Projective coupling for x 3 (2DOF) sRtsRt0T10T1sRtsRt0T10T1 I0I0vTvvTvI0I0vTvvTv K0K00T10T1K0K00T10T1            

15. H has 8 independent variables (DOF), not 9! Decomposable into 3 useful parts: (pg 22)Decomposable into 3 useful parts: (pg 22) H = H S H A H P = H = H S H A H P = Similarity H S :Similarity H S : 2D translate, rotate, uniform scale only (4DOF)2D translate, rotate, uniform scale only (4DOF) Affine H A :Affine H A : non-uniform scale (2DOF)non-uniform scale (2DOF) Projective H P :Projective H P : Projective coupling for x 3 (2DOF)Projective coupling for x 3 (2DOF) The bits and pieces of H The bits and pieces of H s  cos  -s sin  t x s  sin  s cos  t y 0 0 1 0 0 1 1 00 0 1 0 v 1 v 2 v S 1 k20 k3 10 0 01

16. 1-D Projective Geometry (?Why? we use it later) A ‘side’ view of 2D projective geometryA ‘side’ view of 2D projective geometry Convert R 1 scalar b to a 2-vector in P 1Convert R 1 scalar b to a 2-vector in P 1 As with P 2, we can transform points:As with P 2, we can transform points: –( use various H 2x2 s to change cyan lines below) b1 H 2a H 2b

17. 1-D Projective Geometry What is invariant here? all the positions, lengths and ratios change with each transform… Answer: ‘Cross Ratio’: cross(a,b,c,d) let a=, b= …let a=, b= … THEN:THEN: cross = |ab||cd| |ac||bd| cross = |ab||cd| |ac||bd| abcd a1a1a2a2 a1a1a2a2 b1b1b2b2 b1b1b2b2 a’ c’ d’ b’

18. Image Rectifying: Undo parts of H x’ = H xwhere H = H S H A H P GOAL: Put world plane x’ into view plane –Affine Rect.; (find only H P (2DOF)); –Similarity Rect.;(find only H A H P (6DOF)); –Full Rect.;(find all of H (8DOF)); METHODS: (mix & match?) 1.Full: 4-point correspondence 2.‘Vanishing Point’, Infinity line methods 3.Conics and circular points

19. Image Rectifying: Undo parts of H x’ = H xwhere H = H S H A H P GOAL: Put world plane x’ into view plane –Affine Rect.; (find only H P (2DOF)); –Similarity Rect.;(find only H A H P (6DOF)); –Full Rect.;(find all of H (8DOF)); METHODS: (mix & match?) 1.Full: 4-point correspondence 2.‘Vanishing Point’, Infinity line methods 3.Conics and circular points Other DOF? make assumptions, or ignore

20. Vanishing-Point Methods: 1 –In 2D image, find two pairs of lines that are parallel in ‘world’ space. world space 2D image H

21. Vanishing-Point Methods: 1 –Find intersection (vanishing points) p1, p2 (compute line intersections with 3D cross-products (see last lecture) –Horizon line I h connects p1, p2. world space 2D image p1 p2 H IhIhIhIh

22. Vanishing-Point Methods: 1 –H -T transforms world infinity line l   (0,0,1) T to I h – Answer (see book) find, copy the l h coordinates! H p = world space 2D image p1 p2 H IhIhIhIh 1 0 0 0 1 0 l h1 l h2 l h3

23. Vanishing-Point Methods: 1 Limitations: –Accurate point, line-finding can be tricky –Can have high error sensitivity: ‘twitchy’ –Only rectifies H p -- but what if H A, H S needed? world space 2D image p1 p2 H IhIhIhIh

24. Vanishing-Point Methods: 2 No parallel lines? –find 2D image line with a known distance ratio –Use Cross-Ratio (in P 1 along that line) to –Find vanishing point distance at point d’ world space 2D image p1 p2 H IhIhIhIh abc a’ b’ c’ d’ d (at infinity)

25. END --Please download the starter code, compile and run. --Then you can get started on Project 1 right away.

26. Conic Methods Better-behaved, easier to use (? maybe...)Better-behaved, easier to use (? maybe...) Determines H A H P (4DOF) (all but 2D trans, rot, scale)Determines H A H P (4DOF) (all but 2D trans, rot, scale) Must go back and review conics first (pg.8) –‘Conics’ == intersection of cone & plane: –Many possible shapes: circles, ellipses, parabola, hyperbola, degenerates (lines & points

27. Conic Methods –Equation of any/all conics solve a 2D quadratic: ax 2 + bxy + cy 2 +dx +ey +f = 0 –Write in homogeneous coordinates: x T Cx = 0 x T Cx = 0 –C is symmetric, 5DOF ( not 6, because x 3 scaling) –Find any C from 5 homogeneous points (solve for null space—see book pg 9) a b/2 d/2 a b/2 d/2 b/2 c e/2 d/2 e/2 f x1x1x2x2x3x3x1x1x2x2x3x3 = 0 x 1 x 2 x 3 a ‘Point Conic’

28. Conic Methods –Matrix C makes conics from points: x T Cx = 0 C is a ‘point conic’ –Given a point x on a conic curve, the homog. tangent line l is given by l = C x –Matrix C* makes conics from lines: l T C*l = 0C* is a ‘Dual Conic’ defined by tangent lines l instead of points.

29. Conic Methods –If C is non-singular (rank 3), then C* = C -1 –If C (or C* ) has… Rank 3: it is an ellipse, circle, parab., hyperb. Rank 2: it is a pair of lines (forms an ‘x’) THE BIG IDEA---MOST USEFUL PROPERTY Projective transform of a conic C is conic C’: C’ = H -T C H -1 C’ = H -T C H -1