1. CS 395/495-26: Spring 2003 IBMR: Week 3 A 2D Projective Geometry: Conics Measure Angles Jack Tumblin jet@cs.northwestern.edu

2. x’ Apply the 3x3 matrix H x’ = Hx Apply the 3x3 matrix H x’ = Hx Recall: Projective Transform H x3x3x3x3 x2x2x2x2 x1x1x1x1 x y x’ y’ (x,y) x (x’,y’) Homog. coords x’ = [x’,y’,1] T  2D image (x’,y’) 2D image (x,y)  Homog. coords [x,y,1] T = x

3. x’ Apply the 3x3 matrix H x’ = Hx Apply the 3x3 matrix H x’ = Hx Recall: Projective Transform H x3x3x3x3 x2x2x2x2 x1x1x1x1 x y x’ y’ (x,y) x (x’,y’) Homog. coords x’ = [x’,y’,1] T  2D image (x’,y’) 2D image (x,y)  Homog. coords [x,y,1] T = x Goal: Find H for ‘Image Rectification’ ‘World Plane’ ‘View Plane’

4. Recall: The bits and pieces of H Recall: The bits and pieces of H H has 8 independent variables (DOF) Computer Vision Jargon (2D projective):Computer Vision Jargon (2D projective): Isometry --3DOF(2D translate t x,t y ; 2D rotate  z ; ) Similarity --4DOF (add uniform scale s;) Affine --6DOF (add orientable scale s ,/s, s  /s) Projective--8DOF (changes x 3 ; 3D-rotation-like) Rectification “up to a Similarity” removeRectification “up to a Similarity” remove

5. Recall: Rectification Undo parts of H x’ = H xwhere H = H S H A H P GOAL: Put world plane x’ into view plane x –Affine Rect.; (find only H P (2DOF)); –Metric Rect.;(find H A and H P (6DOF)); –Full Rect.;(find all: H S H A H P (8DOF)); METHODS: 1.Affine: ‘Vanishing Point’, Horizon line methods 2.Metric: Conics & Circular Points 3.Full: 4-point correspondence

6. Intuition for Conics: 2D Ellipses Cartesian Circle/Ellipse: a x 2 + b y 2 – r 2 = 0Cartesian Circle/Ellipse: a x 2 + b y 2 – r 2 = 0 P 2 Circle/Ellipse: a x 1 2 + b x 2 2 – r 2 x 3 2 = 0P 2 Circle/Ellipse: a x 1 2 + b x 2 2 – r 2 x 3 2 = 0 5 parameters (deg. of freedom):5 parameters (deg. of freedom): –center point (cx, cy) –major, minor axes (1/a, 1/b) –orientation  1/a 1/b 

7. Intuition for Conics: 2D Ellipses Still a circle/ellipse after P 2 projective transform H1H1H1H1 H2H2H2H2 H3H3H3H3

8. Conics: Key Ideas Conic method: Clever math, but NOT intuitive! (and book gives ugly, scattered description) –P 2 Conics are the family of ALL quadratics of (x 1, x 2, x 3 ) –Includes simpler ‘degenerate’ forms: point, line, line-pair, parabolas, hyperbolae. –Closed: ANY Conic transformed by H == Another Conic (imaginary)

9. Conics: Key Ideas –Transformed conics tell you how H changes angles –one strange, degenerate conic C*  measures angle –Angles between line pairs yield transformed C*  –We can compute H A and H P from transformed C*  2D world space 2D image space H S H A H P C*  = C*  ’ = 1 0 0 0 1 0 0 0 0    L M m L     

10. Conic Review Review conics first (pg.8) –‘Conics’ == intersection of cone & plane: –Many possible shapes: circles, ellipses, parabola, hyperbola, degenerate lines & points Dual Forms: –‘Point Conics’ defined by points on the curve –‘Line Conics’ defined by lines tangent to curve

11. Point Conics: C –All points on any conic solve a 2D quadratic: ax 2 + bxy + cy 2 +dx +ey +f = 0 –In homogeneous coordinates: (mpy. all by x 3 2 ) ax 1 2 + bx 1 x 2 + cx 2 2 +dx 1 x 3 +ex 2 x 3 + fx 3 2 = 0 ax 1 2 + bx 1 x 2 + cx 2 2 +dx 1 x 3 +ex 2 x 3 + fx 3 2 = 0 x T Cx = 0 x T Cx = 0 –C is symmetric, 5DOF ( not 6; ignores x 3 scaling) –Can find any C from 5 homog. point pairs... (SVD solves for null space—see book pg 9) a b/2 d/2 a b/2 d/2 b/2 c e/2 d/2 e/2 f x1x1x2x2x3x3x1x1x2x2x3x3 = 0 x 1 x 2 x 3 (but not very useful for finding H...)

12. Familiar Conics: Circles, Ellipses... x T C x = 0 or General form of point conic:General form of point conic: ax 2 + bxy + cy 2 +dx +ey +f = 0 (or) ax 1 2 + bx 1 x 2 + cx 2 2 +dx 1 x 3 +ex 2 x 3 + fx 3 2 = 0 Circle at d with radius r:Circle at d with radius r: 1x 1 2 + bx 1 x 2 + 1x 2 2 +dx 1 x 3 +ex 2 x 3 –r 2 x 3 2 = 0; C r = ‘Degenerate’ Cases (rank<3): lines, parabolas, hyperbolae... Example:‘Degenerate’ Cases (rank<3): lines, parabolas, hyperbolae... Example: 1x 1 2 + bx 1 x 2 + 0x 2 2 +dx 1 +ex 2 + fx 3 2 = 0; a b/2 d/2 a b/2 d/2 b/2 c e/2 d/2 e/2 f x1x1x2x2x3x3x1x1x2x2x3x3 = 0 x 1 x 2 x 3 1 0 0 0 1 0 0 0 -r 2 a 0 0 0 0 0 0 0 f

13. Matrix C makes conic curves from points x : x T Cx = 0C is a ‘point conic’Matrix C makes conic curves from points x : x T Cx = 0C is a ‘point conic’ The tangent line L for point x is: L = C xThe tangent line L for point x is: L = C x P2 conic transformed by H:P2 conic transformed by H: C’ = H -T C H -1 (messy! requires invertible H !) Point Conics: C L x

14. Matrix C* makes conic curves from lines L : L T C* L = 0C is a ‘line conic’Matrix C* makes conic curves from lines L : L T C* L = 0C is a ‘line conic’ The tangent point x for line L is: x = C* LThe tangent point x for line L is: x = C* L Projective transform by H is:Projective transform by H is: C*’ = H C* H T (better -- no H inversion required) If C is non-singular (rank 3), then C* = C -1If C is non-singular (rank 3), then C* = C -1 ‘Dual’ or Line Conics: C L x

15. Degenerate Conics x T C x = 0 or General form of point conic:General form of point conic: ax 2 + bxy + cy 2 +dx +ey +f = 0 (or) ax 1 2 + bx 1 x 2 + cx 2 2 +dx 1 x 3 +ex 2 x 3 + fx 3 2 = 0 Very special ‘degenerate’ case: ‘infinite radius circle’ as x 3  0, aka C  Very special ‘degenerate’ case: ‘infinite radius circle’ as x 3  0, aka C  1x 1 2 + bx 1 x 2 + 1x 2 2 +dx 1 x 3 +ex 2 x 3 +0x 3 2 = 0 C  = only solutions to x T C  x =0 are 2 complex x vectors: and a b/2 d/2 a b/2 d/2 b/2 c e/2 d/2 e/2 f x1x1x2x2x3x3x1x1x2x2x3x3 = 0 x 1 x 2 x 3 1 0 0 0 1 0 0 0 0 1 i 0 1 -i -i 0

16. Conic Weirdness I ‘Circular Points’ I=, J= ; or C  = –‘Ideal points’; 2 points at infinity / horizon, but… –Complex : real part on x axis, imag. on y –“Where infinite circle hits infinity line L  [0,0,1]” –Where? imaginary! (infinite x,y axes, maybe?) –I,J are not affected by H S. ( only by H A, H P ) ?WHY BOTHER? –To measure angles in projective space –To use angle correspondences to find H A,H P 1 i 0 1 -i -i 0 1 0 0 0 1 0 0 0 0

17. Conic Weirdness 2 ‘Circular Points’ I=, J= ; and C  = C  is a point conic, affected by H A H P only.C  is a point conic, affected by H A H P only. C*  is line conic, but the same matrix (!?!?!)C*  is line conic, but the same matrix (!?!?!) –Is same as this outer product: IJ T + JI T –Includes L , the infinity line (L  C*  L  =0) –Affected by H P and H A only, ( H S does nothing) ALL transforms of C*  have only 4DOF (pg 33 ) 1 i 0 1 -i -i 0 1 0 0 0 1 0 0 0 0

18. Conics: Angle Measuring Matrix H transforms C*  to another space:Matrix H transforms C*  to another space: C*  ’ = H C*  H T Angles: –define ‘world space’ C*  as –(for any two world-space homogeneous lines) L and m are perpendicular iff L T C*  m = 0 –Also true for transformed L, m and C* , so… –C*  ’ in image space  angle in world space –C*  ’ in image space  angle  in world space 1 0 0 0 1 0 0 0 0 (L T C*  ’ m) (L T C*  ’ L)(m T C*  ’ m) cos(  ) = (simpler to transform a line conic)

19. Undoing H: Metric Rectification C*  ’ has only 4DOF (before,after ANY H)C*  ’ has only 4DOF (before,after ANY H) All C*  ’ DOF are contained in (H A H P )All C*  ’ DOF are contained in (H A H P ) SVD can convert C*  ’ to (H A H P ) matrix!SVD can convert C*  ’ to (H A H P ) matrix! Task is then “find C*  ’, then use it to find H”Task is then “find C*  ’, then use it to find H” 2D world space 2D image space H S H A H P C*  = C*  ’ =    L M m L    0 1 0 0 0 1 0 0 0 0

20. ? But how do we find C*  ’ ? One Answer: use perpendicular lines –(Recall) we defined ‘world space’ C*  as –Then: 2 world-space lines L and m are perpendicular iff L T C*  m = 0 –Also true for transformed L, m and C*  –H transforms C*  to image space C*  ’ by: C*  ’ = H C*  H T –Now split up H and try it: H = H S H A H P, so… 1 0 0 0 1 0 0 0 0

21. Undoing H: Metric Rect. 1 H = H S H A H P =H = H S H A H P = Book shows how to write transformed C*  as (simple, tedious):Book shows how to write transformed C*  as (simple, tedious): C*  ’ = C*  ’ = where K is 2x2 symmetric (affine part: 2DOF) v is 2x1 vector, (projective part: 2DOF) v is 2x1 vector, (projective part: 2DOF) sRtsRt0T10T1sRtsRt0T10T1 I0I0vTvvTvI0I0vTvvTv K0K00T10T1K0K00T10T1              KK T  Kv       v T K  0

22. Undoing H: Metric Rect. 1 H = H S H A H P =H = H S H A H P = Book shows how to write transformed C*  as (simple, tedious):Book shows how to write transformed C*  as (simple, tedious): C*  ’ = C*  ’ = where K is 2x2 symmetric (affine part: 2DOF) v is 2x1 vector, (projective part: 2DOF) v is 2x1 vector, (projective part: 2DOF) sRtsRt0T10T1sRtsRt0T10T1 I0I0vTvvTvI0I0vTvvTv K0K00T10T1K0K00T10T1              KK T  Kv       v T K  0 !!!BUT DERIVATION & RESULT IS WRONG!!! SKIP ‘Metric Rectification I’ pg 35,36!

23. END

24. Undoing H: Metric Rect. 1 C*  ’ =  C*  ’ =  First, ignore projective part: set v=0.First, ignore projective part: set v=0. Choose two pairs of perpendicular lines L,mChoose two pairs of perpendicular lines L,m Second, Use L T ’ C*  ’ m’ = 0 to solve for KSecond, Use L T ’ C*  ’ m’ = 0 to solve for K KK T  Kv       v T K  s 2D world space 2D image space H S H A H P L M m L KK T  0       0  s 0  s aka ‘Weak Perspective’

25. Matrix Math & SVDs Matrix multiply is a change of coordinatesMatrix multiply is a change of coordinates –Rows of A = coordinate axis vectors –Ax = a dot product for each axis input space  output space input space  output space SVD ‘factors’ matrix A into 3 parts:SVD ‘factors’ matrix A into 3 parts: –U = orthonormal basis svd(A) = U S V T

26. Matrix Math & SVDs Matrix multiply is a change of coordinatesMatrix multiply is a change of coordinates –Rows of A = coordinate axis vectors –Ax = a dot product for each axis input space  output space input space  output space SVD ‘factors’ matrix A into 3 parts:SVD ‘factors’ matrix A into 3 parts: –U = orthonormal basis svd(A) = U S V

27. Recall: Projective Transform: Hx = x’ x1x1x2x2x3x3x1x1x2x2x3x3 h 11 h 12 h 13 h 21 h 22 h 21 h 31 h 32 h 33 = x’ 1 x’ 2 x’ 3 Finding H from point pairs (correspondences) x’ = = y’ = = = xy1 Input (or output) image is on central plane x1x1x2x2x3x3x1x1x2x2x3x3 h 21 x + h 22 y + h 21 h 31 x + h 32 y + h 33 x’ 1 x’ 3 x’ 2 x’ 3 Write R 2 expressions; (for each x,x’ pair) h 11 x + h 12 y + h 13 h 31 x + h 32 y + h 33 x’ (h 31 x + h 32 y + h 33 ) = (h 11 x + h 12 y + h 13 ) y’ (h 31 x + h 32 y + h 33 ) = (h 11 x + h 12 y + h 13 ) Rearrange, solve as a matrix problem…

28. DLT: Four-Point Correspondence 4 point correspondence:4 point correspondence: –Book shows: –Rearrange: known vector (dot) unknown vector –stack, solve for null space… Hint FilesHint Files –Added ‘max’ commands, examples –Make, test your own H matrices! x’ (h 31 x + h 32 y + h 33 ) = (h 11 x + h 12 y + h 13 ) y’ (h 31 x + h 32 y + h 33 ) = (h 11 x + h 12 y + h 13 ) h 11 h 12 h 13 h 21 h 22 h 23 h 31 h 32 h 33 [x y 1 0 0 0 -x’x -x’y -x’] = 0 [0 0 0 x y 1 -y’x -y’y -y’]

29. Direct Linear Tranform (DLT) Project 2 method: Solve Hx – x’ = 0Project 2 method: Solve Hx – x’ = 0 –Requires constant scale (x 3 =1 in measurement) compare:compare: DLT method: Solve Hx  x’ = 0DLT method: Solve Hx  x’ = 0 –Accepts any scale (x 3 =w  1 OK, xx 3 ) [ 0 0 0 x y 1 -y’x -y’y -y’ ] [ x y 1 0 0 0 -x’x -x’y -x’ ] h 11 h 12 h 13 h 21 h 22 h 23 h 31 h 32 h 33 [ 0 0 0 -w’x -w’y -w’w y’x y’y y’w] = 0 [w’x w’y w’w 0 0 0 -x’x -x’y -x’w]

30. Direct Linear Tranform (DLT) Project 2 method: Solve Hx – x’ = 0Project 2 method: Solve Hx – x’ = 0 –Requires constant scale (x 3 =1 always) compare:compare: DLT method: Solve Hx  x’ = 0DLT method: Solve Hx  x’ = 0 –Accepts any scale (x 3 =w  1, x 3 ’=w’  1) [ 0 0 0 x y 1 -y’x -y’y -y’ ] [ x y 1 0 0 0 -x’x -x’y -x’ ] h1 h1 h2 h2 h3 h3 h1 h1 h2 h2 h3 h3 [ 0 0 0 -w’x -w’y -w’w y’x y’y y’w] = 0 [w’x w’y w’w 0 0 0 -x’x -x’y -x’w] [ 0 T -w’ i x i T y’ I x i T ] [ w’ i x i T 0 T -x’ i x i T ]

31. Direct Linear Tranform (DLT) DLT method: Solve H  x = 0DLT method: Solve H  x = 0 –Accepts any scale, any point (x 3 =w  1 is OK) –‘Pure’, Compatible -- P 2 terms only –Much better suited to error measurements. Subtlety:Subtlety: –Won’t divide-by-zero if w=0 or h 33 =0 (it happens!) –has a 3 rd row; it is not degenerate if w’=0 –OK to use it… (Solve 8x12) h1 h1 h2 h2 h3 h3 h1 h1 h2 h2 h3 h3 0 T -w’ i x i T y’ I x i T 0 T -w’ i x i T y’ I x i T w’ i x i T 0 T -x’ i x i T =0 w’ i x i T 0 T -x’ i x i T =0 -y’ i x i T -x’ i x i T 0 T -y’ i x i T -x’ i x i T 0 T

32. Deceptive ‘Robustness’ Suppose we have 4 pt-correspondences:Suppose we have 4 pt-correspondences: –Use DLT to write 8x9 (or 12x9) matrix A: Ah=0 –Solve for h null space. ALWAYS gives H matrix But what if points are bad / fictional?But what if points are bad / fictional? –3 collinear input pts, crooked out: IMPOSSIBLE! –Yet we get an H solution! Why? A matrix rank is 7 or 6  rank 1 H result(s)A matrix rank is 7 or 6  rank 1 H result(s) –‘Null Space’ of A may contain >1 H solution! –‘Degenerate’ H solution of form aH 1 + bH 2 … Answer: SVD ranks A; reject bad point sets.Answer: SVD ranks A; reject bad point sets.

33. Actual ‘Robustness’ Vectorizing (‘Flatten, Stack, Null Space’ method) works for almost ANY input! (Points, lines, planes, ||, , conics, quadrics, cross-ratios, vanishing points, twisted cubics…Vectorizing (‘Flatten, Stack, Null Space’ method) works for almost ANY input! (Points, lines, planes, ||, , conics, quadrics, cross-ratios, vanishing points, twisted cubics… Use DLT formulation: (Hx  x’) = 0Use DLT formulation: (Hx  x’) = 0 Rearrange as dot product: (known)·(unknown) = 0Rearrange as dot product: (known)·(unknown) = 0 Be careful to have ENOUGH constraintsBe careful to have ENOUGH constraints tricky when you mix types: points, lines,… (pg 75) tricky when you mix types: points, lines,… (pg 75) measured input transformed  measured output measured output = 0