1. Robot Vision SS 2005 Matthias Rüther 1 ROBOT VISION Lesson 3: Projective Geometry Matthias Rüther Slides courtesy of Marc Pollefeys Department of Computer Science University of North Carolina, Chapel Hill

2. Robot Vision SS 2005 Matthias Rüther 2 Content  Projective geometry in 3D –Points –Planes –Lines –Points from planes, planes from points –Hierarchy of transformations / invariants  Parameter Estimation –Problem: intersecting lines -> SVD –Problem: rectification –Normalization –Robust Estimation  Algorithm Evaluation –Error measures

3. Robot Vision SS 2005 Matthias Rüther 3 Projective 3D Geometry  Points, lines, planes and quadrics  Transformations

4. Robot Vision SS 2005 Matthias Rüther 4 3D points in R 3 in P 3 (4x4-1=15 dof) projective transformation 3D point

5. Robot Vision SS 2005 Matthias Rüther 5 Planes Dual: points ↔ planes, lines ↔ lines 3D plane Euclidean representation Transformation

6. Robot Vision SS 2005 Matthias Rüther 6 Planes from points (solve as right nullspace of ) Or implicitly from coplanarity condition

7. Robot Vision SS 2005 Matthias Rüther 7 Points from planes (solve as right nullspace of ) Representing a plane by its span

8. Robot Vision SS 2005 Matthias Rüther 8 Lines Example: X -axis (4dof)

9. Robot Vision SS 2005 Matthias Rüther 9 Points, lines and planes

10. Robot Vision SS 2005 Matthias Rüther 10 Plücker matrices Plücker matrix (4x4 skew-symmetric homogeneous matrix) 1.L has rank 2 2.4dof 3.generalization of 4. L independent of choice A and B 5.Transformation Example: x -axis

11. Robot Vision SS 2005 Matthias Rüther 11 Plücker matrices Dual Plücker matrix L * Correspondence Join and incidence (plane through point and line) (point on line) (intersection point of plane and line) (line in plane) (coplanar lines)

12. Robot Vision SS 2005 Matthias Rüther 12 Plücker line coordinates on Klein quadric

13. Robot Vision SS 2005 Matthias Rüther 13 Plücker line coordinates (Plücker internal constraint) (two lines intersect)

14. Robot Vision SS 2005 Matthias Rüther 14 Estimation  2D Problem: Lines l 1, l 2, l 3 intersect in a point l1l1 l2l2 l3l3 ?? Point of intersection x x How many solutions exist? When does a solution exist? Geometric interpretation?  Solve equation system:

15. Robot Vision SS 2005 Matthias Rüther 15 Estimation  Practical 2D Problem: measured lines l 1, l 2, l 3, … l m approximately intersect in a point x l1l1 l2l2 l3l3 ?? Point of intersection x x Find a solution that minimizes |Lx|. lmlm  Find an approximate solution to the system:

16. Robot Vision SS 2005 Matthias Rüther 16 Singular Value Decomposition (SVD)  SVD: algorithm that decomposes A mxn (m  n) to  Properties: –U has orthogonal columns: U T U = I –U is norm-preserving: |Ux| = |x| –D is diagonal, diagonal elements are the singular values –V is orthogonal: V T V = I

17. Robot Vision SS 2005 Matthias Rüther 17 Singular Value Decomposition (SVD)  Solve Ax=0  minimize |Ax| subject to |x|=1:  minimize |DV T x| subject to |V T x|=1 1 is in the same row as the smallest singular value in D. X is the column of V corresponding to the smallest Singular Value.

18. Robot Vision SS 2005 Matthias Rüther 18 Robust Estimation  Practical 2D Problem: SVD obtains an optimal least squares solution: l1l1 l2l2 l3l3 x  Problem with outliers: l1l1 l2l2 l3l3 x l4l4

19. Robot Vision SS 2005 Matthias Rüther 19 RANdom SAmple Consensus  RANSAC algorithm: given a set of lines l 1,…,l m, find an intersection point x such that |l k x| is minimal for k=1..m, subject to the condition that none of the valid lines deviates from x by more than t. l1l1 l2l2 l3l3 x l4l4 t lmlm

20. Robot Vision SS 2005 Matthias Rüther 20 RANSAC Objective Robust fit of model to data set S which contains outliers Algorithm (i)Randomly select a sample of s data points from S and instantiate the model from this subset. (ii)Determine the set of data points S i which are within a distance threshold t of the model. The set S i is the consensus set of samples and defines the inliers of S. (iii)If the subset of S i is greater than some threshold T, re- estimate the model using all the points in S i and terminate (iv)If the size of S i is less than T, select a new subset and repeat the above. (v)After N trials the largest consensus set S i is selected, and the model is re-estimated using all the points in the subset S i

21. Robot Vision SS 2005 Matthias Rüther 21 Parameter estimation  2D homography Given a set of (x i,x i ’), compute H (x i ’=Hx i )  3D to 2D camera projection Given a set of (X i,x i ), compute P (x i =PX i )  Fundamental matrix Given a set of (x i,x i ’), compute F (x i ’ T Fx i =0 x1x1 x2x2 x3x3 x4x4 x1‘x1‘ x2‘x2‘ x3‘x3‘ x4‘x4‘ ?H?

22. Robot Vision SS 2005 Matthias Rüther 22 Number of measurements required  At least as many independent equations as degrees of freedom required  Example: 2 independent equations / point 8 degrees of freedom 4x2≥8

23. Robot Vision SS 2005 Matthias Rüther 23 Approximate solutions  Minimal solution 4 points yield an exact solution for H  More points –No exact solution, because measurements are inexact (“noise”) –Search for “best” according to some cost function –Algebraic or geometric/statistical cost

24. Robot Vision SS 2005 Matthias Rüther 24 Gold Standard algorithm  Cost function that is optimal for some assumptions  Computational algorithm that minimizes it is called “Gold Standard” algorithm  Other algorithms can then be compared to it

25. Robot Vision SS 2005 Matthias Rüther 25 Direct Linear Transformation (DLT)

26. Robot Vision SS 2005 Matthias Rüther 26 Direct Linear Transformation (DLT)  Equations are linear in h  Only 2 out of 3 are linearly independent (indeed, 2 eq/pt) (only drop third row if w i ’≠0)  Holds for any homogeneous representation, e.g. ( x i ’, y i ’,1)

27. Robot Vision SS 2005 Matthias Rüther 27 Direct Linear Transformation (DLT)  Solving for H size A is 8x9 or 12x9, but rank 8 Trivial solution is h =0 9 T is not interesting 1-D null-space yields solution of interest pick for example the one with

28. Robot Vision SS 2005 Matthias Rüther 28 Direct Linear Transformation (DLT)  Over-determined solution No exact solution because of inexact measurement i.e. “noise” Find approximate solution - Additional constraint needed to avoid 0, e.g. - not possible, so minimize

29. Robot Vision SS 2005 Matthias Rüther 29 DLT algorithm Objective Given n≥4 2D to 2D point correspondences {x i ↔x i ’}, determine the 2D homography matrix H such that x i ’=Hx i Algorithm (i)For each correspondence x i ↔x i ’ compute A i. Usually only two first rows needed. (ii)Assemble n 2x9 matrices A i into a single 2 n x9 matrix A (iii)Obtain SVD of A. Solution for h is last column of V (iv)Determine H from h

30. Robot Vision SS 2005 Matthias Rüther 30 Inhomogeneous solution Since h can only be computed up to scale, pick h j =1, e.g. h 9 =1, and solve for 8-vector Solve using Gaussian elimination (4 points) or using linear least-squares (more than 4 points) However, if h 9 =0 this approach fails also poor results if h 9 close to zero Therefore, not recommended Note h 9 =H 33 =0 if origin is mapped to infinity