1. EECS 274 Computer Vision Geometric Camera Calibration

2. Geometric camera calibration Camera calibration problem Least-squares techniques Linear calibration from points Analytical photogrammetry Reading: Chapter 1 and 22 of FP, Chapters 2, 6 of S

3. Calibration Determine the intrinsic and extrinsic parameters Assume that the camera observes a set of features (points, or lines) with known positions Calibration: modeled as an optimization to minimize the discrepancy between the observed image features and their theoretical projections (using the perspective projection equations)

4. Given n points, P 1, …, P n with known positions and their images points, p 1, …, p n, find ξ Calibration problem

5. A A x xb b = = Square system: unique solution Gaussian elimination Rectangular system ?? underconstrained: infinity of solutions Minimize ||Ax-b|| 2 overconstrained: no solution Linear systems

6. Overconstrained problems

7. In matrix form Can be derived from the perspective of projection matrix

8. A A x x0 0 = = Square system: unique solution: 0 unless Det(A)=0 Rectangular system ?? 0 is always a solution Minimize |Ax| under the constraint |x| =1 2 2 Homogenous linear systems

9. The solution is e. 1 Overconstrained homogenous linear systems

10. Problem: minimize with respect to (a,b,d). Minimize E with respect to d: Minimize E with respect to a,b: where Solution is the unit eigenvector with minimum eigenvalue Example: linear fitting

11. Matrix of second moments of inertia Axis of least inertia in mechanics Note

12. Linear camera calibration

13. Once M is known, need to recover the intrinsic and extrinsic parameters This is a decomposition problem, not an estimation problem Intrinsic parameters Extrinsic parameters  When M is known ρ: scale factor

14. Decomposition of M As the recovered Orthonormal basis vector θ is close to π/2 and has positive sine

15. Are there other solutions besides M ? One solution: ( )=(m 1, m 2, m 3 ) Consider the points P i all lie in some plane, s.t.,  ∙ P i =0 for some  Coplanar points: choose ( )=(  ) or (  ) or (  ), or any linear combination of these vectors yields a solution Does not (usually) happen for 6 or more random points! Degenerate point configuration

16. Radial distortion Depends on the distance between the image center and an image point, d Corners are detected by fitting lines in each square Barrel distortionUsing estimated distortion parameters

17. Correct radial distortion Tsai’s algorithm (1987) exploits radial alignment constraints for estimating extrinsic parameters 11+q parameters

18. Non-Linear Least-Squares Methods Newton Gauss-Newton Levenberg-Marquardt Iterative, quadratically convergent in favorable situations Given n points, P 1, …, P n with known positions and their images situations, p 1, …, p n, find ξ Analytic photogrammetry

19. Mobile Robot Localization (Devy et al., 1997) Application

20. Calibration Numerous ways that exploits properties of projective geometry –e.g. calibration using lines, calibration circular controlled points

21. Camera calibration toolbox Excellent MATLAB toolbox by Jean-Yves Bouguet http://www.vision.caltech.edu/bouguetj/calib_doc/ Steps: –Generate calibration board –Collect images under different views –Select extreme points –Find corner points –Solve optimization problem

22. Calibration images

23. Extreme points

24. Guessed grid corners

25. Corner extraction

26. Repeat for all other images

27. Solving optimization problem

28. Reprojected corners

29. Camera centered view

30. World centered view

31. Applications Augmented reality Image registration Image stitching Panoramic image

33. Notes Camera pose estimation Multi-camera calibration Auto/self calibration Multi-camera self calibration Projective geometry Multi-view geometry RANSAC (RANdom Sample Consensus)