1. Sebastian Thrun CS223B Computer Vision, Winter 2005 1 Stanford CS223B Computer Vision, Winter 2006 Lecture 4 Camera Calibration Professor Sebastian Thrun CAs: Dan Maynes-Aminzade and Mitul Saha [with slides by D Forsyth, D. Lowe, M. Polleyfeys, C. Rasmussen, G. Loy, D. Jacobs, J. Rehg, A, Hanson, G. Bradski,…]

2. Sebastian Thrun CS223B Computer Vision, Winter 2005 2 Today’s Goals Calibration: Problem definition Solution via Singular Value Decomposition Solution by nonlinear Least Squares Distortion

3. Sebastian Thrun CS223B Computer Vision, Winter 2005 3 Intrinsic Camera Parameters Determine the intrinsic parameters of a camera (with lens) What are Intrinsic Parameters?

4. Sebastian Thrun CS223B Computer Vision, Winter 2005 4 Perspective Projection, Remember? fZ X O -x

5. Sebastian Thrun CS223B Computer Vision, Winter 2005 5 Intrinsic Camera Parameters Determine the intrinsic parameters of a camera (with lens) Intrinsic Parameters: –Focal Length f –Pixel size s x, s y –Distortion coefficients k 1, k 2 … –Image center o x, o y

6. Sebastian Thrun CS223B Computer Vision, Winter 2005 6 A Quiz Can we determine all intrinsic parameters by … exposing the camera to many known objects?

7. Sebastian Thrun CS223B Computer Vision, Winter 2005 7 Example Calibration Pattern

8. Sebastian Thrun CS223B Computer Vision, Winter 2005 8 Our Calibration target

9. Sebastian Thrun CS223B Computer Vision, Winter 2005 9 Harris Corner Detector

10. Sebastian Thrun CS223B Computer Vision, Winter 2005 10 Another Quiz (the last today) How Many Flat Calibration Targets are Needed for Calibration? 1: 2: 3: 4: 5: 10 How Many Corner Points do we need in Total? 1: 2: 3: 4: 10: 20

11. Sebastian Thrun CS223B Computer Vision, Winter 2005 11 Experiment 1: Parallel Board

12. Sebastian Thrun CS223B Computer Vision, Winter 2005 12 30cm10cm20cm Projective Perspective of Parallel Board

13. Sebastian Thrun CS223B Computer Vision, Winter 2005 13 Experiment 2: Tilted Board

14. Sebastian Thrun CS223B Computer Vision, Winter 2005 14 30cm10cm20cm 500cm50cm100cm Projective Perspective of Tilted Board

15. Sebastian Thrun CS223B Computer Vision, Winter 2005 15 Perspective Camera Model Object Space

16. Sebastian Thrun CS223B Computer Vision, Winter 2005 16 Calibration: 2 steps Step 1: Transform into camera coordinates Step 2: Transform into image coordinates

17. Sebastian Thrun CS223B Computer Vision, Winter 2005 17 Calibration Model (extrinsic) Homogeneous Coordinates

18. Sebastian Thrun CS223B Computer Vision, Winter 2005 18 Homogeneous Coordinates Idea: Most Operations Become Linear! Extract Image Coordinates by Z- normalization

19. Sebastian Thrun CS223B Computer Vision, Winter 2005 19 Advantage of Homogeneous C’s i-th data point

20. Sebastian Thrun CS223B Computer Vision, Winter 2005 20 Calibration Model (intrinsic) Pixel size Focal length Image center

21. Sebastian Thrun CS223B Computer Vision, Winter 2005 21 Intrinsic Transformation

22. Sebastian Thrun CS223B Computer Vision, Winter 2005 22 Plugging the Model Together!

23. Sebastian Thrun CS223B Computer Vision, Winter 2005 23 Summary Parameters Extrinsic –Rotation –Translation Intrinsic –Focal length –Pixel size –Image center coordinates –(Distortion coefficients)

24. Sebastian Thrun CS223B Computer Vision, Winter 2005 24 Q: Can We recover all Intrinsic Params? No

25. Sebastian Thrun CS223B Computer Vision, Winter 2005 25 Summary Parameters, Revisited Extrinsic –Rotation –Translation Intrinsic –Focal length –Pixel size –Image center coordinates –(Distortion coefficients) Focal length, in pixel units Aspect ratio

26. Sebastian Thrun CS223B Computer Vision, Winter 2005 26 Today’s Goals Calibration: Problem definition Solution via Singular Value Decomposition Solution by nonlinear Least Squares Distortion

27. Sebastian Thrun CS223B Computer Vision, Winter 2005 27 Calibration via SVD

28. Sebastian Thrun CS223B Computer Vision, Winter 2005 28 Calibration via SVD N>=7 points, not coplanar

29. Sebastian Thrun CS223B Computer Vision, Winter 2005 29 Calibration via SVD

30. Sebastian Thrun CS223B Computer Vision, Winter 2005 30 Calibration via SVD A has rank 7 (without proof)

31. Sebastian Thrun CS223B Computer Vision, Winter 2005 31 Calibration via SVD Remaining Problem: See book

32. Sebastian Thrun CS223B Computer Vision, Winter 2005 32 Summary, SVD Solution Replace rotation matrix by arbitrary matrix Transform into linear set of equations Solve via SVD Enforce rotation matrix (see book) Solve for remaining parameters (see book) SVD solution: algebraic minimization, assume Gaussian noise in parameter space

33. Sebastian Thrun CS223B Computer Vision, Winter 2005 33 Today’s Goals Calibration: Problem definition Solution via Singular Value Decomposition Solution by nonlinear Least Squares Distortion

34. Sebastian Thrun CS223B Computer Vision, Winter 2005 34 Calibration by nonlinear Least Squares Calibration Examples: …

35. Sebastian Thrun CS223B Computer Vision, Winter 2005 35 Calibration by nonlinear Least Squares Least Squares

36. Sebastian Thrun CS223B Computer Vision, Winter 2005 36 Calibration by nonlinear Least Squares Least Mean Square Gradient descent:

37. Sebastian Thrun CS223B Computer Vision, Winter 2005 37 Summary Non-Linear Least Squares Solve nonlinear equations via gradient descent Assume Gaussian noise in image space, not parameter space

38. Sebastian Thrun CS223B Computer Vision, Winter 2005 38 SVD Versus LQ SVD Minimization of squared distance in parameter space Globally optimal Nonlin Least Squares Minimization of squared distance in Image space Locally optimal

39. Sebastian Thrun CS223B Computer Vision, Winter 2005 39 Q: How Many Images Do We Need? Assumption: K images with M corners each 4+6K parameters 2KM constraints 2KM  4+6K  M>3 and K  2/(M-3) 2 images with 4 points, but will 1 images with 5 points work? No, since points cannot be co-planar!

40. Sebastian Thrun CS223B Computer Vision, Winter 2005 40 Today’s Goals Calibration: Problem definition Solution via Singular Value Decomposition Solution by nonlinear Least Squares Distortion

41. Sebastian Thrun CS223B Computer Vision, Winter 2005 41 Advanced Calibration: Nonlinear Distortions Barrel and Pincushion Tangential

42. Sebastian Thrun CS223B Computer Vision, Winter 2005 42 Barrel and Pincushion Distortion telewideangle

43. Sebastian Thrun CS223B Computer Vision, Winter 2005 43 Models of Radial Distortion distance from center

44. Sebastian Thrun CS223B Computer Vision, Winter 2005 44 Tangential Distortion cheap glue cheap CMOS chip cheap lense image cheap camera

45. Sebastian Thrun CS223B Computer Vision, Winter 2005 45 Image Rectification (to be continued)

46. Sebastian Thrun CS223B Computer Vision, Winter 2005 46 Summary Calibration: Problem definition Solution via Singular Value Decomposition Solution by nonlinear Least Squares Distortion