1. ROBOT VISION Lesson 4: Camera Models and Calibration Matthias Rüther Slides partial courtesy of Marc Pollefeys Department of Computer Science University of North Carolina, Chapel Hill
Kawada Industries Inc. has introduced the HRP-2P for Robodex 2002.  This humanoid appears to be very impressive. It is 154 cm (60") tall, weighs 58kg (127 lbs) and has 30 DOF. Here is a news release. Notice the LACK of a battery pack.  Here is a new story about HRP2. 

2. Content Camera Models Calibration Pinhole Camera CCD Camera
Finite Projective Camera
Affine Camera
Pushbroom Camera
Calibration
Inner Orientation
Nonlinear Distortion
Calibration using Planar Targets
Calibration using a 3D Target
The Cyclops, 1914 by Odilon Redon

3. Basic Pinhole Camera Model

4. Basic Pinhole Camera Model

5. Basic Pinhole Camera Model

6. Principal Point Offset

7. Principal Point Offset
calibration matrix

8. Camera Rotation and Translation

9. CCD Camera

10. Finite Projective Camera
11 dof (5+3+3)
non-singular
decompose P in K,R,C?
{finite cameras}={P3x4 | det M≠0}
If rank P=3, but rank M<3, then cam at infinity

11. Action of Projective Cameras on Points
Forward projection (3D -> 2D)
D…direction
Back-projection (2D -> 3D)
(pseudo-inverse)

12. Projective Depth of Points
(PC=0)
(dot product)
If ,
then m3 unit vector in positive direction

13. Image from image, s≠0 possible (non coinciding principal axis)
When is skew non-zero? 1 g
arctan(1/s)
for CCD/CMOS, always s=0
Image from image, s≠0 possible
(non coinciding principal axis)
resulting camera:

14. Moving the Camera Center to Infinity
Camera center at infinity
Affine and non-affine cameras
Definition: affine camera has P3T=(0,0,0,1)

15. Affine Cameras

16. Parallel Projection: Summary
canonical representation
affine calibration matrix
principal point is not defined

17. A Hierarchy of Affine Cameras
Orthographic projection
(5dof)
Scaled orthographic projection
(6dof)

18. A Hierarchy of Affine Cameras
Weak perspective projection
(7dof)

19. A Hierarchy of Affine Cameras
(8dof)
Affine camera=camera with principal plane coinciding with P∞
Affine camera maps parallel lines to parallel lines
No center of projection, but direction of projection PAD=0
(point on P∞)

20. Pushbroom Cameras (11dof)
Straight lines are not mapped to straight lines!
(otherwise it would be a projective camera)

21. Line Cameras (5dof) Null-space PC=0 yields camera center
Also decomposition

22. Camera calibration

23. Problem Statement

24. Basic Equations

25. Basic Equations n  6 points minimal solution
P has 11 dof, 2 independent eq./points
5½ correspondences needed (say 6)
Over-determined solution
n  6 points
minimize subject to constraint

26. Geometric Error

27. Gold Standard algorithm
Objective
Given n≥6 2D to 2D point correspondences {Xi↔xi’}, determine the Maximum Likelihood Estimation of P
Algorithm
Linear solution:
Normalization:
DLT:
Minimization of geometric error: using the linear estimate as a starting point minimize the geometric error:
Denormalization: ~ ~ ~

28. Exterior Orientation
Calibrated camera, position and orientation unkown
 Pose estimation
6 dof  3 points minimal (4 solutions in general)

29. short and long focal length
Nonlinear Distortion
Radial Component
short and long focal length

30. Nonlinear Distortion
Radial Component

31. Correction of radial Distortion
Computing the parameters of the distortion function
Minimize with additional unknowns
Straighten lines …

32. Nonlinear Distortion
Tangential Component
Distortion function:

34. Distortion Workflow (KU wp1)
Project world points: x = P*X
Normalize projected points: xn = K-1 * x
Apply distortion to xn : xd = fdistort(xn)
Denormalize: xp = K * xd