1. Automatic Camera Calibration
Lu Zhang
Sep 22, 2005

2. Outline Projective geometry and camera modals
Principles of projective geometry
Camera modals
Camera calibration methods
Basic principles of self-calibration
Stratified self-calibration

3. Projective Geometry Basic principles in projective spaces: Points:
A point of a dimensional projective space is represented by an
n+1 vector of coordinates , are called
the homogeneous or projective coordinates of the points, x is called a
a coordinate vector.
If for the two n+1 vectors represent the
same point.

4. Projective Geometry The Projective Line
The space is known as projective line.
The standard projective basis of projective line is
and A point on the line is , and are not both=0.
The point at infinity:
If let , when =0,
-> infinity, we call this point ‘point at infinity’

5. Projective Geometry Projective space Points: Planes: Lines:
A line is defined as the set of points that are linearly dependent on two points.
Plane at infinity:
The points with =0 are said to be at infinity or ideal points. The set of all ideal points may be written (X; Y; Z; 0). The set of all ideal points lies on a single plane, the plane at infinity.

6. Camera models Camera models
A point M on an object with coordinates (X,Y,Z) will be imaged at some point m=(x, y) in the image plane.
If consider the effect of focal length f, the relationship between image coordinate and 3-D space coordinate can be written as
here u=U/S v=V/S if S≠0, m=PM

7. Camera modals The general intrinsic matrix is
Intrinsic parameter indicates the
property of camera itself.
focal length
pixel width
pixel height,
x coordinate at the optical centre
y coordinate at the optical centre

8. Camera models Camera Motion (Extrinsic parameters)
If we go from the old coordinate system centered at C to the new coordinate system centered at O by a rotation R followed by a translation T, in projective coordinates
The 4*4 matrix K is

9. Camera models Intrinsic calibration
The graph shows the transformation from
retinal plane to itself.
The 3*3 matrix H is given by
Cause
We have
thus

10. Self-calibration
Self-calibration refers to the process of calculating all the intrinsic parameters of the camera using only the information available in the images taken by that camera.
No calibration frame or known object is needed: the only requirement is that there is a static object in the scene, and the camera moves around taking images.

11. Self-calibration Why could we use self-calibration?
projective invariants
Epipolar geometry:
The epipole is the point of
intersection of the line
joining the optical centers
with the image plane.
Thus the epipole is the
image, in one camera, of the
optical centre of the other
camera

12. Self-calibration
a point x in one image generates a line in the other on which its corresponding point x’ must lie.
With two views, the two camera coordinate systems are related by a rotation R and a translation T.

13. Self-calibration Therefore Or if rewrite it as Then we can define E,
the essential matrix:

14. Self-calibration
If we have two views of a point M in three dimensional space, with M imaged at m in view 1 and m' in view 2
m=PM and m’=P’M
If we set C(0,0,1), then we can find p through PC=0,
where P=[P p]
e’ is the projection of C on view2, therefore

15. Self-calibration
We also can get
Then can rewrite this equation as

16. Self-calibration A is the 3*3 left corner matrix of intrinsic matrix P
We get the relationship between F and E

17. Self-calibration We can get three fundamental matrices F
By using the relationship between F,E and A, and the already known coordinate of epipolar points:
finally use a matrix equation we can calculate the parameters in matrix A which is also the parameters in Matrix P, P is the intrinsic Matrix which include camera characteristics.

18. Stratified self-calibration
What is Stratified self-calibration?
Self-calibration is the process of determining internal camera parameters directly from multiple uncalibrated images.
First, Stratified self-calibration performs a projective reconstruction. Then, it obtains an affine reconstruction as the initial value. Finally, it applies metric reconstruction.
Stratification of geometry
Projective Affine Metric

19. Stratified self-calibration
Obtaining Projective camera matrix
Determine the rectifying homography H
is the projective camera matrix for each view i
For the actual cameras, the internal parameter K is the same for each view, but in general the calibration matrix will differ for each view.
Therefore, the purpose for self-calibration is to find a rectifying homography H
Obtaining the metric reconstruction

20. Stratified self-calibration
Stratified self-calibration algorithm
Step 1: affine calibration
formulate the modulus constraint for all pairs of views, need at least 3 views
(for n>3) solve the set of equations
compute the affine projection matrices
Step 2: metric calibration
compute the position of plane at infinity
find the intrinsic parameters K
compute the metric projection matrices

21. Pipeline Projective reconstruction Relating images
Initial reconstruction
Adding views
Self-calibration
Finding plane at infinity
Compute K
Metric reconstruction
Dense depth estimation
Rectification
Dense stereo matching
Modeling

22. Projective reconstruction
Relating images
Detecting feature points
Harris corner detector
Matching feature points
RANSAC (RANdom Sampling Consensus)
Determine Fundamental Matrix
Least square estimation
The whole procedure:
Repeat
take minimal sample(8)
compute F
estimate inliers
Until (inliers, trials)> 95%
Refine F (using all inliers)

23. Projective reconstruction
Results from projective reconstruction
Feature points detection (the original images are captured from a video sequence)
After applying Harris Corner Detector

24. Projective reconstruction
Improvement by using RANSAC
Computation of Fundamental Matrix by applying least square estimation

25. Projective reconstruction
Result for projective reconstruction (con’t)
Feature points detection
After applying Harris Corner Detector
Figure Figure 10

26. Projective reconstruction
Improvement by using RANSAC
Computation of Fundamental Matrix by applying least square estimation

27. Self-calibration Finding initial Projective Matrix
Compute epipoles and epipolar lines
Epipolar lines: &
Epipoles: on all epipolar lines, thus
x  ,

28. Self-calibration Result from Initial Projective Matrix Epipolar lines
Epipoles

29. Self-calibration Result from Initial Projective Matrix Epipolar lines
Epipoles

30. Self-calibration
Projective Matrix

31. Thanks guys, any questions?