1. Many slides and illustrations from J. Ponce
Calibration
Marc Pollefeys
COMP 256
Many slides and illustrations from J. Ponce

2. Tentative class schedule
Aug 26/28 -
Introduction
Sep 2/4
Cameras
Radiometry
Sep 9/11
Sources & Shadows
Color
Sep 16/18
Linear filters & edges
(hurricane Isabel)
Sep 23/25
Pyramids & Texture
Multi-View Geometry
Sep30/Oct2
Stereo
Project proposals
Oct 7/9
Tracking (Welch)
Optical flow
Oct 14/16
Oct 21/23
Silhouettes/carving
(Fall break)
Oct 28/30
Structure from motion
Nov 4/6
Project update
Proj. SfM
Nov 11/13
Camera calibration
Segmentation
Nov 18/20
Fitting
Prob. segm.&fit.
Nov 25/27
Matching templates
(Thanksgiving)
Dec 2/4
Matching relations
Range data
Dec ?
Final project

3. GEOMETRIC CAMERA MODELS
Elements of Euclidean Geometry
The Intrinsic Parameters of a Camera
The Extrinsic Parameters of a Camera
The General Form of the Perspective Projection Equation
Line Geometry
Reading: Chapter 2.

4. Quantitative Measurements and Calibration
Euclidean Geometry

5. Euclidean Coordinate Systems

6. Planes

7. OBP = OBOA + OAP , BP = AP + BOA
Coordinate Changes: Pure Translations
OBP = OBOA + OAP , BP = AP + BOA

8. Coordinate Changes: Pure Rotations

9. Coordinate Changes: Rotations about the z Axis

10. A rotation matrix is characterized by the following properties:
Its inverse is equal to its transpose, and
its determinant is equal to 1.
Or equivalently:
Its rows (or columns) form a right-handed
orthonormal coordinate system.

11. Coordinate Changes:
Pure Rotations

12. Coordinate Changes: Rigid Transformations

13. Block Matrix Multiplication
What is AB ?
Homogeneous Representation of Rigid Transformations

14. Rigid Transformations as Mappings

15. Rigid Transformations as Mappings:
Rotation about the k Axis

16. Pinhole Perspective Equationï î í ì = z y f x '

17. The Intrinsic Parameters of a Camera
Units:
k,l : pixel/m
f : m
a,b
: pixel
Physical Image Coordinates
Normalized Image
Coordinates

18. The Intrinsic Parameters of a Camera
Calibration Matrix
The Perspective
Projection Equation

19. Extrinsic Parameters

20. Explicit Form of the Projection Matrix
Note:
M is only defined up to scale in this setting!!

21. Theorem (Faugeras, 1993)

22. GEOMETRIC CAMERA CALIBRATION
The Calibration Problem
Least-Squares Techniques
Linear Calibration from Points
Linear Calibration from Lines
Analytical Photogrammetry
Reading: Chapter 3

23. Calibration Problem

24. A x b = A x b = Linear Systems Square system: unique solution
Gaussian elimination A x b =
Rectangular system ??
underconstrained:
infinity of solutions A x b =
overconstrained:
no solution 2
Minimize |Ax-b|

25. How do you solve overconstrained linear equations ??

26. A x = A x = Homogeneous Linear Systems Square system:
unique solution: 0
unless Det(A)=0 =
Rectangular system ??
0 is always a solution A x = 2
Minimize |Ax|
under the constraint |x| =1 2

27. remember: EIG(ATA)=SVD(A), i.e. solution is Vn
How do you solve overconstrained
homogeneous linear equations ??
The solution is e . 1
remember: EIG(ATA)=SVD(A), i.e. solution is Vn

28. Linear Camera Calibration

29. Once M is known, you still got to recover the intrinsic and
extrinsic parameters !!!
This is a decomposition problem, not an estimation
problem. r
Intrinsic parameters
Extrinsic parameters

30. Degenerate Point Configurations
Are there other solutions besides M ??
Coplanar points: (l,m,n )=(P,0,0) or (0,P,0) or (0,0,P )
Points lying on the intersection curve of two quadric
surfaces
= straight line + twisted cubic
Does not happen for 6 or more random points!

31. Analytical Photogrammetry
Non-Linear Least-Squares Methods
Newton
Gauss-Newton
Levenberg-Marquardt
Iterative, quadratically convergent in favorable situations

32. Mobile Robot Localization (Devy et al., 1997)

33. From Projective to Euclidean Images
If z , P , R and t are solutions, so are
l z , l P , R and l t .
Absolute scale cannot be recovered! The Euclidean shape
(defined up to an arbitrary similitude) is the best that can be
recovered.

34. From uncalibrated to calibrated cameras
Perspective camera:
Calibrated camera:
Problem: what is Q ?

35. From uncalibrated to calibrated cameras II
Perspective camera:
Calibrated camera:
Problem: what is Q ?
Example: known image center

36. Sequential SfM Initialize motion from two images Initialize structure
For each additional view
Determine pose of camera
Refine and extend structure
Refine structure and motion

37. Initial projective camera motion
Choose P and P´compatible with F
Reconstruction up to projective ambiguity
(reference plane;arbitrary)
Same for more views?
Initialize motion
Initialize structure
For each additional view
Determine pose of camera
Refine and extend structure
Refine structure and motion
different projective basis

38. Initializing projective structure
Reconstruct matches in projective frame
by minimizing the reprojection error
Non-iterative optimal solution
Initialize motion
Initialize structure
For each additional view
Determine pose of camera
Refine and extend structure
Refine structure and motion

39. Projective pose estimation
Infere 2D-3D matches from 2D-2D matches
Compute pose from (RANSAC,6pts) X F x
Initialize motion
Initialize structure
For each additional view
Determine pose of camera
Refine and extend structure
Refine structure and motion
Inliers:

40. Refining and extending structure
Refining structure
Extending structure
2-view triangulation
(Iterative linear)
Initialize motion
Initialize structure
For each additional view
Determine pose of camera
Refine and extend structure
Refine structure and motion

41. Refining structure and motion
use bundle adjustment
                       
Also model radial distortion to avoid bias!

42. Metric structure and motion
use self-calibration (see next class)
Note that a fundamental problem of the uncalibrated approach is
that it fails if a purely planar scene is observed (in one or more views)
(solution possible based on model selection)

43. Dealing with dominant planes

44. PPPgric
HHgric

45. Farmhouse 3D models
(note: reconstruction much larger than camera field-of-view)

46. Application: video augmentation

47. Next class: Segmentation
Reading: Chapter 14