1. Digital Visual Effects, Spring 2007 Yung-Yu Chuang 2007/4/17
Camera calibration
Digital Visual Effects, Spring 2007
Yung-Yu Chuang
2007/4/17
with slides by Richard Szeliski, Steve Seitz, and Marc Pollefyes

2. Announcements
Project #2 is due next Tuesday before the class

3. Outline Camera projection models Camera calibration (tools)
Nonlinear least square methods
Bundle adjustment

4. Camera projection models

5. Pinhole camera

6. Pinhole camera model (X,Y,Z) P p (x,y) origin principal point (optical
center)

7. Pinhole camera model
principal
point

8. Pinhole camera model
principal
point

9. Principal point offset
intrinsic matrix
only related to
camera projection

10. Is this form of K good enough?
Intrinsic matrix
Is this form of K good enough?
non-square pixels (digital video)
skew
radial distortion

11. Distortion Radial distortion of the image Caused by imperfect lenses
No distortion
Pin cushion
Barrel
Radial distortion of the image
Caused by imperfect lenses
Deviations are most noticeable for rays that pass through the edge of the lens

12. Camera rotation and translation
extrinsic matrix

13. Two kinds of parameters
internal or intrinsic parameters such as focal length, optical center, aspect ratio: what kind of camera?
external or extrinsic (pose) parameters including rotation and translation: where is the camera?

14. Other projection models

15. Orthographic projection
Special case of perspective projection
Distance from the COP to the PP is infinite
Also called “parallel projection”: (x, y, z) → (x, y)
Image
World

16. Other types of projections
Scaled orthographic
Also called “weak perspective”
Affine projection
Also called “paraperspective”

17. Fun with perspective

18. Perspective cues

19. Perspective cues

20. Fun with perspective
Ames room

21. Forced perspective in LOTR
LOTR I Disc 4 -> Visual effects -> scale 0:00-10:00 or 15:00

22. Camera calibration

23. Camera calibration Estimate both intrinsic and extrinsic parameters
Mainly, two categories:
Photometric calibration: uses reference objects with known geometry
Self calibration: only assumes static scene, e.g. structure from motion

24. Camera calibration approaches
linear regression (least squares)
nonlinear optimization
multiple planar patterns

25. Chromaglyphs (HP research)

26. Linear regression

27. Linear regression
Directly estimate 11 unknowns in the M matrix using known 3D points (Xi,Yi,Zi) and measured feature positions (ui,vi)

28. Linear regression

29. Linear regression

30. Linear regression
Solve for Projection Matrix M using least-square techniques

31. Normal equation Given an overdetermined system
the normal equation is that which minimizes the sum of the square differences between left and right sides
Why?

32. nxm, n equations, m variables
Normal equation
nxm, n equations, m variables

33. Normal equation

34. Normal equation →

35. Normal equation

36. Linear regression Advantages: Disadvantages:
All specifics of the camera summarized in one matrix
Can predict where any world point will map to in the image
Disadvantages:
Doesn’t tell us about particular parameters
Mixes up internal and external parameters
pose specific: move the camera and everything breaks

37. Nonlinear optimization
A probabilistic view of least square
Feature measurement equations
Likelihood of M given {(ui,vi)}

38. Optimal estimation Log likelihood of M given {(ui,vi)}
It is a least square problem (but not necessarily linear least square)
How do we minimize C?

39. We could have terms like in this
Optimal estimation
Non-linear regression (least squares), because the relations between ûi and ui are non-linear functions M
We can use Levenberg-Marquardt method to minimize it
unknown parameters
We could have terms like in this
known constant

40. A popular calibration tool

41. Multi-plane calibration
Images courtesy Jean-Yves Bouguet, Intel Corp.
Advantage
Only requires a plane
Don’t have to know positions/orientations
Good code available online!
Intel’s OpenCV library:
Matlab version by Jean-Yves Bouget:
Zhengyou Zhang’s web site:

42. Step 1: data acquisition

43. Step 2: specify corner order

44. Step 3: corner extraction

45. Step 3: corner extraction

46. Step 4: minimize projection error

47. Step 4: camera calibration

48. Step 4: camera calibration

49. Step 5: refinement

50. Nonlinear least square methods

51. Least square fitting
number of data points
number of parameters

52. Linear least square fittingy
model
parameters
residual
prediction t
is linear, too.

53. Nonlinear least square fitting

54. Function minimization
Least square is related to function minimization.
It is very hard to solve in general. Here, we only consider a simpler problem of finding local minimum.

55. Function minimization

56. Quadratic functions Approximate the function with
a quadratic function within
a small neighborhood

57. Quadratic functions A is positive definite. All eigenvalues
are positive.
Fall all x,
xTAx>0.
negative definite
A is singular
A is indefinite

58. Function minimization

59. Descent methods

60. Descent direction

61. Steepest descent method
the decrease of F(x) per
unit along h direction →
hsd is a descent direction because hTsd F’(x)=- F’(x)2<0

62. Line search

63. Line search

64. Steepest descent method
isocontour
gradient

65. Steepest descent method
It has good performance in the initial stage of the iterative process. Converge very slow with a linear rate.

66. Newton’s method → → → →
It has good performance in the final stage of the iterative process, where x is close to x*.

67. Hybrid method
This needs to calculate second-order derivative which might not be available.