Download presentation
Presentation is loading. Please wait.
Published byGriselda Byrd Modified over 6 years ago
1
Digital Visual Effects, Spring 2006 Yung-Yu Chuang 2005/4/19
Camera calibration Digital Visual Effects, Spring 2006 Yung-Yu Chuang 2005/4/19 with slides by Richard Szeliski, Steve Seitz, and Marc Pollefyes
2
Announcements Artifacts for assignment #1 voting
3
Outline Camera projection models Camera calibration
Nonlinear least square methods Bundle adjustment
4
Camera projection models
5
Pinhole camera
6
Pinhole camera model origin principal point (optical center)
7
Pinhole camera model
8
Pinhole camera model
9
Principal point offset
intrinsic matrix
10
Is this form of K good enough?
Intrinsic matrix Is this form of K good enough? non-square pixels (digital video) skew radial distortion
11
Radial distortion
12
Camera rotation and translation
extrinsic matrix
13
Two kinds of parameters
internal or intrinsic parameters such as focal length, optical center, aspect ratio: what kind of camera? external or extrinsic (pose) parameters including rotation and translation: where is the camera?
14
Other projection models
15
Orthographic projection
Special case of perspective projection Distance from the COP to the PP is infinite Also called “parallel projection”: (x, y, z) → (x, y) Image World
16
Other types of projections
Scaled orthographic Also called “weak perspective” Affine projection Also called “paraperspective”
17
Fun with perspective
18
Perspective cues
19
Perspective cues
20
Fun with perspective Ames room
21
Forced perspective in LOTR
LOTR I Disc 4 -> Visual effects -> scale 0:00-10:00 or 15:00
22
Camera calibration
23
Camera calibration Estimate both intrinsic and extrinsic parameters
Mainly, two categories: Photometric calibration: uses reference objects with known geometry Self calibration: only assumes static scene, e.g. structure from motion
24
Camera calibration approaches
linear regression (least squares) nonlinear optinization multiple planar patterns
25
Chromaglyphs (HP research)
26
Linear regression
27
Linear regression Directly estimate 11 unknowns in the M matrix using known 3D points (Xi,Yi,Zi) and measured feature positions (ui,vi)
28
Linear regression
29
Linear regression
30
Linear regression Solve for Projection Matrix M using least-square techniques
31
Normal equation Given an overdetermined system
the normal equation is that which minimizes the sum of the square differences between left and right sides Why?
32
nxm, n equations, m variables
Normal equation nxm, n equations, m variables
33
Normal equation
34
Normal equation →
35
Linear regression Advantages: Disadvantages:
All specifics of the camera summarized in one matrix Can predict where any world point will map to in the image Disadvantages: Doesn’t tell us about particular parameters Mixes up internal and external parameters pose specific: move the camera and everything breaks
36
Nonlinear optimization
Feature measurement equations Likelihood of M given {(ui,vi)}
37
Optimal estimation Log likelihood of M given {(ui,vi)}
How do we minimize C? Non-linear regression (least squares), because ûi and vi are non-linear functions of M We can use Levenberg-Marquardt method to minimize it
38
Multi-plane calibration
Images courtesy Jean-Yves Bouguet, Intel Corp. Advantage Only requires a plane Don’t have to know positions/orientations Good code available online! Intel’s OpenCV library: Matlab version by Jean-Yves Bouget: Zhengyou Zhang’s web site:
39
Step 1: data acquisition
40
Step 2: specify corner order
41
Step 3: corner extraction
42
Step 3: corner extraction
43
Step 4: minimize projection error
44
Step 4: camera calibration
45
Step 4: camera calibration
46
Step 5: refinement
47
Nonlinear least square methods
48
Least square fitting number of data points number of parameters
49
Linear least square fitting
y t prediction residual is linear, too.
50
Nonlinear least square fitting
51
Function minimization
Least square is related to function minimization. It is very hard to solve in general. Here, we only consider a simpler problem of finding local minimum.
52
Function minimization
53
Quadratic functions Approximate the function with
a quadratic function within a small neighborhood
54
Quadratic functions A is positive definite. All eigenvalues
are positive. Fall all x, xTAx>0. negative definite A is singular A is indefinite
55
Function minimization
56
Descent methods
57
Descent direction
58
Steepest descent method
the decrease of F(x) per unit along h direction → hsd is a descent direction because hTsd F’(x)=- F’(x)2<0 It has good performance in the initial stage of the iterative process. Converge very slow with a linear rate.
59
Steepest descent method
isocontour gradient
60
Line search
61
Line search
62
Steepest descent method
63
Newton’s method → → → → It has good performance in the final stage of the iterative process, where x is close to x*.
64
Hybrid method This needs to calculate second-order derivative which might not be available.
65
Levenberg-Marquardt method
LM can be thought of as a combination of steepest descent and the Newton method. When the current solution is far from the correct one, the algorithm behaves like a steepest descent method: slow, but guaranteed to converge. When the current solution is close to the correct solution, it becomes a Newton’s method.
66
Nonlinear least square
67
Levenberg-Marquardt method
68
Levenberg-Marquardt method
μ=0 → Newton’s method μ→∞ → steepest descent method Strategy for choosing μ Start with some small μ If F is not reduced, keep trying larger μ until it does If F is reduced, accept it and reduce μ for the next iteration
69
Bundle adjustment
70
Bundle adjustment Bundle adjustment (BA) is a technique for simultaneously refining the 3D structure and camera parameters It is capable of obtaining an optimal reconstruction under certain assumptions on image error models. For zero-mean Gaussian image errors, BA is the maximum likelihood estimator.
71
Bundle adjustment n 3D points are seen in m views
xij is the projection of the i-th point on image j aj is the parameters for the j-th camera bi is the parameters for the i-th point BA attempts to minimize the projection error predicted projection Euclidean distance
72
Bundle adjustment
73
Bundle adjustment 3 views and 4 points
74
Typical Jacobian
75
Block structure of normal equation
76
Bundle adjustment
77
Bundle adjustment Multiplied by
78
Reference Manolis Lourakis and Antonis Argyros, The Design and Implementation of a Generic Sparse Bundle Adjustment Software Package Based on the Levenberg-Marquardt Algorithm, FORTH-ICS/TR K. Madsen, H.B. Nielsen, O. Timgleff, Methods for Non-Linear Least Squares Problems, 2004. Zhengyou Zhang, A Flexible New Techniques for Camera Calibration, MSR-TR-98-71, 1998. Bill Triggs, Philip McLauchlan, Richard Hartley and Andrew Fitzgibbon, Bundle Adjustment - A Modern Symthesis, Proceedings of the International Workshop on Vision Algorithms: Theory and Practice, pp , 1999.
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.