Camera parameters

Extrinisic parameters define location and orientation of camera reference frame with respect to world frame Intrinsic parameters define pixel coordinates of image point with respect to coordinates in camera reference frame

Homogenous coordinates Add an extra coordinate and define equivalence Relationship (X,Y,Z) -> (wX, wY, wZ, w) (x,y) -> (kx, ky, k) Makes it possible to write the Perspective projection as a linear Transformation (matrix) (from projective space to projective plane)

Central projection HCNHC

Scaled orthographic projection HCNHC

In a simpler notation T describes the position of the origin of camera frame with respect to world frame R describes the rotation which aligns the camera frame with the world frame P c = R(P w – T) (here –RT = B O A )

Translation and Rotation

Intrinsic parameters x y x pix y pix Scaling

Intrinsic parameters x y x pix y pix

Intrinsic parameters x y x pix y pix

The internal calibration parameters

with

Properties of matrix M M has 11 degrees of freedom (5 internal 3 rotation, 3 translation parameters), 3x4 matrix defined up to scale The 3x3 submatrix M’=M int R is non- singular (M int is upper triangular, R is orthogonal -> essential QR decomposition)

Radial distortion from lens distortion (pin cushioning effect) Straight lines are not imaged straight (significant error for cheap optics and short focal length) x and x d measured from image center

Radial calibration Using lines to be straight (x’,y’) is radial projection of (x d, y d ) on straight line

Calibration Procedure Calibration target : 2 planes at right angle with checkerboard (Tsai grid) We know positions of corners of grid with respect to a coordinate system of the target Obtain from images the corners Using the equations (relating pixel coordinates to world coordinates) we obtain the camera parameters (the internal parameters and the external (pose) as a side effect)

Image Processing Canny edge detection Straight line fitting to detect long edges Intersection of lines to detect image corners Matching image corners and 3D checkerboard corners

Estimation procedure First estimate M from corresponding image points and scene points (solving homogeneous equation) Second decompose M into internal and external parameters Use estimated parameters as starting point to solve calibration parameters non-linearly.

(homogeneous equation)

Solving A m = 0 Linear homogeneous system Have at least 5 times as many equations as unknowns (28 points) Minimize ||Am|| 2 with the constraint ||m|| 2 =1 M is the unit singular value of A corresponding to the smallest singular value (the last column of V, where A = UDV T is the SVD of A), or the eigenvector (corresponding to smallest eigenvalue ) of A T A

Finding camera translation Let be the homogeneous representation of T is the null vector of M: Null vector is found using SVD ( is the unit singular vector corresponding to the smallest singular value of M) (position of camera center)

Finding camera orientation and internal parameters Left 3x3 submatrix of M is of the form M’= M int R M int upper triangular R orthogonal Any nonsingular matrix can be decomposed into the product of an upper triangular and an orthogonal matrix (RQ factorization—here R refers to upper triangular and Q to orthogonal) (Similar to QR factorization)

RQ factorization of M’ Givens rotations To set M’ 32 to zero, solve equation Thus: Multiply M’ by R x ( such that term (3,2) is 0), then by, R y (choosing c’, s’ such that term (3,1) is zero), then by R z (with c’’, s’’ such that term (2,1) is zero)

Improving solution with nonlinear optimization Find m using the linear constraint Use as initialization for nonlinear optimization (Levenberg-Marquardt iterative minimization)

Algorithm described in Multiple View Geometry in Computer Vision (Hartley, Zisserman)