Calibration ECE 847: Digital Image Processing Stan Birchfield Clemson University

Image projection Project world point (x,y,z) to image point (u,v) Assume pinhole camera model

Image projection (x,y,z) world point(u,v) image point internal calibrationexternal calibration (pose) Total parameters: 11 (5 internal, 6 external)

Internal calibration parameters f x : horizontal focal length f y : vertical focal length  : skew (angle b/w horizontal and vertical axes – p/2 for most real cameras (u 0, v 0 ): principal point (intersection of optical axis with image plane) where

Two approaches to calibration Calibrate to meaningful fixed world coord system –Solve for P directly –Good for fixed camera, specific application (e.g., tracking vehicles on highway, mobile robot on ground plane) –But gives no insight into internal calibration parameters –If camera-world relationship changes, calibration must start from scratch Compute internal and external parameters separately –P = A[R t] –Internal parameters turn camera into a metric device –Can now be used for computing 3D rays (and points) in Euclidean space –Necessary for SFM

Homography mapping from world plane

Bird’s eye view Use normalized DLT to find H Now can unwarp image to provide bird’s eye (top-down) view of scene This upgrades projective to Euclidean

Stratification of geometries Euclidean Similarity Affine Projective more transformations, fewer invariants identify l ∞ identify circular points single known length

Calibration algorithms Tsai 1987 –Uses some known parameters from camera specs –Requires 3D calibration target with orthogonal planes Zhang 2000 –Uses projective geometry –Only requires planar calibration target –Much simpler

Zhang’s algorithm Based on the image of the absolute conic (IAC): Algorithm at a glance: –Find  ∞ –Decompose  ∞ to get K What is the IAC?

Celestial sphere Celestial sphere is imaginary sphere with infinite radius Points on sphere (stars) unaffected by translation

Plane at infinity  ∞ is idealized celestial sphere points at infinity lie on  ∞ =(0,0,0,1) T last coordinate is zero invariant to translation

Line at infinity l ∞ is analogous quantity in 2D points at infinity lie on l ∞, capture direction Euclidean plane l ∞ =(0,0,1) T

Absolute points (2D) Two special points on l ∞ : (1, ±i, 0) These are the absolute points, or circular points All circles intersect at absolute points absolute points satisfy x 2 +y 2 =0, w=0 absolute points  Euclidean

Ellipses and circles Q1: Two ellipses intersect at 4 points, two circles at only 2. Why? Q2: An ellipse is defined by 5 points, circle by only 3. Why?

All circles intersect at absolute points if   Answer: Therefore, the absolute points lie on every circle

Absolute points are invariant to similarity transformations

Absolute points encode Euclidean (similarity) geometry in a single compact representation

Absolute conic (3D) All spheres intersect at absolute conic plane at infinity:  ∞ =(0,0,0,1) points at infinity: (x,y,z,0) absolute conic satisfies x 2 +y 2 +z 2 =0, w=0 Therefore,  ∞ =I 3x3 (identity matrix) in a metric frame  ∞ contains purely imaginary points absolute conic  Euclidean

IAC Projection of the absolute conic on the image plane Image of the absolute conic (IAC) IAC  calibrated camera (internal parameters) Projection of  ∞ onto image plane is homography –Translate camera, no change to this projection (cf. seeing stars on celestial sphere) –Rotate camera, projection does change –But IAC, which is the projection of  ∞, does not change with translation or rotation –H = KR for plane at infinity, but  ∞ = P  ∞ = K -T K -1

Algorithm overview Take images of multiple planes Each plane contains absolute points Projection of absolute points lie on IAC Use multiple planes to perform least squares fit to IAC Decompose IAC to get K (internal parameters)

Zhang’s algorithm Recall: Euclidean constraints on rotation matrix: Substitute to yield constraints on K:

Cholesky decomposition Recall: Cholesky factorizes any symmetric and positive definite matrix into product of lower and upper triangular matrices:  ∞ = LU 0 K: 0 KT:KT: 0 K -1 : 0 K -T :

Zhang’s algorithm (bundle adjustment)

Geometric interpretation Draw projection of IAC

Image of absolute points yields same constraints Projection of plane is homography H Leads to constraint: Splitting into real and imaginary:

Tsai’s algorithm

What can you do with a calibrated camera

Radial lens distortion

r u = r d + k 1 r d 3 where k 1 > 0 for barrel distortion and k 1 < 0 for pincushion. Radial lens distortion

Lens distortion It is frequently asserted that negative values of D correspond to barrel distortion, and positive values to pincushion distortion. For simple distortion curves this is true, but a few lenses with a more complex distortion curve do not follow this simple rule. As a case in point I consider the above Distagon 2.8/21. Fig. 4 shows a grid distorted according to the data in Fig. 3. Although curve A is negative everywhere, the grid reveals significant pincushion distortion toward the corners. Indeed, it is not the sign of D which determines the type of distortion, but the slope of the curve. A negative slope (yellow part) of D implies barrel distortion, a positive slope (orange part) pincushion distortion. The steeper the slope, the more pronounced the distortion. Barrel distortion typically will have a positive term for K1 where as pincushion distortion will have a negative value. --