1 Old summary of camera modelling 3 coordinate frame projection matrix decomposition intrinsic/extrinsic param

2 World coordinate frame: extrinsic parameters Finally, we should count properly...

3 ‘abstract’ camera: projection from P3 to P2 This is the most general camera model without considering optical distortion As lines are preserved so that it is a linear transformation and can be represented by a 3*4 matrix Math: central proj. Physics: pin-hole ‘new’ way of looking at ‘old’ modeling

4 11 d.o.f. Rank(P) = ? ker(P)=c row vectors, planes column vectors, directions principal plane: w=0 calibration, 6 pts decomposition by QR, K intrinsic (5). R, t, extrinsic (6) geometric interpretation of K, R, t (backward from u/x=v/y=f/z to P) internal parameters and absolute conic Properties of the 3*4 matrix P

5 It is the image of the absolute conic, prove it first! Point conic: The dual conic: What is the calibration matrix K?

6

7 Don’t forget: when the world is planar … A general plane homography!

8 Camera calibration Given Estimate C decompose C into intrinsic/extrinsic from image processing or by hand

9 Calibration set-up: 3D calibration object

10 The remaining pb is how to solve this ‘trivial’ system of equations!

11 Review of some basic numerical algorithms linear algebra: how to solve Ax=b? (non-linear optimisation) (statistics)

12 Linear algebra review Gaussian elimination LU decomposition orthogonal decomposition QR (Gram-Schmidt) SVD (the high(est)light of linear algebra!)

13 Solving (full rank) square matrix linear sys Ax =b = elimination = LU factorization 1. factor A into LU 2. solve Lc = b (lower triangular, forward substitution) 3. solve Ux=c (upper tri., backward substitution)

14 Solving for Least squares solution for Ax=b, min||Ax-b|| = pseudo-inverse x = (A^TA)-1(A^T A)b (theoretically, but not numerically) Numerically, QR does it well: as A^TA= R^TR, Orthogonal bases and Gram-Schmidt A = QR

15 Solving for homogeneous system Ax=o subject to ||x||=1, It is equivalent to min||Ax||, i.e. x^T A^T A x, the solution is the eigenvector of A^TA associated with the smallest eigenvalue Triangular systems not bad, but diagonal system is better! Diagonalization = eigen vectors => doable for symmtric matrices

16 SVD gives orthogonal bases for all subspaces row space: first Vs null space: last Vs col space: first Us null space of the trans : last Us A x = b, pseudo-inverse, x = A + b for both square system and least squares sol. Even better with homogeneous sys: A x =0, x = v_n ! You get everything with svd:

17 Linear methods of computing P p34=1 ||p||=1 ||p3||=1 Geometric interpretation of these constraints

18 Decomposition analytical by equating K(R,t)=P (QR (more exactly it is RQ))

19 1.Renormalise by c3 2.tz = c34 3.r3 = c3 4.u0 = c1^T c3 5.v0 = c2^T c3 6.alpha u 7.alpha v 8.…

20 Linear, but non-optimal, but we want optima, but non-linear, methods of computing P

21 How to solve this non-linear system of equations?

22 (Non-linear iterative optimisation) J d = r from vector F(x+d)=F(x)+J d minimize the square of y-F(x+d)=y-F(x)-J d = r – J d normal equation is J^T J d = J^T r (Gauss-Newton) (H+lambda I) d = J^T r (LM) Note: F is a vector of functions, i.e. min f=(y-F)^T(y-F)

23 Using a planar pattern Cf. the paper by Zhengyou Zhang (ICCV99), Sturm and Maybank (CVPR99) (Homework: read these papers.) Why? it is more convenient to have a planar calibration pattern than a 3D calibration object, so it’s very popular now for amateurs.

24 first estimate the plane homogrphies Hi from u and x, 1. How to estimate H? 2. Why one may not be sufficient? extract parameters from the plane homographies

25 Relationship between H and parameters: How to extract intrinsic parameters?

26 (How to extract intrinsic parameters?) The absolute conic in image The (transformed) absolute conic in the plane: The circular points of the Euclidean plane (i,1,0) and (-i,1,0) go thru this conic: two equations on K!

27 It turns the camera into an spherical one, or angular/direction sensor! Direction vector: Angle between two rays... What does the calibration give us? Normalised coordinates:

28 Summary of calibration 1.Get image-space points 2.Solve the linear system 3.Optimal sol. by non-linear method 4.Decomposition by RQ