1. COMP322/S2000/L23/L24/L251 Camera Calibration The most general case is that we have no knowledge of the camera parameters, i.e., its orientation, position, focal length, the distances between pixels in the image. Equations (non-linear) can be established to solve for these parameters (Ref. R.Tsai, in 1987 has an algorithm that will compute all these parameters). For this course, we only need to estimate the transformation matrix to allow us to compute the coordinates of any points w.r.t. the base of the robot given their corresponding pixel values, i.e.,

2. COMP322/S2000/L23/L24/L252 Camera Calibration Let Multiplying out, we have 4 equations: 12 unknowns ==> need 12 equations

3. COMP322/S2000/L23/L24/L253 Camera Calibration Three sets of known points, P 1, P 2, and P 3 : Re-arrange into Matrix form, (Ma = d): we have a (12x12) system of equations:

4. COMP322/S2000/L23/L24/L254 Camera Calibration Note: l The points have to be chosen carefully so that the 12x12 matrix is non-singular, i.e. inverse does exist. l Next question: what about values of b 13, b 23, b 33, and b 43 ? A: Due to the fact that z p is zero in the image plane, no equation can be formulated to derive these 4 values. !! Alternatively, we consider the inverse equation: Let

5. COMP322/S2000/L23/L24/L255 Camera Calibration Multiplying and obtain 4 equations: Since we can compute we can equate and sub. eq.(4) into eq.(1) to obtain: Similarly, sub. eq.(4) into eq.(2) to obtain:

6. COMP322/S2000/L23/L24/L256 Camera Calibration Re-arranging the terms, we have 2 equations: Dividing by m 44 and setting m ij <= (m ij / m 44 ), i.e., M = M/ m 44 and M becomes From the 2 equations, we have 11 unknowns, thus, we need 6 points to set up 11 equations to solve for the 11 unknowns.

7. COMP322/S2000/L23/L24/L257 Camera Calibration 11 linear equations in matrix form (Ma = d): eq.(5):

8. COMP322/S2000/L23/L24/L258 Camera Calibration Note: l Again, the 6 points have to be chosen very carefully so that the matrix is non-singular and a solution exists l Question: What about values of m 31, m 32, m 33, m 34 ? A: consider eq.(3), since all variables have positive values, ==>

9. COMP322/S2000/L23/L24/L259 Camera Calibration Thus, the transformation matrix becomes

10. COMP322/S2000/L23/L24/L2510 Relationship between base, part, and camera Now, we can compute the coordinates of any point w.r.t. the base of a robot given the corresponding pixel values of the point by the equation or Note: l M does not represent the parameters of the camera model l The choice of points is important. (LAB-4)

11. COMP322/S2000/L23/L24/L2511 Notes on LAB-4 l Develop a general program which computes the inverse of an (n by n) matrix. l Obtain test points: l use image processing to find the pixel values l use the robot arm to find the coordinates w.r.t. the base l arrange the values into matrix form l Solve for m ij in eq.(5). l Image processing program to recognize and identify the pick up locations of the part. l Question: How to compare the hole and part?

12. COMP322/S2000/L23/L24/L2512 Pattern (Shape) Matching Recall, we have some object descriptors: l Chain code of the object boundary; l Centroid of the object; l Orientation of the object; and l Bounding rectangle of the object. Some criteria for assembly task: l shape should be rotation invariant; and l shape is size dependent. (details are given in class)