CMPUT 412 3D Computer Vision Presented by Azad Shademan Feb , 2007

Acknowledgement Marc Pollefeys’ slides (University of North Carolina), comp Spring

Motivation Images form one of the most important feedback signals in robotics. Vision-based control: –Robotic Visual Servoing –Mobile Navigation

Eye-In-Hand Visual Servoing

More motivation Course project: –Two cameras looking at one scene –Need to “correspond” similar points in these two images Laser projection on the wall Image plane Left camera Image plane Right camera

Camera Model Mostly pinhole camera model

Pinhole Camera Model

principal point Principle Point Offset

calibration matrix Principle Point Offset

Camera Rotation and Orientation

Hierarchy of transformations Projective 15dof Affine 12dof Similarity 7dof Euclidean 6dof Intersection and tangency Parallellism of planes, Volume ratios, centroids Volume

Camera Calibration Procedure that finds the camera matrix, i.e., –Intrinsic camera parameters –Extrinsic camera parameters Rotation Orientation There is a lot to be done with calibrated cameras, BUT we are more interested in uncalibrated cameras (more challenging)

Some linear algebra to get started Singular value decomposition (SVD) –Decomposes a matrix as follows:

Singular Value Decomposition

Homogeneous least-squares Span and null-space Closest rank r approximation Pseudo inverse

Parameter estimation 2D homography Given a set of (x i,x i ’), compute H (x i ’=Hx i ) 3D to 2D camera projection Given a set of (X i,x i ), compute P (x i =PX i ) Fundamental matrix Given a set of (x i,x i ’), compute F (x i ’ T Fx i =0)

Number of measurements required At least as many independent equations as degrees of freedom required Example: 2 independent equations / point 8 degrees of freedom 4x2≥8

Approximate solutions Minimal solution 4 points yield an exact solution for H More points –No exact solution, because measurements are inexact (“noise”) –Search for “best” according to some cost function –Algebraic or geometric/statistical cost

Direct Linear Transformation (DLT)

Equations are linear in h Only 2 out of 3 are linearly independent (indeed, 2 eq/pt) (only drop third row if w i ’≠0) Holds for any homogeneous representation, e.g. ( x i ’, y i ’,1)

Direct Linear Transformation (DLT) Solving for H size A is 8x9 or 12x9, but rank 8 Trivial solution is h =0 9 T is not interesting 1-D null-space yields solution of interest pick for example the one with

Direct Linear Transformation (DLT) Over-determined solution No exact solution because of inexact measurement i.e. “noise” Find approximate solution - Additional constraint needed to avoid 0, e.g. - not possible, so minimize

DLT algorithm Objective Given n≥4 2D to 2D point correspondences {x i ↔x i ’}, determine the 2D homography matrix H such that x i ’=Hx i Algorithm (i)For each correspondence x i ↔x i ’ compute A i. Usually only two first rows needed. (ii)Assemble n 2x9 matrices A i into a single 2 n x9 matrix A (iii)Obtain SVD of A. Solution for h is last column of V (iv)Determine H from h

Normalized DLT algorithm Objective Given n≥4 2D to 2D point correspondences {x i ↔x i ’}, determine the 2D homography matrix H such that x i ’=Hx i Algorithm (i)Normalize points (ii)Apply DLT algorithm to (iii)Denormalize solution

Reprojection error

OpenCV cvFindHomography() cvFindFundamentalMat() …

cvFindHomography Finds perspective transformation between two planes void cvFindHomography( const CvMat* src_points, const CvMat* dst_points, CvMat* homography ); src_points Point coordinates in the original plane, 2xN, Nx2, 3xN or Nx3 array (the latter two are for representation in homogeneous coordinates), where N is the number of points. dst_points Point coordinates in the destination plane, 2xN, Nx2, 3xN or Nx3 array (the latter two are for representation in homogeneous coordinates) homography Output 3x3 homography matrix. The function cvFindHomography finds perspective transformation H=||hij|| between the source and the destination planes: [x',,i,,] [x,,i,,] s,,i,,[y',,i,,]~H*[y,,i,,] [1 ] [ 1] So that the back-projection error is minimized: sum_i((x',,i,,-(h11*x,,i,, + h12*y,,i,, + h13)/(h31*x,,i,, + h32*y,,i,, + h33))^2^+ (y',,i,,- (h21*x,,i,, + h22*y,,i,, + h23)/(h31*x,,i,, + h32*y,,i,, + h33))^2^) -> min The function is used to find initial intrinsic and extrinsic matrices. Homography matrix is determined up to a scale, thus it is normalized to make h33=1.