Sebastian Thrun CS223B Computer Vision, Winter Stanford CS223B Computer Vision, Winter 2005 Lecture 2 Lenses and Camera Calibration Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp, Stanford

Sebastian Thrun CS223B Computer Vision, Winter News of the Day Homework assignment 1 is up(?) Reading list on the Web 14 projects on the Web

Sebastian Thrun CS223B Computer Vision, Winter Today’s Goals Thin Lens Aberrations Calibration

Sebastian Thrun CS223B Computer Vision, Winter Pinhole Camera Marc Pollefeys comp256, Lect 2 -- Brunelleschi, XVth Century

Sebastian Thrun CS223B Computer Vision, Winter Snell’s Law Snell’s law n 1 sin  1 = n 2 sin  2

Sebastian Thrun CS223B Computer Vision, Winter Thin Lens: Definition optical axis focus f Spherical lense surface: Parallel rays are refracted to single point

Sebastian Thrun CS223B Computer Vision, Winter Thin Lens: Projection optical axis z Spherical lense surface: Parallel rays are refracted to single point Image plane f

Sebastian Thrun CS223B Computer Vision, Winter Thin Lens: Projection optical axis z Spherical lense surface: Parallel rays are refracted to single point Image plane ff

Sebastian Thrun CS223B Computer Vision, Winter Thin Lens: Properties 1.Any ray entering a thin lens parallel to the optical axis must go through the focus on other side 2.Any ray entering through the focus on one side will be parallel to the optical axis on the other side

Sebastian Thrun CS223B Computer Vision, Winter Thin Lens: Model Zff O P Q R FrFr FlFl p z

Sebastian Thrun CS223B Computer Vision, Winter The Thin Lens Law Zff O P Q R FrFr FlFl p z

Sebastian Thrun CS223B Computer Vision, Winter The Thin Lens Law

Sebastian Thrun CS223B Computer Vision, Winter Limits of the Thin Lens Model 3 assumptions : 1.all rays from a point are focused onto 1 image point Remember thin lens small angle assumption 2. all image points in a single plane 3. magnification is constant Deviations from this ideal are aberrations

Sebastian Thrun CS223B Computer Vision, Winter Today’s Goals Thin Lens Aberrations Calibration

Sebastian Thrun CS223B Computer Vision, Winter Aberrations chromatic : refractive index function of wavelength 2 types : geometrical : geometry of the lense, small for paraxial rays Marc Pollefeys

Sebastian Thrun CS223B Computer Vision, Winter Geometrical Aberrations q spherical aberration q astigmatism q distortion q coma aberrations are reduced by combining lenses

Sebastian Thrun CS223B Computer Vision, Winter Spherical Aberration rays parallel to the axis do not converge outer portions of the lens yield smaller focal lenghts

Sebastian Thrun CS223B Computer Vision, Winter Astigmatism Marc Pollefeys Different focal length for inclined rays

Sebastian Thrun CS223B Computer Vision, Winter Distortion Can be corrected! (if parameters are know) pincushion (tele-photo) barrel (wide-angle) Marc Pollefeys magnification/focal length different for different angles of inclination

Sebastian Thrun CS223B Computer Vision, Winter Coma point off the axis depicted as comet shaped blob Marc Pollefeys

Sebastian Thrun CS223B Computer Vision, Winter Chromatic Aberration rays of different wavelengths focused in different planes cannot be removed completely Marc Pollefeys

Sebastian Thrun CS223B Computer Vision, Winter Vignetting Effect: Darkens pixels near the image boundary

Sebastian Thrun CS223B Computer Vision, Winter CCD vs. CMOS Mature technology Specific technology High production cost High power consumption Higher fill rate Blooming Sequential readout Recent technology Standard IC technology Cheap Low power Less sensitive Per pixel amplification Random pixel access Smart pixels On chip integration with other components Marc Pollefeys

Sebastian Thrun CS223B Computer Vision, Winter Today’s Goals Thin Lens Aberrations Calibration –Problem definition –Solution with Homogeneous Parameters –Solution by nonlinear Least Squares method –Distortion

Sebastian Thrun CS223B Computer Vision, Winter Intrinsic Camera Parameters Determine the intrinsic parameters of a camera (with lens) What are Intrinsic Parameters?

Sebastian Thrun CS223B Computer Vision, Winter Intrinsic Camera Parameters Determine the intrinsic parameters of a camera (with lens) Intrinsic Parameters: –Focal Length f –Pixel size s x, s y –Distortion coefficients k 1, k 2 … –Image center o x, o y

Sebastian Thrun CS223B Computer Vision, Winter A Quiz Can we determine all intrinsic parameters by … exposing the camera to many known objects?

Sebastian Thrun CS223B Computer Vision, Winter Example Calibration Pattern

Sebastian Thrun CS223B Computer Vision, Winter Another Quiz (the last today) How Many Flat Calibration Targets are Needed for Calibration? 1: 2: 3: 4: 5: 10 How Many Corner Points do we need in Total? 1: 2: 3: 4: 10: 20

Sebastian Thrun CS223B Computer Vision, Winter Experiment 1: Parallel Board

Sebastian Thrun CS223B Computer Vision, Winter cm10cm20cm Projective Perspective of Parallel Board

Sebastian Thrun CS223B Computer Vision, Winter Experiment 2: Tilted Board

Sebastian Thrun CS223B Computer Vision, Winter cm10cm20cm 500cm50cm100cm Projective Perspective of Tilted Board

Sebastian Thrun CS223B Computer Vision, Winter Perspective Camera Model Object Space

Sebastian Thrun CS223B Computer Vision, Winter Calibration: 2 steps Step 1: Transform into camera coordinates Step 2: Transform into image coordinates

Sebastian Thrun CS223B Computer Vision, Winter Calibration Model (extrinsic) Homogeneous Coordinates

Sebastian Thrun CS223B Computer Vision, Winter Homogeneous Coordinates Idea: Most Operations Become Linear! Extract Image Coordinates by Z- normalization

Sebastian Thrun CS223B Computer Vision, Winter Advantage of Homogeneous C’s i-th data point

Sebastian Thrun CS223B Computer Vision, Winter Calibration Model (intrinsic) Pixel size Focal length Image center

Sebastian Thrun CS223B Computer Vision, Winter Intrinsic Transformation

Sebastian Thrun CS223B Computer Vision, Winter Plugging the Model Together!

Sebastian Thrun CS223B Computer Vision, Winter Summary Parameters Extrinsic –Rotation –Translation Intrinsic –Focal length –Pixel size –Image center coordinates –(Distortion coefficients)

Sebastian Thrun CS223B Computer Vision, Winter Q: Can We recover all Intrinsic Params? No

Sebastian Thrun CS223B Computer Vision, Winter Summary Parameters, Revisited Extrinsic –Rotation –Translation Intrinsic –Focal length –Pixel size –Image center coordinates –(Distortion coefficients) Focal length, in pixel units Aspect ratio

Sebastian Thrun CS223B Computer Vision, Winter Today’s Goals Thin Lens Aberrations Calibration –Problem definition –Solution with Homogeneous Parameters –Solution by nonlinear Least Squares method –Distortion

Sebastian Thrun CS223B Computer Vision, Winter Calibration a la Trucco Substitute Advantage: Equations are linear in params If over-constrained, minimize Least Mean Square fct One possible solution: Enforce constraint that R is rotation matrix Lots of considerations to recover individual params…

Sebastian Thrun CS223B Computer Vision, Winter Today’s Goals Thin Lens Aberrations Calibration –Problem definition –Solution with Homogeneous Parameters –Solution by nonlinear Least Squares method –Distortion

Sebastian Thrun CS223B Computer Vision, Winter Calibration by nonlinear Least Squares Calibration Examples: …

Sebastian Thrun CS223B Computer Vision, Winter Calibration by nonlinear Least Squares Least Squares

Sebastian Thrun CS223B Computer Vision, Winter Calibration by nonlinear Least Squares Least Mean Square Gradient descent:

Sebastian Thrun CS223B Computer Vision, Winter Trucco Versus LQ Trucco: Minimization of Squared distance in parameter space Nonlin Least Squares Minimization of Squared distance in Image space

Sebastian Thrun CS223B Computer Vision, Winter Q: How Many Images Do We Need? Assumption: K images with M corners each 4+6K parameters 2KM constraints 2KM  4+6K  M>3 and K  2/(M-3) 2 images with 4 points, but will 1 images with 5 points work? No, since points cannot be co-planar!

Sebastian Thrun CS223B Computer Vision, Winter Today’s Goals Thin Lens Aberrations Calibration –Problem definition –Solution with Homogeneous Parameters –Solution by nonlinear Least Squares method –Distortion

Sebastian Thrun CS223B Computer Vision, Winter Advanced Calibration: Nonlinear Distortions Barrel and Pincushion Tangential

Sebastian Thrun CS223B Computer Vision, Winter Barrel and Pincushion Distortion telewideangle

Sebastian Thrun CS223B Computer Vision, Winter Models of Radial Distortion distance from center

Sebastian Thrun CS223B Computer Vision, Winter Tangential Distortion cheap glue cheap CMOS chip cheap lense image cheap camera

Sebastian Thrun CS223B Computer Vision, Winter Image Rectification (to be continued)

Sebastian Thrun CS223B Computer Vision, Winter Summary Thin Lens Aberrations Calibration –Problem definition –Solution with Homogeneous Parameters –Solution by nonlinear Least Squares method –Distortion