Sebastian Thrun CS223B Computer Vision, Winter 2005 1 Stanford CS223B Computer Vision, Winter 2005 Lecture 2 Lenses and Camera Calibration Sebastian Thrun,

Slides:



Advertisements
Similar presentations
C280, Computer Vision Prof. Trevor Darrell Lecture 2: Image Formation.
Advertisements

Single-view geometry Odilon Redon, Cyclops, 1914.
Last 4 lectures Camera Structure HDR Image Filtering Image Transform.
CS 691 Computational Photography Instructor: Gianfranco Doretto 3D to 2D Projections.
Chapter 23:Mirrors and Lenses Flat Mirrors Homework assignment : 20,24,42,45,51  Image of a point source P P’ The reflected rays entering eyes look as.
Chapter 31 Images.
Camera Calibration. Issues: what are intrinsic parameters of the camera? what is the camera matrix? (intrinsic+extrinsic) General strategy: view calibration.
Computer vision. Camera Calibration Camera Calibration ToolBox – Intrinsic parameters Focal length: The focal length in pixels is stored in the.
Light: Geometric Optics
CSc D Computer Vision – Ioannis Stamos 3-D Computer Vision CSc Camera Calibration.
Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.
Advanced Computer Vision Cameras, Lenses and Sensors.
Stanford CS223B Computer Vision, Winter 2005 Lecture 11: Structure From Motion 2 Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and.
Computer Vision Cameras, lenses and sensors Marc Pollefeys COMP 256.
Image Formation and Optics
Announcements. Projection Today’s Readings Nalwa 2.1.
Stanford CS223B Computer Vision, Winter 2006 Lecture 5 Stereo I
Lecture 5: Projection CS6670: Computer Vision Noah Snavely.
Epipolar geometry. (i)Correspondence geometry: Given an image point x in the first view, how does this constrain the position of the corresponding point.
Uncalibrated Geometry & Stratification Sastry and Yang
Announcements Mailing list (you should have received messages) Project 1 additional test sequences online Talk today on “Lightfield photography” by Ren.
CS485/685 Computer Vision Prof. George Bebis
Uncalibrated Epipolar - Calibration
Computer Vision - A Modern Approach
Announcements Mailing list Project 1 test the turnin procedure *this week* (make sure it works) vote on best artifacts in next week’s class Project 2 groups.
Sebastian Thrun & Jana Kosecka CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2006 Lecture 2: Lenses and Vision Software.
Lecture 13: Projection, Part 2
Sebastian Thrun and Jana Kosecha CS223B Computer Vision, Winter 2007 Stanford CS223B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors.
1 CS 223-B Lecture 1 Sebastian Thrun Gary Bradski CORNEA AQUEOUS HUMOR.
Single-view geometry Odilon Redon, Cyclops, 1914.
Projected image of a cube. Classical Calibration.
CS223b, Jana Kosecka Rigid Body Motion and Image Formation.
Image Formation III Chapter 1 (Forsyth&Ponce) Cameras “Lenses”
The Camera : Computational Photography Alexei Efros, CMU, Fall 2008.
Stereo Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with slides by James Rehg and.
Cameras, lenses, and calibration
Imaging Science FundamentalsChester F. Carlson Center for Imaging Science Mirrors and Lenses.
Sebastian Thrun CS223B Computer Vision, Winter Stanford CS223B Computer Vision, Winter 2006 Lecture 4 Camera Calibration Professor Sebastian Thrun.
Lecture 14: Projection CS4670 / 5670: Computer Vision Noah Snavely “The School of Athens,” Raphael.
Cameras Course web page: vision.cis.udel.edu/cv March 22, 2003  Lecture 16.
Introduction à la vision artificielle III Jean Ponce
Design of photographic lens Shinsaku Hiura Osaka University.
776 Computer Vision Jan-Michael Frahm Fall Camera.
Course 12 Calibration. 1.Introduction In theoretic discussions, we have assumed: Camera is located at the origin of coordinate system of scene.
Geometric Models & Camera Calibration
Lecture 15 Refraction, Lenses, Aberrations Chapter 23.4  23.7 Outline Atmospheric Refraction Thin Lenses Aberrations.
Geometrical Optics Chapter 24 + Other Tidbits 1. On and on and on …  This is a short week.  Schedule follows  So far, no room available for problem.
Geometric Camera Models
Optical Density - a property of a transparent medium that is an inverse measure of the speed of light through the medium. (how much a medium slows the.
Vision Review: Image Formation Course web page: September 10, 2002.
Peripheral drift illusion. Multiple views Hartley and Zisserman Lowe stereo vision structure from motion optical flow.
Single-view geometry Odilon Redon, Cyclops, 1914.
Camera Calibration Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision (with material from.
Ch. 3: Geometric Camera Calibration
CS-498 Computer Vision Week 7, Day 2 Camera Parameters Intrinsic Calibration  Linear  Radial Distortion (Extrinsic Calibration?) 1.
1 Chapter 2: Geometric Camera Models Objective: Formulate the geometrical relationships between image and scene measurements Scene: a 3-D function, g(x,y,z)
Physics 203/204 4: Geometric Optics Images formed by refraction Lens Makers Equation Thin lenses Combination of thin lenses Aberration Optical Instruments.
Single-view geometry Odilon Redon, Cyclops, 1914.
Cameras, lenses, and sensors 6.801/6.866 Profs. Bill Freeman and Trevor Darrell Sept. 10, 2002 Reading: Chapter 1, Forsyth & Ponce Optional: Section 2.1,
Image Formation III Chapter 1 (Forsyth&Ponce) Cameras “Lenses” Guido Gerig CS-GY 6643, Spring 2016 (slides modified from Marc Pollefeys, UNC Chapel Hill/
Lecture 14: Projection CS4670 / 5670: Computer Vision Noah Snavely “The School of Athens,” Raphael.
Spherical Aberration. Rays emanating from an object point that are incident on a spherical mirror or lens at different distances from the optical axis,
Geometric Model of Camera
Professor Sebastian Thrun CAs: Dan Maynes-Aminzade and Mitul Saha
Lecturer: Dr. A.H. Abdul Hafez
Announcements Midterm out today Project 1 demos.
Mirrors, Plane and Spherical Spherical Refracting Surfaces
Credit: CS231a, Stanford, Silvio Savarese
Presentation transcript:

Sebastian Thrun CS223B Computer Vision, Winter Stanford CS223B Computer Vision, Winter 2005 Lecture 2 Lenses and Camera Calibration Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp, Stanford

Sebastian Thrun CS223B Computer Vision, Winter News of the Day Homework assignment 1 is up(?) Reading list on the Web 14 projects on the Web

Sebastian Thrun CS223B Computer Vision, Winter Today’s Goals Thin Lens Aberrations Calibration

Sebastian Thrun CS223B Computer Vision, Winter Pinhole Camera Marc Pollefeys comp256, Lect 2 -- Brunelleschi, XVth Century

Sebastian Thrun CS223B Computer Vision, Winter Snell’s Law Snell’s law n 1 sin  1 = n 2 sin  2

Sebastian Thrun CS223B Computer Vision, Winter Thin Lens: Definition optical axis focus f Spherical lense surface: Parallel rays are refracted to single point

Sebastian Thrun CS223B Computer Vision, Winter Thin Lens: Projection optical axis z Spherical lense surface: Parallel rays are refracted to single point Image plane f

Sebastian Thrun CS223B Computer Vision, Winter Thin Lens: Projection optical axis z Spherical lense surface: Parallel rays are refracted to single point Image plane ff

Sebastian Thrun CS223B Computer Vision, Winter Thin Lens: Properties 1.Any ray entering a thin lens parallel to the optical axis must go through the focus on other side 2.Any ray entering through the focus on one side will be parallel to the optical axis on the other side

Sebastian Thrun CS223B Computer Vision, Winter Thin Lens: Model Zff O P Q R FrFr FlFl p z

Sebastian Thrun CS223B Computer Vision, Winter The Thin Lens Law Zff O P Q R FrFr FlFl p z

Sebastian Thrun CS223B Computer Vision, Winter The Thin Lens Law

Sebastian Thrun CS223B Computer Vision, Winter Limits of the Thin Lens Model 3 assumptions : 1.all rays from a point are focused onto 1 image point Remember thin lens small angle assumption 2. all image points in a single plane 3. magnification is constant Deviations from this ideal are aberrations

Sebastian Thrun CS223B Computer Vision, Winter Today’s Goals Thin Lens Aberrations Calibration

Sebastian Thrun CS223B Computer Vision, Winter Aberrations chromatic : refractive index function of wavelength 2 types : geometrical : geometry of the lense, small for paraxial rays Marc Pollefeys

Sebastian Thrun CS223B Computer Vision, Winter Geometrical Aberrations q spherical aberration q astigmatism q distortion q coma aberrations are reduced by combining lenses

Sebastian Thrun CS223B Computer Vision, Winter Spherical Aberration rays parallel to the axis do not converge outer portions of the lens yield smaller focal lenghts

Sebastian Thrun CS223B Computer Vision, Winter Astigmatism Marc Pollefeys Different focal length for inclined rays

Sebastian Thrun CS223B Computer Vision, Winter Distortion Can be corrected! (if parameters are know) pincushion (tele-photo) barrel (wide-angle) Marc Pollefeys magnification/focal length different for different angles of inclination

Sebastian Thrun CS223B Computer Vision, Winter Coma point off the axis depicted as comet shaped blob Marc Pollefeys

Sebastian Thrun CS223B Computer Vision, Winter Chromatic Aberration rays of different wavelengths focused in different planes cannot be removed completely Marc Pollefeys

Sebastian Thrun CS223B Computer Vision, Winter Vignetting Effect: Darkens pixels near the image boundary

Sebastian Thrun CS223B Computer Vision, Winter CCD vs. CMOS Mature technology Specific technology High production cost High power consumption Higher fill rate Blooming Sequential readout Recent technology Standard IC technology Cheap Low power Less sensitive Per pixel amplification Random pixel access Smart pixels On chip integration with other components Marc Pollefeys

Sebastian Thrun CS223B Computer Vision, Winter Today’s Goals Thin Lens Aberrations Calibration –Problem definition –Solution with Homogeneous Parameters –Solution by nonlinear Least Squares method –Distortion

Sebastian Thrun CS223B Computer Vision, Winter Intrinsic Camera Parameters Determine the intrinsic parameters of a camera (with lens) What are Intrinsic Parameters?

Sebastian Thrun CS223B Computer Vision, Winter Intrinsic Camera Parameters Determine the intrinsic parameters of a camera (with lens) Intrinsic Parameters: –Focal Length f –Pixel size s x, s y –Distortion coefficients k 1, k 2 … –Image center o x, o y

Sebastian Thrun CS223B Computer Vision, Winter A Quiz Can we determine all intrinsic parameters by … exposing the camera to many known objects?

Sebastian Thrun CS223B Computer Vision, Winter Example Calibration Pattern

Sebastian Thrun CS223B Computer Vision, Winter Another Quiz (the last today) How Many Flat Calibration Targets are Needed for Calibration? 1: 2: 3: 4: 5: 10 How Many Corner Points do we need in Total? 1: 2: 3: 4: 10: 20

Sebastian Thrun CS223B Computer Vision, Winter Experiment 1: Parallel Board

Sebastian Thrun CS223B Computer Vision, Winter cm10cm20cm Projective Perspective of Parallel Board

Sebastian Thrun CS223B Computer Vision, Winter Experiment 2: Tilted Board

Sebastian Thrun CS223B Computer Vision, Winter cm10cm20cm 500cm50cm100cm Projective Perspective of Tilted Board

Sebastian Thrun CS223B Computer Vision, Winter Perspective Camera Model Object Space

Sebastian Thrun CS223B Computer Vision, Winter Calibration: 2 steps Step 1: Transform into camera coordinates Step 2: Transform into image coordinates

Sebastian Thrun CS223B Computer Vision, Winter Calibration Model (extrinsic) Homogeneous Coordinates

Sebastian Thrun CS223B Computer Vision, Winter Homogeneous Coordinates Idea: Most Operations Become Linear! Extract Image Coordinates by Z- normalization

Sebastian Thrun CS223B Computer Vision, Winter Advantage of Homogeneous C’s i-th data point

Sebastian Thrun CS223B Computer Vision, Winter Calibration Model (intrinsic) Pixel size Focal length Image center

Sebastian Thrun CS223B Computer Vision, Winter Intrinsic Transformation

Sebastian Thrun CS223B Computer Vision, Winter Plugging the Model Together!

Sebastian Thrun CS223B Computer Vision, Winter Summary Parameters Extrinsic –Rotation –Translation Intrinsic –Focal length –Pixel size –Image center coordinates –(Distortion coefficients)

Sebastian Thrun CS223B Computer Vision, Winter Q: Can We recover all Intrinsic Params? No

Sebastian Thrun CS223B Computer Vision, Winter Summary Parameters, Revisited Extrinsic –Rotation –Translation Intrinsic –Focal length –Pixel size –Image center coordinates –(Distortion coefficients) Focal length, in pixel units Aspect ratio

Sebastian Thrun CS223B Computer Vision, Winter Today’s Goals Thin Lens Aberrations Calibration –Problem definition –Solution with Homogeneous Parameters –Solution by nonlinear Least Squares method –Distortion

Sebastian Thrun CS223B Computer Vision, Winter Calibration a la Trucco Substitute Advantage: Equations are linear in params If over-constrained, minimize Least Mean Square fct One possible solution: Enforce constraint that R is rotation matrix Lots of considerations to recover individual params…

Sebastian Thrun CS223B Computer Vision, Winter Today’s Goals Thin Lens Aberrations Calibration –Problem definition –Solution with Homogeneous Parameters –Solution by nonlinear Least Squares method –Distortion

Sebastian Thrun CS223B Computer Vision, Winter Calibration by nonlinear Least Squares Calibration Examples: …

Sebastian Thrun CS223B Computer Vision, Winter Calibration by nonlinear Least Squares Least Squares

Sebastian Thrun CS223B Computer Vision, Winter Calibration by nonlinear Least Squares Least Mean Square Gradient descent:

Sebastian Thrun CS223B Computer Vision, Winter Trucco Versus LQ Trucco: Minimization of Squared distance in parameter space Nonlin Least Squares Minimization of Squared distance in Image space

Sebastian Thrun CS223B Computer Vision, Winter Q: How Many Images Do We Need? Assumption: K images with M corners each 4+6K parameters 2KM constraints 2KM  4+6K  M>3 and K  2/(M-3) 2 images with 4 points, but will 1 images with 5 points work? No, since points cannot be co-planar!

Sebastian Thrun CS223B Computer Vision, Winter Today’s Goals Thin Lens Aberrations Calibration –Problem definition –Solution with Homogeneous Parameters –Solution by nonlinear Least Squares method –Distortion

Sebastian Thrun CS223B Computer Vision, Winter Advanced Calibration: Nonlinear Distortions Barrel and Pincushion Tangential

Sebastian Thrun CS223B Computer Vision, Winter Barrel and Pincushion Distortion telewideangle

Sebastian Thrun CS223B Computer Vision, Winter Models of Radial Distortion distance from center

Sebastian Thrun CS223B Computer Vision, Winter Tangential Distortion cheap glue cheap CMOS chip cheap lense image cheap camera

Sebastian Thrun CS223B Computer Vision, Winter Image Rectification (to be continued)

Sebastian Thrun CS223B Computer Vision, Winter Summary Thin Lens Aberrations Calibration –Problem definition –Solution with Homogeneous Parameters –Solution by nonlinear Least Squares method –Distortion