776 Computer Vision Jan-Michael Frahm Spring 2012.

Slides:



Advertisements
Similar presentations
The Camera : Computational Photography Alexei Efros, CMU, Fall 2006.
Advertisements

776 Computer Vision Jan-Michael Frahm, Enrique Dunn Spring 2013.
CSE 473/573 Computer Vision and Image Processing (CVIP) Ifeoma Nwogu Lecture 4 – Image formation(part I)
CS 691 Computational Photography Instructor: Gianfranco Doretto 3D to 2D Projections.
Computer Vision CS 776 Spring 2014 Cameras & Photogrammetry 1 Prof. Alex Berg (Slide credits to many folks on individual slides)
Paris town hall.
The Camera CS194: Image Manipulation & Computational Photography Alexei Efros, UC Berkeley, Fall 2014.
Projection Readings Szeliski 2.1. Projection Readings Szeliski 2.1.
Image formation and cameras CSE P 576 Larry Zitnick Many slides courtesy of Steve Seitz.
Cameras. Overview The pinhole projection model Qualitative properties Perspective projection matrix Cameras with lenses Depth of focus Field of view Lens.
Announcements. Projection Today’s Readings Nalwa 2.1.
Lecture 5: Projection CS6670: Computer Vision Noah Snavely.
Announcements Mailing list (you should have received messages) Project 1 additional test sequences online Talk today on “Lightfield photography” by Ren.
CS485/685 Computer Vision Prof. George Bebis
Lecture 5: Cameras and Projection CS6670: Computer Vision Noah Snavely.
CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 1 The plan for today Leftovers and from last time Camera matrix Part A) Notation,
Announcements Mailing list Project 1 test the turnin procedure *this week* (make sure it works) vote on best artifacts in next week’s class Project 2 groups.
Lecture 13: Projection, Part 2
Lecture 4a: Cameras CS6670: Computer Vision Noah Snavely Source: S. Lazebnik.
Lecture 12: Projection CS4670: Computer Vision Noah Snavely “The School of Athens,” Raphael.
Lecture 6: Image Warping and Projection CS6670: Computer Vision Noah Snavely.
CS223b, Jana Kosecka Rigid Body Motion and Image Formation.
Today: The Camera. Overview The pinhole projection model Qualitative properties Perspective projection matrix Cameras with lenses Depth of focus Field.
Single-view Metrology and Camera Calibration Computer Vision Derek Hoiem, University of Illinois 02/26/15 1.
Computer Vision Cameras, lenses and sensors Cosimo Distante Introduction to Image Processing Image.
The Camera : Computational Photography Alexei Efros, CMU, Fall 2008.
Cameras, lenses, and calibration
Image formation and cameras
Perspective projection
Cameras CSE 455, Winter 2010 January 25, 2010.
EEM 561 Machine Vision Week 10 :Image Formation and Cameras
Building a Real Camera.
Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL.
Image formation How are objects in the world captured in an image?
Northeastern University, Fall 2005 CSG242: Computational Photography Ramesh Raskar Mitsubishi Electric Research Labs Northeastern University September.
Lecture 14: Projection CS4670 / 5670: Computer Vision Noah Snavely “The School of Athens,” Raphael.
776 Computer Vision Jan-Michael Frahm Fall Camera.
Recap from Wednesday Two strategies for realistic rendering capture real world data synthesize from bottom up Both have existed for 500 years. Both are.
776 Computer Vision Jan-Michael Frahm, Enrique Dunn Spring 2013.
COMPUTER VISION D10K-7C02 CV02: Camera Dr. Setiawan Hadi, M.Sc.CS. Program Studi S-1 Teknik Informatika FMIPA Universitas Padjadjaran.
776 Computer Vision Jan-Michael Frahm Fall Last class.
EECS 274 Computer Vision Cameras.
CSE 185 Introduction to Computer Vision Cameras. Camera models –Pinhole Perspective Projection –Affine Projection –Spherical Perspective Projection Camera.
Projection Readings Nalwa 2.1 Szeliski, Ch , 2.1
Human Vision and Cameras CS 691 E Spring Outline Human vision system Human vision for computer vision Cameras and image formation Projection geometry.
Instructor: Mircea Nicolescu Lecture 3 CS 485 / 685 Computer Vision.
More with Pinhole + Single-view Metrology
Announcements Project 1 grading session this Thursday 2:30-5pm, Sieg 327 –signup ASAP:signup –10 minute slot to demo your project for a TA »have your program.
Lecture 14: Projection CS4670 / 5670: Computer Vision Noah Snavely “The School of Athens,” Raphael.
Lecture 18: Cameras CS4670 / 5670: Computer Vision KavitaBala Source: S. Lazebnik.
CSE 185 Introduction to Computer Vision
PNU Machine Vision Lecture 2a: Cameras Source: S. Lazebnik.
CS5670: Computer Vision Lecture 9: Cameras Noah Snavely
Jan-Michael Frahm Fall 2016
The Camera : Computational Photography
Announcements Project 1 Project 2
CSE 185 Introduction to Computer Vision
CSCE 441 Computer Graphics 3-D Viewing
CS 790: Introduction to Computational Vision
Jan-Michael Frahm Fall 2016
Lecture 13: Cameras and geometry
The Camera : Computational Photography
Announcements Midterm out today Project 1 demos.
Human Vision and Cameras
Projection Readings Nalwa 2.1.
Credit: CS231a, Stanford, Silvio Savarese
Projection Readings Szeliski 2.1.
Announcements Midterm out today Project 1 demos.
The Camera.
Presentation transcript:

776 Computer Vision Jan-Michael Frahm Spring 2012

Scheduling January 20 th is the first Friday class 2:00 pm – 3:15 pm in SN 115 3:30pm to 4:45 pm in SN 115

Camera

Let’s design a camera Idea 1: put a piece of film in front of an object Do we get a reasonable image? Slide by Steve Seitz

Pinhole camera Add a barrier to block off most of the rays o This reduces blurring o The opening is known as the aperture Slide by Steve Seitz

Pinhole Camera (Camera obscura) Interior of camera obscura (Sunday Magazine, 1838) Camera obscura (France, 1830)

Pinhole camera model Pinhole model: o Captures pencil of rays – all rays through a single point o The point is called Center of Projection (focal point) o The image is formed on the Image Plane Slide by Steve Seitz

Figures © Stephen E. Palmer, 2002 Dimensionality reduction: from 3D to 2D 3D world2D image What is preserved? Straight lines, incidence (except for parallel) What have we lost? Angles, lengths Slide by A. Efros

Projection properties Many-to-one: any points along same visual ray map to same point in image Points → points o But projection of points on focal plane is undefined Lines → lines (collinearity is preserved) o But lines through focal point (visual rays) project to a point Planes → planes (or half-planes) o But planes through focal point project to lines slide: S. Lazebnik

Vanishing points Each direction in space has its own vanishing point o All lines going in that direction converge at that point o Exception: directions parallel to the image plane slide: S. Lazebnik

Vanishing points Each direction in space has its own vanishing point o All lines going in that direction converge at that point o Exception: directions parallel to the image plane How do we construct the vanishing point of a line? o What about the vanishing line of a plane? image plane camera center line on ground plane vanishing point slide: S. Lazebnik

One-point perspective Masaccio, Trinity, Santa Maria Novella, Florence, One of the first consistent uses of perspective in Western art slide: S. Lazebnik

Perspective distortion Problem for architectural photography: converging verticals Source: F. Durand

Perspective distortion Problem for architectural photography: converging verticals Solution: view camera (lens shifted w.r.t. film) Source: F. Durand Tilting the camera upwards results in converging verticals Keeping the camera level, with an ordinary lens, captures only the bottom portion of the building Shifting the lens upwards results in a picture of the entire subject

Perspective distortion Problem for architectural photography: converging verticals Result: Source: F. Durand

Perspective distortion image: Wikipedia

Perspective distortion What does a sphere project to? Image source: F. Durand

Perspective distortion What does a sphere project to? slide: S. Lazebnik

Perspective distortion The exterior columns appear bigger The distortion is not due to lens flaws Problem pointed out by Da Vinci Slide by F. Durand

Perspective distortion: People slide: S. Lazebnik

Modeling projection Projection equations o Compute intersection with image plane of ray from P = (x,y,z) to O o Derived using similar triangles Source: J. Ponce, S. Seitz We get the projection by throwing out the last coordinate: x y z f

Homogeneous coordinates Is this a linear transformation? no—division by z is nonlinear Slide by Steve Seitz

Affine coordinates e i : affine basis vectors o: coordinate origin e2e2 e3e3 e1e1 o M Vector relative to o: a1a1 a3a3 a2a2 Point in affine coordinates: Vector: relative to some origin Point: absolute coordinates

Homogeneous coordinates e2e2 e3e3 e1e1 o M Unified notation: include origin in affine basis Affine basis matrix Homogeneous Coordinates of M a1a1 a3a3 a2a2

Properties of affine transformation Parallelism is preserved ratios of length, area, and volume are preserved Transformations can be concatenated: Transformation T affine combines linear mapping and coordinate shift in homogeneous coordinates  Linear mapping with A 3x3 matrix  coordinate shift with t 3 translation vector

Special transformation: Rotation Rigid transformation: Angles and lengths preserved R is orthonormal matrix defined by three angles around three coordinate axes ezez eyey exex  Rotation with angle  around e z

Special transformation: Rotation Rotation around the coordinate axes can be concatenated: ezez eyey exex    Inverse of rotation matrix is transpose:

Projective geometry in 2D Projective space is space of rays emerging from O o view point O forms projection center for all rays o rays v emerge from viewpoint into scene o ray g is called projective point, defined as scaled v: g= v x y w O

Projective and homogeneous points Given: Plane  in R 2 embedded in R 3 at coordinates w=1 o viewing ray g intersects plane at v (homogeneous coordinates) o all points on ray g project onto the same homogeneous point v o projection of g onto  is defined by scaling v=g/ = g/w w=1 O  (R 2 ) w x y

Finite and infinite points All rays g that are not parallel to  intersect at an affine point v on . w=1 O  The ray g(w=0) does not intersect . Hence v  is not an affine point but a direction. Directions have the coordinates (x,y,0) T Projective space combines affine space with infinite points (directions). (R 2 ) Remember exception of no vanishing point for directions parallel to the image plane

Perspective projection Perspective projection models pinhole camera:  scene geometry is affine R 3 space with coordinates M=(X,Y,Z,1) T  camera focal point in O=(0,0,0,1) T, camera viewing direction along Z  image plane (x,y) in  (R 2 ) aligned with plane (X,Y) at Z= Z 0  scene point M projects onto point M p on plane surface X Y Z0Z0 O  Image plane Camera center Z (Optical axis) x y (R 2 )

Projective Transformation Projective Transformation maps M onto M p X Y O Projective Transformation linearizes projection

Perspective Projection Dimension reduction from R 3 into R 2 by projection onto  (R 2 ) X Y O  (R 2 )

Perspective Projection Dimension reduction from R 3 into R 2 by projection onto  (R 2 ) X Y O  (R 2 )

Projection in general pose Rotation [R] Projection center C M World coordinates Projection: mpmp

X Y Image center c= (c x, c y ) T Projection center Z (Optical axis) Pixel scale f= (f x,f y ) T x y Pixel coordinates m = (y,x) T Image plane and image sensor A sensor with picture elements (Pixel) is added onto the image plane Image sensor Image-sensor mapping: Pixel coordinates are related to image coordinates by affine transformation K with five parameters:  Image center c=(c x,c y ) T defines optical axis  Pixel size and pixel aspect ratio defines scale f=(f x,f y ) T  image skew s to model angle between pixel rows and columns

Projection matrix P Camera projection matrix P combines:  inverse affine transformation T cam -1 from general pose to origin  Perspective projection P 0 to image plane at Z 0 =1  affine mapping K from image to sensor coordinates World to camera coord. trans. matrix (4x4) Perspective projection matrix (3x4) Camera to pixel coord. trans. matrix (3x3) = 2D point (3x1) 3D point (4x1)

Orthographic Projection Special case of perspective projection o Distance from center of projection to image plane is infinite o Also called “parallel projection” o What’s the projection matrix? Image World Slide by Steve Seitz

Home-made pinhole camera Why so blurry? Slide by A. Efros

Shrinking the aperture Why not make the aperture as small as possible? o Less light gets through o Diffraction effects… Slide by Steve Seitz

Shrinking the aperture

Adding a lens A lens focuses light onto the film o Thin lens model: Rays passing through the center are not deviated (pinhole projection model still holds) Slide by Steve Seitz

Adding a lens A lens focuses light onto the film o Thin lens model: Rays passing through the center are not deviated (pinhole projection model still holds) All parallel rays converge to one point on a plane located at the focal length f Slide by Steve Seitz focal point f

Adding a lens A lens focuses light onto the film o There is a specific distance at which objects are “in focus” other points project to a “circle of confusion” in the image “circle of confusion” Slide by Steve Seitz

Thin lens formula What is the relation between the focal length (f), the distance of the object from the optical center (D), and the distance at which the object will be in focus (D’)? f D D’ Slide by Frédo Durand object image plane lens

Thin lens formula f D D’ Similar triangles everywhere! Slide by Frédo Durand object image plane lens

Thin lens formula f D D’ Similar triangles everywhere! y’ y y’/y = D’/D Slide by Frédo Durand object image plane lens

Thin lens formula f D D’ Similar triangles everywhere! y’ y y’/y = D’/D y’/y = (D’-f)/f Slide by Frédo Durand object image plane lens

Thin lens formula f D D’ 1 D 11 f += Any point satisfying the thin lens equation is in focus. Slide by Frédo Durand object image plane lens

Depth of Field Slide by A. Efros

How can we control the depth of field? Changing the aperture size affects depth of field o A smaller aperture increases the range in which the object is approximately in focus o But small aperture reduces amount of light – need to increase exposure Slide by A. Efros

Varying the aperture Large aperture = small DOFSmall aperture = large DOF Slide by A. Efros

Field of View Slide by A. Efros What does FOV depend on?

f Field of View Smaller FOV = larger Focal Length Slide by A. Efros f FOV depends on focal length and size of the camera retina

Field of View / Focal Length Large FOV, small f Camera close to car Small FOV, large f Camera far from the car Sources: A. Efros, F. Durand

Same effect for faces standard wide-angletelephoto Source: F. Durand

The dolly zoom Continuously adjusting the focal length while the camera moves away from (or towards) the subject slide: S. Lazebnik

The Dolly Zoom

Real Lenses Zoom lens image: Simal

slide: S. Lazebnik Lens Flaws: Chromatic Aberration Lens has different refractive indices for different wavelengths: causes color fringing Near Lens Center Near Lens Outer Edge

Lens flaws: Spherical aberration Spherical lenses don’t focus light perfectly Rays farther from the optical axis focus closer slide: S. Lazebnik

Lens flaws: Vignetting slide: S. Lazebnik

No distortionPin cushionBarrel Radial Distortion o Caused by imperfect lenses o Deviations are most noticeable near the edge of the lens slide: S. Lazebnik

Digital camera A digital camera replaces film with a sensor array o Each cell in the array is light-sensitive diode that converts photons to electrons o Two common types Charge Coupled Device (CCD) Complementary metal oxide semiconductor (CMOS) o Slide by Steve Seitz

Color sensing in camera: Color filter array Source: Steve Seitz Estimate missing components from neighboring values (demosaicing) Why more green? Bayer grid Human Luminance Sensitivity Function

Problem with demosaicing: color moire Slide by F. Durand

The cause of color moire detector Fine black and white detail in image misinterpreted as color information Slide by F. Durand

Color sensing in camera: Prism Requires three chips and precise alignment More expensive CCD(B) CCD(G) CCD(R) slide: S. Lazebnik

Color sensing in camera: Foveon X3 Source: M. Pollefeys CMOS sensor Takes advantage of the fact that red, blue and green light penetrate silicon to different depths better image quality

Digital camera artifacts Noise low light is where you most notice noisenoise light sensitivity (ISO) / noise tradeoff stuck pixels In-camera processing oversharpening can produce haloshalos Compression JPEG artifacts, blocking Blooming charge overflowing into neighboring pixelsoverflowing Smearing o columnwise overexposue Color artifacts purple fringing from microlenses, purple fringing white balance Slide by Steve Seitz

Historic milestones Pinhole model: Mozi ( BCE), Aristotle ( BCE) Principles of optics (including lenses): Alhacen ( CE) Camera obscura: Leonardo da Vinci ( ), Johann Zahn ( ) First photo: Joseph Nicephore Niepce (1822) Daguerréotypes (1839) Photographic film (Eastman, 1889) Cinema (Lumière Brothers, 1895) Color Photography (Lumière Brothers, 1908) Television (Baird, Farnsworth, Zworykin, 1920s) First consumer camera with CCD Sony Mavica (1981) First fully digital camera: Kodak DCS100 (1990) Niepce, “La Table Servie,” 1822 CCD chip Alhacen’s notes

Early color photography Sergey Prokudin-Gorskii ( ) Photographs of the Russian empire ( ) Lantern projector

First digitally scanned photograph 1957, 176x176 pixels