Lecture 13: Projection, Part 2

Slides:



Advertisements
Similar presentations
Last 4 lectures Camera Structure HDR Image Filtering Image Transform.
Advertisements

The Camera : Computational Photography Alexei Efros, CMU, Fall 2006.
CS 691 Computational Photography Instructor: Gianfranco Doretto 3D to 2D Projections.
More Mosaic Madness : Computational Photography Alexei Efros, CMU, Fall 2011 © Jeffrey Martin (jeffrey-martin.com) with a lot of slides stolen from.
RANSAC and mosaic wrap-up cs195g: Computational Photography James Hays, Brown, Spring 2010 © Jeffrey Martin (jeffrey-martin.com) most slides from Steve.
Computer Vision CS 776 Spring 2014 Cameras & Photogrammetry 1 Prof. Alex Berg (Slide credits to many folks on individual slides)
Paris town hall.
Projection Readings Szeliski 2.1. Projection Readings Szeliski 2.1.
Image formation and cameras CSE P 576 Larry Zitnick Many slides courtesy of Steve Seitz.
Camera Models A camera is a mapping between the 3D world and a 2D image The principal camera of interest is central projection.
Announcements. Projection Today’s Readings Nalwa 2.1.
Lecture 5: Projection CS6670: Computer Vision Noah Snavely.
Lecture 7: Image Alignment and Panoramas CS6670: Computer Vision Noah Snavely What’s inside your fridge?
Announcements Mailing list (you should have received messages) Project 1 additional test sequences online Talk today on “Lightfield photography” by Ren.
CS485/685 Computer Vision Prof. George Bebis
Lecture 5: Cameras and Projection CS6670: Computer Vision Noah Snavely.
Lecture 6: Image transformations and alignment CS6670: Computer Vision Noah Snavely.
Camera model Relation between pixels and rays in space ?
Announcements Mailing list Project 1 test the turnin procedure *this week* (make sure it works) vote on best artifacts in next week’s class Project 2 groups.
1Jana Kosecka, CS 223b Cylindrical panoramas Cylindrical panoramas with some slides from R. Szeliski, S. Seitz, D. Lowe, A. Efros,
Lecture 12: Projection CS4670: Computer Vision Noah Snavely “The School of Athens,” Raphael.
Lecture 6: Image Warping and Projection CS6670: Computer Vision Noah Snavely.
Lecture 20: Two-view geometry CS6670: Computer Vision Noah Snavely.
The Pinhole Camera Model
More Mosaic Madness : Computational Photography Alexei Efros, CMU, Fall 2005 © Jeffrey Martin (jeffrey-martin.com) with a lot of slides stolen from.
Panoramas and Calibration : Rendering and Image Processing Alexei Efros …with a lot of slides stolen from Steve Seitz and Rick Szeliski.
More Mosaic Madness : Computational Photography Alexei Efros, CMU, Fall 2006 © Jeffrey Martin (jeffrey-martin.com) with a lot of slides stolen from.
CS4670 / 5670: Computer Vision KavitaBala Lecture 15: Projection “The School of Athens,” Raphael.
The Camera : Computational Photography Alexei Efros, CMU, Fall 2008.
Announcements Project 1 artifact voting Project 2 out today (help session at end of class)
Cameras, lenses, and calibration
Image formation and cameras
Perspective projection
EEM 561 Machine Vision Week 10 :Image Formation and Cameras
Announcements Midterm due Friday, beginning of lecture Guest lecture on Friday: Antonio Criminisi, Microsoft Research.
Lecture 14: Projection CS4670 / 5670: Computer Vision Noah Snavely “The School of Athens,” Raphael.
776 Computer Vision Jan-Michael Frahm Fall Camera.
Recap from Wednesday Two strategies for realistic rendering capture real world data synthesize from bottom up Both have existed for 500 years. Both are.
776 Computer Vision Jan-Michael Frahm, Enrique Dunn Spring 2013.
CS559: Computer Graphics Lecture 9: Projection Li Zhang Spring 2008.
Geometric Camera Models
Projection Readings Nalwa 2.1 Szeliski, Ch , 2.1
1 Chapter 2: Geometric Camera Models Objective: Formulate the geometrical relationships between image and scene measurements Scene: a 3-D function, g(x,y,z)
Image Formation and Cameras CSE 455 Linda Shapiro 1.
CS559: Computer Graphics Lecture 9: 3D Transformation and Projection Li Zhang Spring 2010 Most slides borrowed from Yungyu ChuangYungyu Chuang.
Announcements Project 1 grading session this Thursday 2:30-5pm, Sieg 327 –signup ASAP:signup –10 minute slot to demo your project for a TA »have your program.
Example: warping triangles Given two triangles: ABC and A’B’C’ in 2D (12 numbers) Need to find transform T to transfer all pixels from one to the other.
Lecture 14: Projection CS4670 / 5670: Computer Vision Noah Snavely “The School of Athens,” Raphael.
Lecture 18: Cameras CS4670 / 5670: Computer Vision KavitaBala Source: S. Lazebnik.
CSE 185 Introduction to Computer Vision
Computer vision: models, learning and inference
Prof. Adriana Kovashka University of Pittsburgh October 3, 2016
CS4670 / 5670: Computer Vision Kavita Bala Lecture 20: Panoramas.
More Mosaic Madness © Jeffrey Martin (jeffrey-martin.com)
CS5670: Computer Vision Lecture 9: Cameras Noah Snavely
Jan-Michael Frahm Fall 2016
Announcements Project 1 Project 2
Announcements Project 1 Project 2 Due Wednesday at 11:59pm
CS 790: Introduction to Computational Vision
More Mosaic Madness : Computational Photography
Overview Pin-hole model From 3D to 2D Camera projection
More Mosaic Madness © Jeffrey Martin (jeffrey-martin.com)
More Mosaic Madness : Computational Photography
Lecture 13: Cameras and geometry
Geometric Camera Models
Lecturer: Dr. A.H. Abdul Hafez
Announcements Midterm out today Project 1 demos.
Projection Readings Nalwa 2.1.
Credit: CS231a, Stanford, Silvio Savarese
Projection Readings Szeliski 2.1.
Presentation transcript:

Lecture 13: Projection, Part 2 CS4670: Computer Vision Noah Snavely Lecture 13: Projection, Part 2 Perspective study of a vase by Paolo Uccello

Reading Szeliski 2.1.3-2.1.6

Announcements Project 2a due Friday, 8:59pm Project 2b out Friday Take-home prelim after Fall break

Orthographic projection Special case of perspective projection Distance from the COP to the PP is infinite Good approximation for telephoto optics Also called “parallel projection”: (x, y, z) → (x, y) What’s the projection matrix? Image World

Orthographic projection

Perspective projection

Projection properties Parallel lines converge at a vanishing point Each direction in space has its own vanishing point But parallels parallel to the image plane remain parallel

Camera parameters How can we model the geometry of a camera? y x z COP v w u x y z COP Camera Three important coordinate systems: 1. World coordinate system (3D) 2. Camera coordinate system (3D) 3. Image coordinate system (2D) o “The World” We will define a projection matrix that maps from 1  3 (via 2)

Camera parameters To project a point (x,y,z) in world coordinates into a camera First transform (x,y,z) into camera coordinates (u,v,w) Need to know Camera position (in world coordinates) Camera orientation (in world coordinates) Then project into the image plane Need to know camera intrinsics

Camera parameters A camera is described by several parameters Translation T of the optical center from the origin of world coords Rotation R of the image plane focal length f, principle point (x’c, y’c), pixel size (sx, sy) blue parameters are called “extrinsics,” red are “intrinsics” Projection equation The projection matrix models the cumulative effect of all parameters Useful to decompose into a series of operations projection intrinsics rotation translation identity matrix The definitions of these parameters are not completely standardized especially intrinsics—varies from one book to another

Extrinsics How do we get the camera to “canonical form”? (Center of projection at the origin, x-axis points right, y-axis points up, z-axis points backwards) Step 1: Translate by -c

Extrinsics How do we get the camera to “canonical form”? (Center of projection at the origin, x-axis points right, y-axis points up, z-axis points backwards) Step 1: Translate by -c How do we represent translation as a matrix multiplication?

Extrinsics How do we get the camera to “canonical form”? (Center of projection at the origin, x-axis points right, y-axis points up, z-axis points backwards) Step 1: Translate by -c Step 2: Rotate by R 3x3 rotation matrix

Extrinsics How do we get the camera to “canonical form”? (Center of projection at the origin, x-axis points right, y-axis points up, z-axis points backwards) Step 1: Translate by -c Step 2: Rotate by R

Perspective projection (converts from 3D rays in camera coordinate system to pixel coordinates) (intrinsics) in general, (upper triangular matrix) : aspect ratio (1 unless pixels are not square) : skew (0 unless pixels are shaped like rhombi/parallelograms) : principal point ((0,0) unless optical axis doesn’t intersect projection plane at origin)

Focal length Can think of as “zoom” Also related to field of view 24mm

Projection matrix intrinsics projection rotation translation (t in book’s notation)

Projection matrix = (in homogeneous image coordinates)

Questions?

Perspective distortion Problem for architectural photography: converging verticals Source: F. Durand

Perspective distortion Problem for architectural photography: converging verticals Solution: view camera (lens shifted w.r.t. film) Tilting the camera upwards results in converging verticals Keeping the camera level, with an ordinary lens, captures only the bottom portion of the building Shifting the lens upwards results in a picture of the entire subject http://en.wikipedia.org/wiki/Perspective_correction_lens Source: F. Durand

Perspective distortion Problem for architectural photography: converging verticals Result: Source: F. Durand

Perspective distortion What does a sphere project to? Image source: F. Durand

Perspective distortion The exterior columns appear bigger The distortion is not due to lens flaws Problem pointed out by Da Vinci Slide by F. Durand

Perspective distortion: People

Distortion Radial distortion of the image Caused by imperfect lenses No distortion Pin cushion Barrel Radial distortion of the image Caused by imperfect lenses Deviations are most noticeable for rays that pass through the edge of the lens

Correcting radial distortion from Helmut Dersch

Other types of projection Lots of intriguing variants… (I’ll just mention a few fun ones)

360 degree field of view… Basic approach Take a photo of a parabolic mirror with an orthographic lens (Nayar) Or buy one a lens from a variety of omnicam manufacturers… See http://www.cis.upenn.edu/~kostas/omni.html

Titlt-shift images from Olivo Barbieri and Photoshop imitations Tilt-shift http://www.northlight-images.co.uk/article_pages/tilt_and_shift_ts-e.html Titlt-shift images from Olivo Barbieri and Photoshop imitations

Rotating sensor (or object) Rollout Photographs © Justin Kerr http://research.famsi.org/kerrmaya.html Also known as “cyclographs”, “peripheral images”

Back to mosaics How to we align the images?

Creating a panorama Basic Procedure Take a sequence of images from the same position Rotate the camera about its optical center Compute transformation between second image and first Transform the second image to overlap with the first Blend the two together to create a mosaic If there are more images, repeat

Geometric interpretation of mosaics 2 1

Geometric Interpretation of Mosaics Image 2 Optical Center Image 1 If we capture all 360º of rays, we can create a 360º panorama The basic operation is projecting an image from one plane to another The projective transformation is scene-INDEPENDENT This depends on all the images having the same optical center

Image reprojection Basic question Answer PP2 PP1 How to relate two images from the same camera center? how to map a pixel from PP1 to PP2 PP2 Answer Cast a ray through each pixel in PP1 Draw the pixel where that ray intersects PP2 PP1

What is the transformation? Image 1 Image 2 Optical Center 3D ray coords (in camera 2) image coords (in image 2) 3D ray coords (in camera 1) image coords (in image 2) How do we transform image 2 onto image 1’s projection plane? image 1 image 2 image coords (in image 1) 3x3 homography image coords (in image 2)

Image alignment

Can we use homography to create a 360 panorama?