Slides:



Advertisements
Similar presentations
Chapter 4 Companion site for Light and Video Microscopy Author: Wayne.
Advertisements

MIT Media Lab Camera Culture Image Destabilization: Programmable Defocus using Lens and Sensor Motion Ankit Mohan, Douglas Lanman, Shinsaku Hiura, Ramesh.
S INGLE -I MAGE R EFOCUSING AND D EFOCUSING Wei Zhang, Nember, IEEE, and Wai-Kuen Cham, Senior Member, IEEE.
The Camera : Computational Photography Alexei Efros, CMU, Fall 2006.
Foundations of Physics
Chapter 31 Images.
In all cameras, light enters through the lens and hits the recording medium. (In film cameras, the film plane, in digital cameras (for our purposes),
SLAC National Accelerator Center
Depth of Field. What the what?? Is Depth of Field.
© 2005 Pearson Prentice Hall This work is protected by United States copyright laws and is provided solely for the use of instructors in teaching their.
Digital Photography Part 1 A crash course in optics.
Announcements No lab tonight due to Dark Sky Observing Night last night Homework: Chapter 6 # 1, 2, 3, 4, 5 & 6 First Quarter Observing Night next Wednesday.
Aperture Pupil (stop) Exit Pupil Entrance Pupil.
Chris A. Mack, Fundamental Principles of Optical Lithography, (c) Figure 3.1 Examples of typical aberrations of construction.
3. Geometrical Optics. Geometric optics—process of light ray through lenses and mirrors to determine the location and size of the image from a given object.
Image Formation and Optics
Imaging Science FundamentalsChester F. Carlson Center for Imaging Science Classical Photography and Geometric Optics Imaging Science Fundamentals.
Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.
What are Good Apertures for Defocus Deblurring? Columbia University ICCP 2009, San Francisco Changyin Zhou Shree K. Nayar.
Wide-field, triple spectrograph with R=5000 for a fast 22 m telescope Roger Angel, Steward Observatory 1 st draft, December 4, 2002 Summary This wide-field,
Fiber Optics Defining Characteristics: Numerical Aperture Spectral Transmission Diameter.
CCU VISION LABORATORY Object Speed Measurements Using Motion Blurred Images 林惠勇 中正大學電機系
Interactive Matting Christoph Rhemann Supervised by: Margrit Gelautz and Carsten Rother.
Linear View Synthesis Using a Dimensionality Gap Light Field Prior
1 Diffusion Coded Photography for Extended Depth of Field SIGGRAPH 2010 Oliver Cossairt, Changyin Zhou, Shree Nayar Columbia University Supported by ONR.
Design Realization lecture 25 John Canny 11/20/03.
7. Optical instruments 1) Cameras
The Camera : Computational Photography Alexei Efros, CMU, Fall 2008.
Multispectral Camera Simon Belkin, Audrey Finken, Grant George, & Matthew Walczak Comprehensive Design Review Team Parente.
Imaging Science FundamentalsChester F. Carlson Center for Imaging Science Mirrors and Lenses.
Basic Principles of Imaging and Photometry Lecture #2 Thanks to Shree Nayar, Ravi Ramamoorthi, Pat Hanrahan.
Multi-Aperture Photography Paul Green – MIT CSAIL Wenyang Sun – MERL Wojciech Matusik – MERL Frédo Durand – MIT CSAIL.
Northeastern University, Fall 2005 CSG242: Computational Photography Ramesh Raskar Mitsubishi Electric Research Labs Northeastern University September.
Depth from Diffusion Supported by ONR Changyin ZhouShree NayarOliver Cossairt Columbia University.
Cameras Course web page: vision.cis.udel.edu/cv March 22, 2003  Lecture 16.
Design of photographic lens Shinsaku Hiura Osaka University.
Optical Design Work for a Laser-Fiber Scanned
Chapter 12 Optical Instruments Physics Beyond 2000.
1 CS6825: Image Formation How are images created. How are images created.
Introduction to Computational Photography. Computational Photography Digital Camera What is Computational Photography? Second breakthrough by IT First.
SPECTROSCOPIC DIAGNOSTIC COMPLEX FOR STUDYING PULSED TOKAMAK PLASMA Yu. Golubovskii, Yu. Ionikh, A. Mestchanov, V. Milenin, I. Porokhova, N. Timofeev Saint-Petersburg.
01/28/05© 2005 University of Wisconsin Last Time Improving Monte Carlo Efficiency.
Magnifiers, Projectors, CamerasPaul Avery (PHY 3400)1 Magnifiers, Projectors, Cameras Applied Optics Paul Avery University of Florida
Recap from Wednesday Two strategies for realistic rendering capture real world data synthesize from bottom up Both have existed for 500 years. Both are.
Lecture Exposure/histograms. Exposure - Four Factors A camera is just a box with a hole in it. The correct exposure is determined by four factors: 1.
Optics Real-time Rendering of Physically Based Optical Effects in Theory and Practice Masanori KAKIMOTO Tokyo University of Technology.
Computational Photography
1 Finding depth. 2 Overview Depth from stereo Depth from structured light Depth from focus / defocus Laser rangefinders.
Tutorial on Computational Optical Imaging University of Minnesota September David J. Brady Duke University
Eusoballoon optics test Baptiste Mot, Gilles Roudil, Camille Catalano, Peter von Ballmoos Test configuration Calibration of the light beam Exploration.
1 32 Optical Images image formation reflection & refraction mirror & lens equations Human eye Spherical aberration Chromatic aberration.
WFIRST IFU -- Preliminary “existence proof” Qian Gong & Dave Content GSFC optics branch, Code 551.
1. What is depth of field? 2. Everything else equal, what effect will each of the following have on depth of field (larger, smaller?): -Larger aperture.
Telescopes. Light Hitting a Telescope Mirror huge mirror near a star * * * small mirror far from 2 stars In the second case (reality), light rays from.
Thin-lens equation: 1/f = 1/d 0 + 1/d i. Magnification equation: h i /h o = d i /d o.
Extracting Depth and Matte using a Color-Filtered Aperture Yosuke Bando TOSHIBA + The University of Tokyo Bing-Yu Chen National Taiwan University Tomoyuki.
The law of reflection: The law of refraction: Image formation
Prof. Charles A. DiMarzio Northeastern University Fall 2003 July 2003
Camera LENSES, APERTURE AND DEPTH OF FIELD. Camera Lenses Wide angle lenses distort the image so that extreme wide angle can look like its convex such.
Image Formation III Chapter 1 (Forsyth&Ponce) Cameras “Lenses” Guido Gerig CS-GY 6643, Spring 2016 (slides modified from Marc Pollefeys, UNC Chapel Hill/
Today Defocus Deconvolution / inverse filters. Defocus.
Refraction & Lenses. Refraction of Light When a ray of light traveling through a transparent medium encounters a boundary leading into another transparent.
Extended Depth of Field For Long Distance Biometrics
7. Optical instruments 1) Cameras
The Camera : Computational Photography
Depth of Field Objective: to photograph a subject with a foreground and background, giving a better understanding of aperture and its relation to depth.
Deconvolution , , Computational Photography
Exposing Digital Forgeries Through Chromatic Aberration Micah K
The Camera : Computational Photography
Chaper 6 Image Quality of Optical System
Presentation transcript:

Spectral Focal Sweep (SFS): Extended Depth of Field from Chromatic Aberrations ICCP 2010 Ollie Cossairt, Shree Nayar Columbia University

Conventional Camera (F/4) BLURRY Spectral Focal Sweep: ICCP 2010

Conventional Camera (F/16) NOISY Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 EDOF Camera (F/4) CAPTURED Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 EDOF Camera (F/4) DEBLURRED Spectral Focal Sweep: ICCP 2010

Conventional Camera (F/16) NOISY Spectral Focal Sweep: ICCP 2010

Conventional Camera (F/4) BLURRY Spectral Focal Sweep: ICCP 2010

Conventional Camera (F/16) NOISY Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 EDOF Camera (F/4) CODED Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 EDOF Camera (F/4) DEBLURRED Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 Related Work Coded Aperture [Welford ’60] [Levin et al. ‘07] [Veeraraghavan et al. ’07] [Zhou and Nayar ’09] Phase Plates [Dowski and Cathey ’95] [Chi and George ’01] [Levin et al. ’09] Focus Sweep [Nagahara et al. ’08] [Hausler ’72] Lens Focal Plane Sensor Color [Guichard et al. ’09] Lens Focal Planes RGB Sensor Red Green Blue Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep Idea Broadband reflectance spectra Poorly Corrected Lens + B&W Sensor + Focus Sweep Lens Focal Plane Sensor = No moving parts or custom elements Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep Idea Broadband reflectance spectra Works exactly for black and white scenes Many naturally occurring reflectance spectra are also broadband [Parkkinen ‘89] Poorly corrected lens Simpler lens design optimization Black & White Sensor Can also approximate with luminance channel from color sensor Spectral Focal Sweep: ICCP 2010

Conventional Focus Sweep PSF u = object depth (fixed) v = sensor distance (varying) Instantaneous PSF Lens EFL = f Focal Plane Sensor v1 u Scene Depth Range Spectral Focal Sweep: ICCP 2010

Conventional Focus Sweep PSF u = object depth (fixed) v = sensor distance (varying) Instantaneous PSF Lens EFL = f Focal Plane Sensor d2 v2 u Scene Depth Range Spectral Focal Sweep: ICCP 2010

Conventional Focus Sweep PSF u = object depth (fixed) v = sensor distance (varying) Instantaneous PSF Lens EFL = f Focal Plane Sensor d3 v3 u Scene Depth Range Spectral Focal Sweep: ICCP 2010

Conventional Focus Sweep PSF Defocus disk diameter (A = aperture diameter) Integrate over time to get final PSF Circ/Top-hat function Varying either object/sensor distance, or focal length will produce the same PSF Spectral Focal Sweep: ICCP 2010

Refractive Dispersion (prism) Snell’s Law of refraction Refraction depends on wavelength Spectral Focal Sweep: ICCP 2010

Refractive Dispersion (lens) For an uncorrected lens, Focal length depends on wavelength R1 , R2 = lens radii of curvature u v Sensor PSF Focal Planes Scene Depth Range Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep PSF Each wavelength produces a different defocus disk A B&W Sensor integrates over wavelength, weighted by the spectral reflectance of the scene For white objects, a SFS PSF is identical to a mechanical focal sweep PSF Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 SFS Lens Design Design Criteria Wavelength range 400-700nm 75mm EFL F/4 1/3” Sensor, 10um pixel size Optimization Criteria Zemax Optical Design Software Maximize chromatic defocus across wavelength range Minimize spot size at 550nm Fit to off-the-shelf lenses Maximum defocus diameter 100um at 400nm and 700nm Spectral Focal Sweep: ICCP 2010

Zemax SFS PSFs (white spectrum) Corrected lens SFS Lens field 15x depth Spectral Focal Sweep: ICCP 2010

Zemax SFS PSFs (white spectrum) Corrected lens field SFS Lens 1 15 x depth Spectral Focal Sweep: ICCP 2010

Simulating Real Spectra Munsell database of 1250 measured reflectance spectra Zemax simulation of SFS PSF at 12 depth locations (1.5m-3m, 100um max defocus dia.) For each Munsell spectrum Calculate average deblurring error when a white spectrum is used for deblurring Spectral Focal Sweep: ICCP 2010

Average Deblurring Error Input Image: Munsell color PSFs: White PSF at center depth: PSF Distance Metric: [Zhou and Nayar ‘09] Average Deblurring Error: Averaged over many images Spectral Focal Sweep: ICCP 2010

Munsell Spectra PSFs Most Munsell spectra show little PSF variation Corrected lens 1 15 x depth Most Munsell spectra show little PSF variation Spectral Focal Sweep: ICCP 2010

Average Deblurring Error (%95 of Munsell spectra) Most Munsell PSFs introduce negligible deblurring artifacts Spectral Focal Sweep: ICCP 2010

Doublet SFS Implementation Zemax Design Implementation Edmund Optics #48184 1” Dia. 100mm FL Plano-convex asphere Thorlabs #LE1929 1” Dia. 300mm FL positive meniscus Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 Experimental Setup Corrected Lens SFS Doublet Lens B&W Examples (Basler 1/3” sensor) F/4 SFS 75mm EFL Doublet EFL Cosmicar F/1.4 75mm Color Examples (Canon 450D sensor) F/4 SFS 75mm EFL Doublet F/2.8 100mm EFL Cannon Spectral Focal Sweep: ICCP 2010

Measured SFS PSFs (white spectrum) depth Cosmicar SFS lens Measured SFS PSF approximately depth-invariant Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 Deblurring Deblur with measured white PSF B&W Images Wiener deconvolution Color Images Transform from RGB to YUV space Wiener deconvolution of Y image Transform back to RGB space Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 EDOF Examples Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 F/4 Corrected Lens 8ms exposure Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 F/4 SFS Camera (Captured) 8ms exposure Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 F/4 SFS Camera (Deblurred) 8ms exposure Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 F/4 Corrected Lens Background Foreground 8ms exposure Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 F/4 SFS Lens (Captured) 8ms exposure Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 F/4 SFS Lens (Deblurred) 8ms exposure Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 F/4 Corrected Lens Foreground Background 16ms exposure Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 F/4 SFS Lens (Captured) 16ms exposure Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 F/4 SFS Lens (Deblurred) 16ms exposure Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 F/4 Corrected Lens Background Foreground 16ms exposure Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 F/4 SFS Lens (Captured) 16ms exposure Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 F/4 SFS Lens (Deblurred) 16ms exposure Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 F/4 Corrected Lens Background Foreground 16ms exposure Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 F/4 SFS Lens (Captured) 16ms exposure Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 F/4 SFS Lens (Deblurred) 16ms exposure Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 F/4 Corrected Lens Background Foreground 16ms exposure Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 F/4 SFS Lens (Captured) 16ms exposure Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 F/4 SFS Lens (Deblurred) 16ms exposure Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 Limitations Broadband assumption does not always hold (e.g. %5 of Munsell Spectra) deblurring artifacts introduced Color deblurring method does not correct blur in chrominance channels Spectral Focal Sweep: ICCP 2010

Spectral Focal Sweep: ICCP 2010 Conclusion Broadband spectra Chromatic aberrations B&W sensor Focus sweep + = Works exactly for B&W scenes Works well for many naturally occurring spectra Results for color images “look” good Easy to build EDOF camera Spectral Focal Sweep: ICCP 2010