MIT Media Lab Camera Culture Image Destabilization: Programmable Defocus using Lens and Sensor Motion Ankit Mohan, Douglas Lanman, Shinsaku Hiura, Ramesh.

Slides:

Advertisements

Similar presentations

Elements of Exposure.

Advertisements

Chapter 4 Companion site for Light and Video Microscopy Author: Wayne.

Aperture, Exposure and Depth of Field

PHYSICS InClass by SSL Technologies with S. Lancione Exercise-53

Exposure Basics Introduction to Photography. What is Exposure  In photography, exposure is the total amount of light allowed to fall on the digital sensor.

Design of photographic lens Shinsaku Hiura Osaka University.

Camera Basics What is a photograph and why do we take pictures? What is a camera?

Lens  The lens is the most important part of the camera  Lens captures light and focuses the light on the part of the camera that receives the image.

The Camera : Computational Photography Alexei Efros, CMU, Fall 2006.

Raskar, Camera Culture, MIT Media Lab Camera Culture Ramesh Raskar Camera Culture MIT Media Lab Ramesh Raskar

Depth of Field. What the what?? Is Depth of Field.

Lesson 1: The Art and Physics of Photography Digital Photography MITSAA IAP 2003 Rob Zehner.

Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.

Linear View Synthesis Using a Dimensionality Gap Light Field Prior

1 Diffusion Coded Photography for Extended Depth of Field SIGGRAPH 2010 Oliver Cossairt, Changyin Zhou, Shree Nayar Columbia University Supported by ONR.

Compound lenses --- what’s the purpose? Correct bad optics with additional optics --- Hubble telescope, eyeglasses, etc. Correct or minimize aberations:

CS223b, Jana Kosecka Rigid Body Motion and Image Formation.

7. Optical instruments 1) Cameras

7. Optical instruments 1) Cameras

The Camera : Computational Photography Alexei Efros, CMU, Fall 2008.

The Ray Vector A light ray can be defined by two co-ordinates: x in,  in x out,  out its position, x its slope,  Optical axis optical ray x  These.

Light field photography and microscopy Marc Levoy Computer Science Department Stanford University.

... M A K E S Y O U R N E T W O R K S M A R T E R Lenses & Filters.

Evaluate. Review: Light Volume Control: 1. f stops are light transmission standards: f 8 transmits the same amount of light for any lens. 2.Lenses can.

6.098 Digital and Computational Photography Advanced Computational Photography Photography Survival Kit Bill Freeman Frédo Durand MIT - EECS.

Aperture and Depth of Field. APERTURE (F/STOP) 1.What is it? 2.Where is it? 3.What does it do? 4.When do you use it? 5.Why would you use F/2.8? 6.Why.

16421: Vision Sensors Lecture 2: Camera Obscura & View Camera

Multi-Aperture Photography Paul Green – MIT CSAIL Wenyang Sun – MERL Wojciech Matusik – MERL Frédo Durand – MIT CSAIL.

How the Camera Works ( both film and digital )

Northeastern University, Fall 2005 CSG242: Computational Photography Ramesh Raskar Mitsubishi Electric Research Labs Northeastern University September.

Lenses Why so many lenses and which one is right for me?

Depth from Diffusion Supported by ONR Changyin ZhouShree NayarOliver Cossairt Columbia University.

MERL, MIT Media Lab Reinterpretable Imager Agrawal, Veeraraghavan & Raskar Amit Agrawal, Ashok Veeraraghavan and Ramesh Raskar Mitsubishi Electric Research.

Cameras Course web page: vision.cis.udel.edu/cv March 22, 2003  Lecture 16.

Design of photographic lens Shinsaku Hiura Osaka University.

What does the Shutter do? Controls the amount of time light is allowed to strike the film (compare to Aperture- which controls the amount of light allowed.

1 CS6825: Image Formation How are images created. How are images created.

Introduction to Computational Photography. Computational Photography Digital Camera What is Computational Photography? Second breakthrough by IT First.

Fundamental of Optical Engineering Lecture 3.  Aberration happens when the rays do not converge to a point where it should be. We may distinguish the.

Photography Lesson 2 Pinhole Camera Lenses. The Pinhole Camera.

776 Computer Vision Jan-Michael Frahm Fall Camera.

Dynamically Reparameterized Light Fields Aaron Isaksen, Leonard McMillan (MIT), Steven Gortler (Harvard) Siggraph 2000 Presented by Orion Sky Lawlor cs497yzy.

Computational photography CS4670: Computer Vision Noah Snavely.

Recap from Wednesday Two strategies for realistic rendering capture real world data synthesize from bottom up Both have existed for 500 years. Both are.

Circular aperture Rectangular aperture Fraunhofer Diffraction.

Optics Real-time Rendering of Physically Based Optical Effects in Theory and Practice Masanori KAKIMOTO Tokyo University of Technology.

1 Finding depth. 2 Overview Depth from stereo Depth from structured light Depth from focus / defocus Laser rangefinders.

Tutorial on Computational Optical Imaging University of Minnesota September David J. Brady Duke University

Mitsubishi Electric Research Labs (MERL) Super-Res from Single Motion Blur PhotoAgrawal & Raskar Amit Agrawal and Ramesh Raskar Mitsubishi Electric Research.

CS348B Lecture 7Pat Hanrahan, 2005 Camera Simulation EffectCause Field of viewFilm size, stops and pupils Depth of field Aperture, focal length Motion.

Film/Sensor Where the light is recorded Lens Bends the light Trajectory of light Subject Source of light Focusing A look at the overall camera system.

 Marc Levoy  Light Field = Array of (virtual) Cameras Sub-aperture Virtual Camera = Sub-aperture View.

Lab 9: Geometrical optics Only 3 more labs to go!! Today we are going to do three experiments: 1. Measure the intensity of light as a function of distance.

Understanding Aperture (a beginner’s guide) Understanding Aperture (a beginner’s guide)

Mirrors. Types of mirror There are two types of mirror Plane (flat) Curved Concave (curves in) Convex (curves out)

Video Quality Lens Once you have selected a camera, the next step is to select the appropriate lenses and any other relevant components necessary in the.

Extended Depth of Field For Long Distance Biometrics

Media Production Richard Trombly Contact :

Aperture, Exposure and Depth of Field

Depth Of Field (DOF).

Camera Culture Camera Culture Ramesh Raskar Ramesh Raskar

Unit 57 – Photography Depth of field

Photographic Image Formation I

Depth Of Field.

Presentation transcript:

MIT Media Lab Camera Culture Image Destabilization: Programmable Defocus using Lens and Sensor Motion Ankit Mohan, Douglas Lanman, Shinsaku Hiura, Ramesh Raskar MIT Media Lab MIT Media Lab Camera Culture

Lots of glass; Heavy; Bulky; Expensive Defocus Blur

MIT Media Lab Camera Culture Image Destabilization Lens Sensor Camera Static Scene

MIT Media Lab Camera Culture Image Destabilization Static Scene Lens Motion Sensor Motion Camera

MIT Media Lab Camera Culture Related Work [Bae and Durand 2007] [Hasinoff and Kutulakos 2007] f8 f/4 f/2 f/1 extrapolated aperture [Hiura et al. 2009] [Vaish et al. 2004]

MIT Media Lab Camera Culture Laminography X-Ray Source X-Ray Sensor Plane of focus Related technique: Time Delay and Integration (TDI) Motion direction

MIT Media Lab Camera Culture A B A B Lens based Focusing LensSensor

MIT Media Lab Camera Culture A B A B Lens based Focusing LensSensor

MIT Media Lab Camera Culture A B B A Lens Smaller aperture Smaller defocus blur Sensor

MIT Media Lab Camera Culture A B A B Pinhole Pinhole: All In-Focus Sensor

MIT Media Lab Camera Culture A B A B Pinhole Shifting Pinhole Sensor vpvp

MIT Media Lab Camera Culture A B Pinhole A B Shifting Pinhole Sensor vpvp

MIT Media Lab Camera Culture A B Pinhole A B Shifting Pinhole Sensor vpvp

MIT Media Lab Camera Culture A B Pinhole A B Shifting Pinhole Sensor vpvp

MIT Media Lab Camera Culture B PinholeSensor A B Shifting Pinhole A dada dbdb dsds tptp vpvp

MIT Media Lab Camera Culture A B A B Pinhole Shifting Pinhole and Sensor vpvp Sensor vsvs dada dbdb dsds Focus Here

MIT Media Lab Camera Culture A B Pinhole A B vpvp Sensor vsvs dada dbdb dsds Focus Here Shifting Pinhole and Sensor

MIT Media Lab Camera Culture A B Pinhole A B vpvp Sensor vsvs dada dbdb dsds Focus Here Shifting Pinhole and Sensor

MIT Media Lab Camera Culture A B Pinhole A B vpvp Sensor vsvs dada dbdb dsds Focus Here Shifting Pinhole and Sensor

MIT Media Lab Camera Culture A Lens in Time! Analogous to shift and sum based Light field re-focusing. Lens Equation: Virtual Focal Length: Virtual F-Number:

MIT Media Lab Camera Culture Our Prototype

MIT Media Lab Camera Culture Adjusting the Focus Plane all-in-focus pinhole image

MIT Media Lab Camera Culture Adjusting the Focus Plane focused in the front using destabilization (10 second exposure)

MIT Media Lab Camera Culture Adjusting the Focus Plane focused in the middle using destabilization (5 second exposure)

MIT Media Lab Camera Culture Adjusting the Focus Plane focused in the back using destabilization (10 second exposure)

MIT Media Lab Camera Culture Adjusting the Virtual Aperture focused in the middle using destabilization (5mm pinhole translation)

MIT Media Lab Camera Culture Adjusting the Virtual Aperture focused in the middle using destabilization (30mm pinhole translation)

MIT Media Lab Camera Culture Shifting Lens and Sensor Defocus Physical vs. synthetic focusSimilar physical and synthetic focus aperture sensor real focus virtual focus Defocus Exaggeration

MIT Media Lab Camera Culture Defocus Exaggeration static lens with an f/2.8 aperture

MIT Media Lab Camera Culture Defocus Exaggeration destabilization simulates a reduced f-number

MIT Media Lab Camera Culture Defocus Invariance aperture sensor real focus virtual focus Differing physical and synthetic focus

MIT Media Lab Camera Culture * = real PSF virtual PSF overall PSF Defocus Invariance Related work –[Nagahara et al. 2008] –[Cathey and Dowski 1995] –[Levin et al. 2008] PSF not depth invariant –only size is depth invariant Gaussian special case –depth invariant PSF –inversion is ill-conditioned * = real PSF virtual PSF overall PSF [Nagahara et al. 2008]

MIT Media Lab Camera Culture Defocus Invariance depth-invariant blur size (horizontal slit + destabilization)

MIT Media Lab Camera Culture Defocus Invariance Richardson-Lucy deconvolution result

MIT Media Lab Camera Culture Tilted Sensor C D C D focus plane aperture planesensor plane Scheimpflug intersection dCdC dDdD dCdC dDdD

MIT Media Lab Camera Culture Tilted Sensor C D vs?vs? focus plane aperture planesensor plane vpvp dCdC dDdD

MIT Media Lab Camera Culture Tilted Sensor C D vpvp C D vsvs focus plane aperture planesensor plane dCdC dDdD dCdC dDdD

MIT Media Lab Camera Culture Tuning the PSF aperture sensor real focus pinhole image (static f/22 aperture)

MIT Media Lab Camera Culture Tuning the PSF real focus aperture sensor large aperture image (static f/2.8 aperture)

MIT Media Lab Camera Culture Tuning the PSF virtual focus aperture sensor destabilized image using a pinhole (translated f/22 aperture)

MIT Media Lab Camera Culture Tuning the PSF aperture sensor real focus virtual focus destabilized image using a large aperture (translated f/2.8 aperture)

MIT Media Lab Camera Culture Tuning the PSF real focus virtual focus aperture sensor simulated aspheric lens using a vertical slit aperture and destabilization

MIT Media Lab Camera Culture Extension to 2D Displacements elliptical linearcircular figure 8hypocycloidaltrispiral

MIT Media Lab Camera Culture Large apertures with tiny lenses? Benefits No time or light inefficiency wrt cheap cameras Exploits unused area around the lens Compact design With near-pinhole apertures (mobile phones) many possibilities Limitations Coordinated mechanical movement required Diffraction (due to small aperture) cannot be eliminated [Zhang and Levoy, tomorrow] [Our group: augmented LF for wave analysis] Scene motion during exposure

MIT Media Lab Camera Culture Acknowledgements Grace Woo MIT CSAIL Quinn Smithwick MIT Media Lab Gabriel Taubin Brown University Jaewon Kim MIT Media Lab MIT Media Lab: Camera Culture

MIT Media Lab Camera Culture SLRs with tiny lenses? Analysis of space of relative lens/sensor displacement Destabilization as virtual focusing mechanism Shallower depth of field than physical aperture Depth-independent defocus blur size