Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University

Conventional Camera (F/1.8)

Focused Image PSFImage Noise Captured Image Camera Blur Model Spatial domain Focused ImageMTF Image Noise Captured Image Frequency domain

Deblurring Problems MTFCaptured image Low SNR low MTF values Problem 1: Focused image Lens Sensor P Object Q Problem 2: Variation with depth

Extending Depth of Field: Previous Work Focus Sweep Cameras [Hausler ’72] [Nagahara et al. ’08] LensFocal PlaneSensor Wavefront Coding Cameras [Dowski and Cathey ’95] [Chi and George ’01] [Garcia-Guerrero et al. ‘07] Other Related Work [Levin et al. ’07] [Veeraraghavan et al. ’07] [Levin et al. ’09]

Focus Sweep Camera Lens Scene Sensor = Instantaneous PSF t = 1 t = 2t = 3t = 4t = 5t = 6t = 7 Final PSF [Hausler ’72] [Nagahara et al. ’08]

Focus Sweep Camera = Instantaneous PSF t = 1t = 2t = 3t = 4t = 5t = 6t = 7 Final PSF Instantaneous PSF t = 1 + t = 2 + t = 3 + t = 4t = 5 + t = 6 + t = 7 + depth 1 depth 2 [Levin et al. ’09] 2D MTF = Final PSF Lens Scene Sensor [Hausler ’72] [Nagahara et al. ’08]

Wavefront Coding Lens Scene Sensor [Levin et al. ’09] 2D MTF Cubic Phase Plate xu [Dowski and Cathey ’95] Ambiguity Function slice MTF

Lens Resolution Target Sensor Focus Sweep Wavefront Coding EDOF Camera Comparison depth

Focus Sweep Wavefront Coding Deblurred image EDOF Camera Comparison

Deblurring Error vs. Depth Deblurring Error noise Deblurring Error Depth Wavefront Coding Focus Sweep Wavefront Coding

Is it possible to achieve the performance of focus sweep without moving parts?

Optical Diffusers Circular diffuser [ Diffuser sheets w SensorDiffuser Light ray x x Scatter function w SEM image

Diffuser Kernels LensSensor x u A u x A/2 -A/2 With diffuser A/2 -A/2 u x Without diffuser Light field space

w w A/2 -A/2 u x Without diffuser Diffuser Kernels LensSensor w u x With diffuser w Light field space x u

Diffuser Kernels LensSensor w u x A/2 -A/2 u x u x Diffuser kernel Light fieldDiffuser kernelCoded light field x u Without diffuserWith diffuser Light field space

Diffusion Coded PSF A/2 -A/2 u x Without diffuser u x With diffuser u x Diffuser kernel x project x Light field space x project Sensor space Camera PSFScatter functionCoded PSF

Radially Symmetric Light Field For an on-axis, isotropic point source:

Radially Symmetric Diffuser For a radially-symmetric diffuser kernel

Radially Symmetric Diffuser PSFs Radially symmetric diffuser Coded PSFScatter functionCamera PSF Coded PSFScatter functionCamera PSF Conventional diffuser Normalized frequency PSF Vs. Depth MTF Vs. Depth -50px50px-50px50px-50px50px-50px50px depth

Diffusion Coding Performance Deblurring Error vs. Depth Depth Wavefront Coding Focus Sweep noise Diffusion Coding (light field) Diffusion Coding (wave optics) Similar performance to focus sweep without moving parts

Diffuser Implementation Diffuser scatter function r (mm) 110 Thickness (um) r (mm) [ Diffuser heightmapFabricated Diffuser Diffuser surface profile [Sales et al. ‘03]

Garcia-Guerrero Comparison with Prior Work Deblurring Error vs. Depth Depth Diffusion Coding Diffusion coding significantly outperforms prior work

Diffusion Coding Experiments Fabricated DiffuserCannon 50mm EF lensCannon 450D Sensor Experimental Setup Measured PSFs depth Without diffuser with diffuser BM3D Deblurring Algorithm [Dabov et al. ‘08]

Examples

Conventional Camera f-number = 1.8, exposure time = 16ms

Conventional Camera f-number = 18, exposure time = 16ms

Diffusion Coding Captured f-number = 1.8, exposure time = 16ms

Diffusion Coding Deblurred f-number = 1.8, exposure time = 16ms

Conventional Camera f-number = 1.8 exposure time = 10ms

Diffusion Coding f-number = 1.8 exposure time = 10ms Captured

Diffusion Coding f-number = 1.8 exposure time = 10ms Deblurred

Conventional Camera f-number = 1.8 exposure time = 12.5ms

Diffusion Coding f-number = 1.8 exposure time = 12.5ms Captured

Diffusion Coding f-number = 1.8 exposure time = 12.5ms Deblurred

Conventional Camera f-number = 1.8, exposure time = 16ms

Diffusion Coding Captured f-number = 1.8, exposure time = 16ms

Diffusion Coding Deblurred f-number = 1.8, exposure time = 16ms

Limitations Conventional CameraDiffusion Coding Loss of image texture Loss of contrast Occlusion errors

Conclusions Diffusion Coding Theory Diffusion Coding Examples Radially Symmetric Diffusers Lens Sensor Diffuser Diffusion Coding Implementation