1 Diffusion Coded Photography for Extended Depth of Field SIGGRAPH 2010 Oliver Cossairt, Changyin Zhou, Shree Nayar Columbia University Supported by ONR and NSF

2 Conventional Camera (F/1.8)

3 Conventional Camera (F/18)

4 Focused Image PSF Image Noise Captured Image Camera Blur Model Spatial domain Focused ImageMTFImage NoiseCaptured Image Frequency domain (Modulated Transfer Function)

5 Deblurring Problems MTFCaptured image Low SNR Low MTF values Problem 1: Focused image Lens Sensor P Object Q Problem 2: Variation with depth

6 Extending Depth of Field (EDOF): Previous Work Focal Sweep Cameras [Hausler ’72] [Nagahara et al. ’08] LensFocal PlaneSensor Other Related Work [Levin et al. ’07] [Veeraraghavan et al. ’07] [Levin et al. ’09] Wavefront Coding Cameras [Dowski and Cathey ’95] [Chi and George ’01] [Garcia-Guerrero et al. ‘07] Lens Sensor Cubic Phase Plate

7 Focal sweep vs. wavefront coding Focal Sweep Wavefront Coding Conventional Camera near far focus depth Note: only a single PSF will be used to deblur the whole image.

8 Deblurring Error vs. Depth Deblurring Error noise Depth Wavefront Coding Focal Sweep Focal sweep vs. wavefront coding

9 Achieve the performance of focal sweep without any moving parts?

10 Optical Diffusers Diffuser [ Diffuser sheets w SensorDiffuser Light ray x x Scatter function w SEM image

11 Diffuser Kernels x u A/2 -A/2 u x Light Field Without a diffuser: Lens A Sensor

12 Diffuser Kernels A/2 -A/2 u x Light Field Without a diffuser: With a diffuser: A/2 -A/2 u x x u Lens Sensor x u Lens Sensor A A

13 Diffuser Kernels A/2 -A/2 u x Light Field Without a diffuser: With a diffuser: u x A/2 -A/2 x u Lens Sensor x u Lens Sensor A A

14 Diffusion Kernels A/2 -A/2 u x Without diffuser u x With diffuser u x Diffuser kernel Light field

15 A/2 -A/2 u x u x u x Light field Diffusion Kernels x project x x PSF Diffused PSF Scatter function Camera PSF Without diffuser With diffuserDiffuser kernel

16 Radially Symmetric Diffuser Lens Sensor PSF Without a diffuser:

17 Without a diffuser: Lens Sensor With a diffuser: Lens Sensor PSF Radially Symmetric Diffuser

18 Lens Sensor With a diffuser: Lens Sensor PSF Without a diffuser: Radially Symmetric Diffuser

19 Lens Sensor Lens Sensor PSF With a diffuser: Without a diffuser: Radially Symmetric Diffuser

20 Lens Sensor With a diffuser: Radially Symmetric Diffuser Lens Sensor PSF Without a diffuser:

21 Lens Sensor Lens Sensor PSF With a diffuser: Without a diffuser: Radially Symmetric Diffuser

22 Lens Sensor Lens Sensor PSF With a diffuser: Without a diffuser: Radially Symmetric Diffuser

23 Radially Symmetric Diffuser PSF: Scatter function Camera PSF (1D slice) MTF (1D slice) Normalized frequency -50px50px-50px50px-50px50px-50px50px depth

24 Diffusion Coding Performance Deblurring Error vs. Depth Depth Wavefront Coding Focus Sweep noise Diffusion Coding (light field) Diffusion Coding (wave optics) Similar performance to focal sweep without moving parts Deblurring Error

25 Diffuser Implementation Diffuser scatter function r (mm) [ Diffuser height map Fabricated diffuser 110 Thickness (um) r (mm) Diffuser surface profile RPC Photonics

26 Garcia-Guerrero Comparison with Prior Work Deblurring Error vs. Depth Depth Diffusion Coding Diffusion coding significantly outperforms prior work. Deblurring Error x

27 Diffusion Coding Experiments Fabricated DiffuserCannon 50mm EF lensCannon 450D Sensor Experimental Setup Measured PSFs depth Without diffuser With diffuser

28 Examples

29 Stuffed Toys f-number = 1.8, exposure time = 16ms Conventional Camera

30 Stuffed Toys f-number = 18, exposure time = 16ms Conventional Camera

31 Stuffed Toys Diffusion Coding Camera: Captured f-number = 1.8, exposure time = 16ms

32 Stuffed Toys f-number = 1.8, exposure time = 16ms Diffusion Coding Camera: Deblurred

33 Statues f-number = 1.8; exposure time = 10ms Captured Deblurred

34 People and Flowers f-number = 1.8, exposure time = 16ms Conventional Camera

35 People and Flowers f-number = 1.8, exposure time = 16ms Diffusion Coding Camera: Captured

36 People and Flowers f-number = 1.8, exposure time = 16ms Diffusion Coding Camera: Deblurred

37 Limitations Conventional CameraDiffusion Coding Loss of image texture Loss of contrast Occlusion errors

38 Conclusions Diffusion Coding Theory Diffusion Coding ImplementationDiffusion Coding Examples Radially Symmetric Diffusers

39 Diffusion Coded Photography for Extended Depth of Field SIGGRAPH 2010 Oliver Cossairt, Changyin Zhou, Shree Nayar Columbia University Supported by ONR and NSF