1. Depth from Diffusion Supported by ONR Changyin ZhouShree NayarOliver Cossairt Columbia University

2. Optical Diffuser

3. Micrograph of a Holographic Diffuser (RPC Photonics) ~ 10 micron [Gray, 1978] [Chang et al., 2006] [Garcia-Guerrero et al. 2007]

4. Diffusers as Accessories Diffuser to preview the image (B&H) Diffusers to soften the image Diffusers for illumination (B&H)

5. Camera Diffuser Object Camera Diffuser Object The amount of diffusion varies with depth. Diffusion Encodes Depth

6. Geometry of Diffusion: A Pinhole Camera Sensor Pinhole Object P Q Miss

7. Object P Geometry of Diffusion: A Pinhole Camera Sensor θ Pinhole Diffuser

8. Geometry of Diffusion: A Pinhole Camera Pinhole θ θ Sensor Diffuser Object P A B

9. Geometry of Diffusion: A Pinhole Camera O V U Z θ θ Pinhole Diffusion Law: Sensor A B Object 2r P Object Diffuser

10. Geometry of Diffusion: A Pinhole Camera O V U Z Pinhole Sensor A B Object 2r P Diffusion Size and Depth: Object Diffuser θ θ

11. Geometry of Diffusion: A Pinhole Camera O V U Pinhole Sensor Object 2r Diffuser as a proxy object Diffusion Size and Depth: P Z Diffuser

12. Diffusion as Convolution: A Pinhole Camera Latent clear image Captured Image Diffusion Size Diffusion PSF Assume field angle and depth are constant for small image patches, we have:

13. Geometry of Diffusion: A Lens Camera

14. O V U Pinhole Sensor Object 2r Diffuser as a proxy object P Z Diffuser

15. Geometry of Diffusion: A Lens Camera V U Sensor Object 2r Diffuser as a proxy object P Z Diffuser Lens The captured image can be further blurred due to defocus.

16. Diffusion as Convolution: A Lens Camera The Final PSF Defocus PSF Diffusion PSF For a lens camera with a diffuser, we have: is the diffusion PSF if a pinhole were used. is the defocus PSF if the diffuser were removed.

17. Depth from Diffusion (DFDiff) Algorithm Same form as in DFD 2. Estimate Blur Size r 3. Compute Depth Z 1. Capture Two Images With a diffuser Without a diffuser

18. Depth from Diffusion vs. Depth from Defocus [Pentland, 1987] [Subbarao, 1988] [Watanabe & Nayar, 1996] [Chaudhuri & Rajagopalan, 1999] [Favaro & Soatto, 2005] [Schechner & Kiryati, 2000] Depth from Defocus Aperture pattern P Lens Focal Plane Sensor Z r Depth from Diffusion Diffusion pattern Sensor Pinhole θ θ Diffuser Z P r

19. Depth from Diffusion vs. Depth from Defocus Depth from Diffusion P Any lens is fine! Suppose 22.5x15mm Sensor, 10 um pixel, 100 mm EFL Object distance = 1000 mm Object A Diffuser of 21.8 o Depth precision is about 0.1 mm. Field of View

20. Depth from Diffusion vs. Depth from Defocus Depth from Defocus P Lens Aperture diameter ? Suppose 22.5x15mm Sensor, 10 um pixel, 100 mm EFL Depth precision is about 0.1 mm. Object distance = 1000 mm Object Field of View

21. Depth from Diffusion vs. Depth from Defocus Depth from Defocus Object distance = 1000 mm Suppose 22.5x15mm Sensor, 10 um pixel, 100 mm EFL Depth precision is about 0.1 mm. Aperture diameter 800 mm Object P

22. Depth from Diffusion vs. Depth from Defocus Depth from Diffusion Suppose 22.5x15mm Sensor, 10 um pixel, 100 mm EFL P Depth precision is about 1.0 mm. Object distance = 5000 mm Object Any lens is fine! A Diffuser of 11.2 o

23. Depth from Diffusion vs. Depth from Defocus Depth from Defocus P Lens Aperture diameter ? Suppose 22.5x15mm Sensor, 10 um pixel, 100 mm EFL Depth precision is about 1.0 mm. Object distance = 5000 mm Object

24. Depth from Diffusion vs. Depth from Defocus Depth from Defocus Suppose 22.5x15mm Sensor, 10 um pixel, 100 mm EFL Depth precision is about 1.0 mm. Aperture diameter 2000 mm P Object distance = 5000 mm Object

25. PSF Measurement: A Pinhole Camera F/22, Field Angle = 0 o Z = 2 mm Z = 5 mm CapturedModeled - Canon EOS T1i; EF 50mm F/1.8 Lens; - Luminit Holographic Diffuser (10 o Gaussian ); - Diffuser distance: U = 1m

26. Z = 2 mm Z = 5 mm PSF Measurement: A Pinhole Camera CapturedModeled F/22, Field Angle = 10 o - Canon EOS T1i; EF 50mm F/1.8 Lens; - Luminit Holographic Diffuser (10 o Gaussian ); - Diffuser distance: U = 1m

27. PSF Measurement: A Lens Camera F/1.8, Field Angle = 10 o CapturedModeled - Canon EOS T1i; EF 50mm F/1.8 Lens; - Luminit Holographic Diffuser (10 o Gaussian ); - Diffuser distance: U = 1m

28. Experiments Five playing cards, 0.29mm thick each Canon 20D + 50mm Lens Luminit Diffuser (20 o )

29. Experiments Captured WITHOUT a DiffuserCaptured WITH a Diffuser

30. Experiments Computed Depth Map (~ 0.1 mm precision) (mm) Five playing cards, 0.29mm thick each

31. Experiments A small sculpture of about 4mm thickness Canon G5 Compact Camera Luminit Diffuser (5 o )

32. Experiments Captured WITHOUT a DiffuserCaptured WITH a Diffuser

33. Experiments Computed Depth Map A 3D View of Depth Map A small sculpture of about 4mm thickness

34. Experiments Canon 20D; Gaussian Diffuser (10 o ) 450 mm 650 mm

35. Experiments Stitched Depth Map (precision) (mm)

36. Summary Formulated the image formation with optical diffusers Formulated the image formation with optical diffusers Proposed Depth from Diffusion Proposed Depth from Diffusion - Require a diffuser on the object side + High-precision depth estimation + Distant objects + Less sensitive to lens aberrations Demonstrated high-precision depth estimation Demonstrated high-precision depth estimation Camera Diffuser Object

37. Depth from Diffusion Supported by ONR Changyin ZhouShree NayarOliver Cossairt Columbia University