1. Introduction to Computational Photography

2. Computational Photography Digital Camera What is Computational Photography? Second breakthrough by IT First : electronic image sensor (digital camera) Digital representation of “image formed by lens” Second : Re-definition of whole camera (optics, usage) Image is reconstructed by computation Image processing Image Image sensor Optics Whole part of camera is affected by computational photography Film camera Digital Camera

3. Light field (Light space) What is camera? Camera is a machine to record the distribution of the light in a scene How to represent the distribution of the light in the scene? 3-D coordinate of the point where the light passing through : X, Y, Z Direction of the light : θ，Φ Wavelength of the light (color of the light) : λ Time : t The 7 parameters function P which represent the distribution of the light is called “Plenoptic function” Light source Object

4. Light field (light ray)Optics (lens)Image sensor (pixel) Integration of camera Camera integrates the light for all 7 parameters Position（range of X, Y, Z : aperture size should not be zero) Direction（range of θ，Φ : pixel size is not zero) Wavelength（ range of λ : No single wavelength filter) Exposure time ( range of t : shutter speed should not be too fast) Multiple samples - θ,Φ:number of pixel, λ：RGB，t : burst shot So, what is multiple sampling for X, Y, Z?

5. Camera array Measuring the distribution of the light at multiple position ProFUSION25 (ViewPlus, Inc.) The Stanford Multi-Camera Array (Marc Levoy @Stanford University)

6. Use of camera array Free-viewpoint image Defocus generation by synthetic aperture 3-D video (Matsuyama lab, Kyoto Univ.)Synthetic aperture (Vaish@Stanford)Vaish@Stanford

7. Defocus control by Uncalibrated Synthetic Aperture Natsumi Kusumoto, Shinsaku Hiura and Kosuke Sato, Uncalibrated Synthetic Aperture for Defocus Control, CVPR2009 (Jun. 2009)

8. Reviewing “integration” Some part of information is lost by integration Sine wave which period is just as same as the integration duration is lost Blur of object within an exposure time Defocus by misfocus × = 0

9. Coded Exposure Coded exposure : exposure is coded in time axis Flutter Shutter Camera (Raskar@MERL)

10. This Traditional Coded Exposure Image of Static Object Deblurred Image Slide by R. Raskar

11. Coded Exposure Temporal 1-D broadband code: Motion Deblurring Coded Aperture Spatial 2-D broadband mask: Focus Deblurring Slide by R. Raskar

12. Captured Blurred Photo Slide by R. Raskar

13. Refocused on Person Slide by R. Raskar

14. Coded Aperture Levin@MIT(2007) Depth estimation by single image (manual operation is necessary)

15. Coded Aperture Levin@MIT(2007)

16. Coded Aperture Levin@MIT(2007)

17. Multi-focus camera with Coded Aperture Stabilizing the depth estimation and deblur by coded aperture Simultaneous capture of 3 images with different focused distance Hiura et al, CVPR(1998), SSII(1999)

18. Multi-Focus Range Sensor using Coded Aperture

19. Invariant integration Defocus : changed according to the distance Blur : changed according to the speed of the object Reconstruction is not easy because the estimation of the speed or distance is necessary  Is it possible to make defocus or blur invariant to the distance or speed?

20. Invariant integration Defocus Special optics : Wavefront Coding Motion of the image sensor while exposure Blur Reciprocal motion of the camera CDM Optics, Inc.

21. Motion of the image sensor for invariant defocus H. Nagahara, S. Kuthirummal, C. Zhou, and S.K. Nayar, Flexible Depth of Field Photography, ECCV2008 H. Nagahara, S. Kuthirummal, C. Zhou, and S.K. Nayar, Flexible Depth of Field Photography, ECCV2008

22. H. Nagahara, S. Kuthirummal, C. Zhou, and S.K. Nayar, Flexible Depth of Field Photography, ECCV2008 Motion of the image sensor for invariant defocus

23. Deblur by reciprocal motion of the camera A. Levin, P. Sand, T. S. Cho, F. Durand, W. T. Freeman. Motion- Invariant Photography. SIGGRAPH2008. A. Levin, P. Sand, T. S. Cho, F. Durand, W. T. Freeman. Motion- Invariant Photography. SIGGRAPH2008. Input imageDeblurred image

24. A. Levin, P. Sand, T. S. Cho, F. Durand, W. T. Freeman. Motion- Invariant Photography. SIGGRAPH2008. EquipmentConceptual figure for Light sources with different speed Deblur by reciprocal motion of the camera

25. More.. Resources on www Wikipedia : computational photographycomputational photography http://computationalphotography.org/ http://www1.cs.columbia.edu/CAVE/projects/what_is/ http://projects.csail.mit.edu/photo/ Conferences International Conference on Computational Photography SIGGRAPH, CVPR,.. Session about computational photography