1.  Marc Levoy  Light Field = Array of (virtual) Cameras Sub-aperture Virtual Camera = Sub-aperture View

2. MERLMask-Enhanced Cameras: Heterodyned Light Fields & Coded Aperture Veeraraghavan, Raskar, Agrawal, Mohan & Tumblin Samples individual rays Predefined spectrum for lenses Chromatic abberration High alignment precision Peripheral pixels wasted pixels Negligible Light Loss Samples coded combination of rays Supports any wavelength Reconfigurable f/#, Easier alignment No wastage High resolution image for parts of scene in focus 50 % Light Loss due to mask Mask Sensor Microlens array Sensor Plenoptic CameraHeterodyne Camera

3. x θ l(x,θ) x θ x1x1 x2x2 θiθi θjθj θjθj x1x1 x2x2 x 1 ’ = x 1 + θ i *z Shear of Light Field θiθi θjθj x' 1 x1x1

4. Light Propagation (Defocus Blur) 1-D FFT 2-D FFT fxfx fθfθ L(f x,f θ ) Central Slice x θ l(x,θ) x θ Line Integral Captured Photo FFT of Captured Photo

5. Space of LF representations Time-frequency representations Phase space representations Quasi light field WDF Traditional light field Augmented LF Observable LF Rihaczek Distribution Function incoherent coherent Other LF representations

6. Quasi light fields the utility of light fields, the versatility of Maxwell We form coherent images by formulating, capturing, and integrating quasi light fields. WDF Traditiona l light field Augmente d LF Observable LF Rihaczek Distribution Function incoherent coherent Other LF representatio ns

7. (i) Observable Light Field move aperture across plane look at directional spread continuous form of plenoptic camera scene aperture position s direction u

8. (ii) Augmented Light Field with LF Transformer 8 WDF Light Field Augmented LF Interaction at the optical elements LF propagation (diffractive) optical element LF LF propagation light field transformer negative radiance

9. Virtual light projector with real valued (possibly negative radiance) along a ray 9 real projector first null (OPD = λ/2) virtual light projector

10. (ii) ALF with LF Transformer 10

11. (iii) Rihaczek Distribution Function Tradeoff between cross-interference terms and localization y u 0 m 3 m 0 m3 m y 0 m3 m y 0 m3 m u y (i) Spectrogram non-negative localization (ii) Wigner localization cross terms (iii) Rihaczek localization complex

12. Property of the Representation Constant along rays Non-negativityCoherenceWavelength Interference Cross term Traditional LF always constant always positive only incoherent zerono Observable LF nearly constant always positive any coherence state anyyes Augmented LF only in the paraxial region positive and negative any yes WDF only in the paraxial region positive and negative any yes Rihaczek DFno; linear driftcomplexany reduced

13. Benefits & Limitations of the Representation Ability to propagate Modeling wave optics Simplicity of computatio n Adaptability to current pipe line Near FieldFar Field Traditional LF x-shearnovery simplehighnoyes Observable LF not x-shearyesmodestlowyes Augmented LF x-shearyesmodesthighnoyes WDFx-shearyesmodestlowyes Rihaczek DF x-shearyes better than WDF, not as simple as LF lownoyes

14. Motivation What is the difference between a hologram and a lenticular screen?What is the difference between a hologram and a lenticular screen? How they capture ‘phase’ of a wavefront for telescope applications?How they capture ‘phase’ of a wavefront for telescope applications? What is ‘wavefront coding’ lens for extended depth of field imaging?What is ‘wavefront coding’ lens for extended depth of field imaging?

15. Application - Wavefront Coding Dowski and Cathey 1995 same aberrant blur regardless of depth of focus cubic phase plate point in scene small change in blur shape

16. Can they be part of Computer Vision? Moving away from 2D images or 4D lightfields? Wavefront coding: WLC mobile phone cameras Holography: Reference targets Rendering: New perspective projection methods Gaussian beam lasers: Modern active illumination Rotating PSF: Depth from defocus

17. Raskar, Camera Culture, MIT Media Lab Computational Photography 1.Epsilon Photography –Low-level Vision: Pixels –Multiphotos by bracketing (HDR, panorama) –‘Ultimate camera’ 2.Coded Photography –Mid-Level Cues: Regions, Edges, Motion, Direct/global –Single/few snapshot Reversible encoding of data, Lightfield –Additional sensors/optics/illum –‘Smart Camera’ 3.Essence Photography –Not mimic human eye –Beyond single view/illum –‘New artform’ http://computationalphotography.org

18. Resources WebsiteWebsite –http://scripts.mit.edu/~raskar/lightfields/ –Or follow http://cvpr2009.org tutorial pages Key new papersKey new papers Wigner Distributions and How They Relate to the Light Field Zhengyun Zhang and Marc Levoy, ICCP 2009 (best paper)Wigner Distributions and How They Relate to the Light Field Zhengyun Zhang and Marc Levoy, ICCP 2009 (best paper) Augmenting Light Field to Model Wave Optics Effects, Se Baek Oh, Barbastathis, Raskar (in Preparation)Augmenting Light Field to Model Wave Optics Effects, Se Baek Oh, Barbastathis, Raskar (in Preparation) Quasi light fields: extending the light field to coherent radiation, Anthony Accardi, Wornell (in Preparation)Quasi light fields: extending the light field to coherent radiation, Anthony Accardi, Wornell (in Preparation) WDF Traditional light field Augmented LF Observable LF Rihaczek Distribution Function

19. Acknowledgements DarthmuthDarthmuth –Marcus Testorf, MITMIT –Ankit Mohan, Ahmed Kirmani, Jaewon Kim –George Barbastathis StanfordStanford –Marc Levoy, Ren Ng, Andrew Adams AdobeAdobe –Todor Georgiev, MERLMERL –Ashok Veeraraghavan, Amit Agrawal

20. MIT Media Lab Camera Culture Ramesh Raskar MIT Media Lab http:// CameraCulture. info/ Light Fields___

21. Light Fields in Ray and Wave Optics Introduction to Light Fields: Ramesh Raskar Wigner Distribution Function to explain Light Fields: Zhengyun Zhang Augmenting LF to explain Wigner Distribution Function: Se Baek Oh Q&A Break Light Fields with Coherent Light: Anthony Accardi New Opportunities and Applications: Raskar and Oh Q&A: All