1. Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Lighting

2. Computer Graphics Inf4/MSc 13/11/20072 Global Illumination and Image-Based Lighting Traditional Computer Graphics involves: –Modelling with matter: geometry with reflectance properties. Image-Based Lighting allows: –Combination of real and synthetic graphics with consistent illumination, using images as light sources. –Extension of Environment mapping

3. Computer Graphics Inf4/MSc Environment Mapping Computing the color according to the reflection vector No self-shadowing, no inter-reflection

4. Computer Graphics Inf4/MSc 13/11/20074 Image Based Lighting Add models or objects to scenes and allow them to be manipulated in the scenes. Modelling with light allows added objects to be illuminated consistent with image existent lighting - photorealistic inclusions. We start with the answer by finding the scene illumination.

5. Computer Graphics Inf4/MSc 13/11/20075 Image Based Lighting Real Scene Goal: place synthetic objects on table (all pictures P. Debevec 98-99)‏

6. Computer Graphics Inf4/MSc 13/11/20076 Extracting scene lighting Capture illumination using illumination sphere

7. Computer Graphics Inf4/MSc 13/11/20077 Image Based Lighting Real scene

8. Computer Graphics Inf4/MSc 13/11/20078 Image Based Lighting captured illumination field

9. Computer Graphics Inf4/MSc 13/11/20079 Image Based Lighting Real scene light based model local scene synthetic objects

10. Computer Graphics Inf4/MSc 13/11/200710 Image Based Lighting Use global illumination - compute effects of synthetic objects on local scene local scene (brdf estimated)‏synthetic objects (brdf known)‏ light based model

11. Computer Graphics Inf4/MSc 13/11/200711 Image Based Lighting Render into the scene background

12. Computer Graphics Inf4/MSc 13/11/200712 Image Based Lighting Render synthetic objects

13. Computer Graphics Inf4/MSc 13/11/200713 Image Based Lighting Effect of local scene on real scene

14. Computer Graphics Inf4/MSc 13/11/200714 Image Based Lighting Add differences to image

15. Computer Graphics Inf4/MSc Some more results

16. Computer Graphics Inf4/MSc Light Probe Images Spherical Environment Maps In most digital images, pixel values aren’t proportional to the light levels in the scene. –light levels are encoded nonlinearly so they appear either more correctly or more pleasingly on CRTs

17. Computer Graphics Inf4/MSc High-Dynamic Range Photography Standard digital images typically represent only a small fraction of the dynamic range—the ratio between the dimmest and brightest regions accurately represented— present in most realworld lighting environments. The bright light saturates the pixel colour Need a high dynamic range image which can be produced by combining images of different shutter speed

18. Computer Graphics Inf4/MSc High-Dynamic Range Photography

19. Computer Graphics Inf4/MSc HDR images

20. Computer Graphics Inf4/MSc

21. Comparison: Radiance map versus single image

22. Computer Graphics Inf4/MSc Eyes For Relighting Extracting environment maps from eyes for use in integrating and relighting scenes Apply IBL to the 3D objects added into the image http://www.youtube.com/watch?v=MllTSofXt8E

23. Computer Graphics Inf4/MSc Shape from Shading Obtaining the 3D shape from an image Assuming Lambertian reflection Can recover the normal vector if we cast the light from 3 different directions Object N L  Example: sphere (lit from left)‏

24. Computer Graphics Inf4/MSc Integrate the normals over the surface to compute the 3D surface We can also estimate the colour information and use this for texture

25. Computer Graphics Inf4/MSc Surface inserted into new scenes lit according to the global illumination of that scene. This illumination can of course come from eyes!

26. Computer Graphics Inf4/MSc Acquiring the Reflectance Field of a Human Face Simulating reflectance on human faces requires a lot of effort and computation –Need to acquire the reflectance model –Subsurface scattering Instead of simulating the global illumination, we can simply illuminate the face by real- lights from different directions Creating images of arbitrary light conditions by combining the captured data

27. Computer Graphics Inf4/MSc The Light Stage: 60-second exposure

28. Computer Graphics Inf4/MSc Light Stage Data Original Resolution: 64  32 Lighting through image recombination: Haeberli ‘92, Nimeroff ‘94, Wong ‘97

29. Computer Graphics Inf4/MSc Reflectance Functions

30. Computer Graphics Inf4/MSc Lighting Reflectance Functions normalized light map reflectance function lighting product rendered pixel 1

31. Computer Graphics Inf4/MSc 13/11/200731 References Rendering Synthetic Objects into Real Scenes, SIGGRAPH 98 Acquiring the Reflectance Field of a Human Face, SIGGRAPH 2000 http://www.debevec.org http://www.debevec.org Eyes For Relighting, Nishino SIGGRAPH 2004