1. Real-Time Relighting Digital Image Synthesis Yung-Yu Chuang 1/10/2008 with slides by Ravi Ramamoorthi, Robin Green and Milos Hasan

2. Realistic rendering We have talked about photorealistic rendering for complex materials, complex geometry and complex lighting. They are realistic but slow.

3. Real-time rendering Its goal is to achieve interactive rendering with reasonable quality. It ’ s important in many applications such as games, visualization, computer-aided design, …

4. Real-Time relighting Lighting is the process of adjusting lights. It is an important but time-consuming step in animation production pipeline. Relighting algorithms for two kinds of lights –Distant environment lights –Near-field lights for production

5. Relighting algorithms for distant environment lights

6. Natural illumination People perceive materials more easily under natural illumination than simplified illumination. Images courtesy Ron Dror and Ted Adelson

7. Natural illumination Rendering with natural illumination is more expensive compared to using simplified illumination directional source natural illumination

8. Reflection maps Blinn and Newell, 1976

9. Environment maps Miller and Hoffman, 1984

10. HDR lighting

11. Examples of complex environment light

13. Direct lighting with complex illumination p q

14. Function approximation G(x): the function to approximate B 1 (x), B 2 (x), … B n (x): basis functions We want Storing a finite number of coefficients c i gives an approximation of G(x)

15. Function approximation How to find coefficients c i ? –Minimize an error measure What error measure? –L 2 error Coefficients

16. Basis Functions are pieces of signal that can be used to produce approximations to a function Function approximation

17. We can then use these coefficients to reconstruct an approximation to the original signal Function approximation

18. We can then use these coefficients to reconstruct an approximation to the original signal Function approximation

19. Orthogonal basis functions Orthogonal Basis Functions –These are families of functions with special properties –Intuitively, it’s like functions don’t overlap each other’s footprint A bit like the way a Fourier transform breaks a functions into component sine waves

20. Integral of product

21. Basis functions Transform data to a space in which we can capture the essence of the data better Spherical harmonics, similar to Fourier transform in spherical domain, is used in PRT.

22. Real spherical harmonics A system of signed, orthogonal functions over the sphere Represented in spherical coordinates by the function where l is the band and m is the index within the band

23. Real spherical harmonics

24. Reading SH diagrams – + Not this direction This direction

25. Reading SH diagrams – + Not this direction This direction

26. The SH functions

28. Spherical harmonics

29. -2012 0 1 2 m l

30. SH projection First we define a strict order for SH functions Project a spherical function into a vector of SH coefficients

31. SH reconstruction To reconstruct the approximation to a function We truncate the infinite series of SH functions to give a low frequency approximation

32. Examples of reconstruction

33. An example Take a function comprised of two area light sources –SH project them into 4 bands = 16 coefficients

34. Low frequency light source We reconstruct the signal –Using only these coefficients to find a low frequency approximation to the original light source

35. Harr wavelets Scaling functions ( V j ) Wavelet functions ( W j ) The set of scaling functions and wavelet functions forms an orthogonal basis

36. Harr wavelets

37. Example for wavelet transform Delta functions, f=(9,7,3,5) in V 2

38. Wavelet transform V 1, W 1

39. Example for wavelet transform V 0, W 0, W 1

40. Example for wavelet transform

41. Quadratic B–spline scaling and wavelets

42. 2D Harr wavelets

43. Example for 2D Harr wavelets

44. Applications 19% 5% L 2 1% 15% L 2 3% 10% L 2

45. Relighting algorithms for animation production

46. Relighting for production Lighting is a time-consuming process. Artists adjust lighting parameters and wait for a couple of hours or days to get feedback. Local shading with complex scene and many lights Interactive relighting –Interative visual eedback –Fixed scene and camera –Lower quality –Scalable with sene complexity and number of lights

47. Deep framebuffer Gershbein and Hanrahan, SIGGRAPH 2000

48. Deep framebuffer

50. LPICS Pixar, SIGGRPH 2005. A practical realization for the deep framebuffer approach on GPUs LPICS 0.1s Final renderer 2,000s video

51. Lightspeed ILM, SIGGRAPH 2007 An even more practical system with automatic shader conversion. (2.7s v.s. 57m)

52. Direct-to-indirect transfer Hasan et. al. SIGGRAPH 2006 Deep framebuffer approaches only support local shading, but not indirect lighting direct lightingWith indirect lighting

53. Concept Distribute gather samples on scene surfaces

54. Concept Direct illumination on both gather samples and view samples

55. Concept Inter-reflections between gather samples

56. Concept Final gather on view samples

57. Inter-reflections between gather samples gather sample

58. Inter-reflections between gather samples Assume all gather samples are diffuse

59. Inter-reflections between gather samples

61. Final gathering

62. Direct on gather Indirect on view Final Transfer matrix Direct on view Concept

63. Scene: Still Life Precomputation: 1.6 hours 11.4 – 18.7 fps Polygon: 107k

64. Scene: Temple Precomputation: 2.5 hours8.5 – 25.8 fpsPolygon: 2M

65. Scene: Hair Ball Precomputation: 2.9 hours9.7 – 24.7 fpsPolygon: 320k

66. Scene: Sponza Atrium Precomputation: 1.5 hours13.7 – 24.9 fpsPolygon: 66k

67. Comparison DTI: 8-25 fps (2.5 hr precomputation) Monte Carlo path tracer: 32 hours