1. On robust Monte Carlo algorithms for multi-pass global illumination Frank Suykens – De Laet 17 September 2002

2. Overview Introduction –Realistic image synthesis –Global illumination Algorithms for global illumination Contributions –Weighted multi-pass methods –Path differentials –Density control for photon maps Conclusion

3. Overview Introduction –Realistic image synthesis –Global illumination Algorithms for global illumination Contributions –Weighted multi-pass methods –Path differentials –Density control for photon maps Conclusion

4. Realistic image synthesis Goal: Compute images that appear to an observer as real photographs Which one is real?

5. Realistic image synthesis Applications –Architecture –Movie industry –Lighting design –Computer games –Archeology –Product design –…

6. Realistic image synthesis Scene description Light Transport Simulation Compute illumination Image

7. Scene description Geometry Materials Light sources Camera / Eye Position, size, … (e.g., CAD)

8. Scene description Geometry Materials Light sources Camera / Eye Diffuse paint, glass, metal, …  BSDF

9. Materials: BSDF Bidirectional scattering distribution function (reflection & transmission)  x  Fraction of incoming radiance L(x   ) that is scattered into the direction θ

10. BSDF Components Diffuse (D)Glossy (G)Specular (S) Diffuse, glossy and specular: (D|G|S) = X

11. Scene description Geometry Materials Light sources Camera / Eye Position, brightness, spotlight, …

12. Scene description Geometry Materials Light sources Camera / Eye Position, viewing angle, …

13. Realistic image synthesis Scene description Light Transport Simulation Compute illumination Image Geometry Materials Light sources Camera/Eye

14. Compute illumination For every pixel: how much light passes through? Account for all possible paths from light to eye!  Global illumination Light Transport Simulation

15. Global illumination Mathematical basis for light transport Outgoing radiance L in x in direction θ ? x  L  Rendering equation Light Transport Simulation

16. Rendering equation =+ Radiance x  L Integration over all directions BSDF Unknown incoming radiance x  LeLe Self emitted radiance  LrLr Reflected (& refracted) radiance x Light Transport Simulation Recursive

17. Realistic image synthesis Scene description Light Transport Simulation Compute illumination Image Geometry Materials Light sources Camera/Eye Global illumination Rendering equation

18. Overview Introduction –Realistic image synthesis –Global illumination Algorithms for global illumination Contributions –Weighted multi-pass methods –Path differentials –Density control for photon maps Conclusion

19. Example scene Specular refraction Caustics Indirect caustics Indirect illumination Many different illumination features: We want a full global illumination solution!

20. Algorithms for global illumination Computation: Numerical integration –Monte Carlo integration Algorithms –Image space algorithms Stochastic ray tracing Particle tracing Bidirectional path tracing –Object space algorithms Radiosity

21. Monte Carlo integration Estimate integrals by random sampling –draw a number of random samples –average their contribution  estimate of integral Statistical errors  Noise in images Convergence: More samples, less noise

22. Stochastic ray tracing Trace paths starting from the eye 9 paths/pixel L E Monte Carlo integration

23. Particle tracing Trace paths starting from the light 9 paths/pixel L E Pattanaik ’92, Dutré ’93

24. Bidirectional path tracing Trace paths starting from the light AND the eye L E Lafortune ’93, Veach ’94

25. Comparison Same computation time (± 5 min.) Stochastic ray tracing (9 samples per pixel) Particle tracing (9 samples per pixel) Bidirectional path tracing (4 samples per pixel)

26. Radiosity methods Object space method Diffuse surfaces only View independent Galerkin radiosity

27. Overview Introduction –Realistic image synthesis –Global illumination Algorithms for global illumination Contributions –Weighted multi-pass methods –Path differentials –Density control for photon maps Conclusion

28. Overview Introduction –Realistic image synthesis –Global illumination Algorithms for global illumination Contributions –Weighted multi-pass methods –Path differentials –Density control for photon maps Conclusion

29. Multi-pass methods Combine different algorithms Separate light transport –Based on BSDF components –Different algorithms  different illumination –Preserve strengths of individual algorithms Regular expressions (e.g., LD *, LX * E ) –derive path evaluation from regular expression

30. Radiosity & stochastic ray tracing LD * (G|S)X * E  LX*E E D|G|S LD * G|S Full global illumination but drawbacks of stoch. ray tracing  Combine with bidirectional path tracing 1. Radiosity 2. Stochastic ray tracing Use radiosity solution at end points

31. Multi-pass configuration ++ BPTUse weighting Rad + SR L(G|S)X*E LD(G|S)X*E + LDE ??? Self-emitted light Direct diffuse Indirect diffuse LDD + (G|S)X*E + LDD + E

32. Weighting instead of separation –allow overlapping transport between different algorithms –weight individual paths  automatic ‘separation’ Technique –General Monte Carlo variance reduction technique –Constraints, weighting heuristics Weighted multi-pass methods

33. Results (unweighted)  Bidirectional path tracingRadiosity + stoch. ray tracing LD(G|S)X*E + LDE

34. Results (weighted) + Bidirectional path tracingRadiosity + stoch. ray tracing LD(G|S)X*E + LDE

35. Final result BPT only Radiosity + Stoch. RT Weighted combination Radiosity + Stoch. RT and Bidirectional path tracing

36. Conclusion: WMP Multi-pass methods –separation: path evaluation from regular expression –weighting: each path is weighted individually  automatic ‘separation’ General technique Robust combination of bidirectional path tracing and radiosity

37. Overview Introduction –Realistic image synthesis –Global illumination Algorithms for global illumination Contributions –Weighted multi-pass methods –Path differentials –Density control for photon maps Conclusion

38. Path differentials Idea –Many algorithms trace paths –A path is infinitely thin: no neighborhood information –Knowledge about ‘region of influence’ or ‘footprint ’ would be useful in many applications: bias-noise trade-off Footprint definition Path differentials

39. Path footprint Path = function of random variables –direction sampling, light source sampling, …

40. Path footprint Variables change  path perturbation

41. Path footprint Set of path perturbations  footprint

42. Path differentials Partial derivatives –approximate perturbations –combine into footprint (first order Taylor approx.) –footprint estimate from a single path!

43. Applications Path differentials widely applicable –Any Monte Carlo path sampling algorithm Texture filtering Hierarchical particle tracing radiosity Importance maps

44. Application: hierarchical radiosity Particle tracing radiosity L Trace light paths Each hit contributes to the illumination of the element In which level should the particle contribute? Path differentials: size of footprint  size of element Small elements  noise Large elements  blur fixed hierarchical

45. Application: hierarchical radiosity Fixed size (large) Fixed size (small) Path differentials

46. Application: hierarchical radiosity Fixed size (large) Fixed size (small) Path differentials

47. Conclusion: Path differentials New, robust technique to compute path footprint Handles general BSDFs, complex geometry Many applications in global illumination

48. Overview Introduction –Realistic image synthesis –Global illumination Algorithms for global illumination Contributions –Weighted multi-pass methods –Path differentials –Density control for photon maps Conclusion

49. Photon mapping Popular 2-pass global illumination algorithm Jensen ’96, … 1. Particle tracing trace light paths

50. 1. Particle tracing trace light paths record all hitpoints Photon mapping Popular 2-pass global illumination algorithm  Set of photons: ‘Photon map’ Jensen ’96, …

51. Photon mapping Density of photons  radiance estimate Photon hits Radiance estimate

52. Photon mapping: second pass Global map: indirect visualization Caustic map: direct visualization Global map Caustic map Final image 2. Stochastic ray tracing indirect direct

53. Photon mapping examples

54. Photon mapping Advantages –efficient, full global illumination –robust (photon map independent of geometrical complexity) Difficulties –many photons  a lot of memory! –how many photons needed?  Density control

55. Density control Only store photons when more photons are needed –choose target density –new photon hit: target density reached? No  store photon Yes  redistribute photon power among neighbors

56. Density control Target density?  Importance maps Path differentials can be used! Trace ‘importons’ from eye  importance map OverviewViewpoint Target density Error analysis

57. Results: photon map construction Actual density of photon map Radiance estimate No density control, 400.000 photons Density control, 57.000 photons

58. Results: final image No density control, 400.000 photons With density control, 57.000 photons No visible difference with 1/7 th of the photons

59. Conclusion: Density control Fewer photons: memory efficient Global & Caustic map Important step towards error control

60. Overview Introduction –Realistic image synthesis –Global illumination Algorithms for global illumination Contributions –Weighted multi-pass methods –Path differentials –Density control for photon maps Conclusion

61. Conclusion General techniques to construct better, more robust global illumination methods –Weighted multi-pass methods –Path differentials –Density control for photon maps Wide applicability (general scenes, other algorithms) Future work: –improved techniques –more applications RenderPark: our freely available global illumination software (www.renderpark.be)

62. Future work Improved techniques –Path differentials: second order derivatives –Error control for photon maps More applications –Weighted multi-pass & photon maps –Path differentials New techniques

63. Light transport simulation For every pixel: how much light passes?

64. Weighted multi-pass methods 1.Separation –derived directly from regular expressions 2.Weighting –weight overlapping transport –automatically preserves strengths –extension of multiple importance sampling

65. Path footprint Region of influence? Distance to neighboring paths?

66. Regular expressions Convenient representation of partial light transport Examples –LD * E : paths with one or more diffuse reflections  radiosity solution –L(D|G|S) * E = LX * E : paths including all BSDF components  full global illumination

67. Multi-pass example Radiosity preprocess (LD * )+ stochastic ray tracing ((G|S) * E)  LD * ELD * (G|S) * E

68. Path footprint

69. Bidirectional path tracing Trace paths starting from the light AND the eye Different ways to generate the same path  Multiple importance sampling L E

70. Weighting instead of separation Extension of multiple importance sampling –combine paths of different length General Monte Carlo variance reduction technique –constraints, weighting heuristics Weighted multi-pass methods L E L E LD Rad + SR BPT

71. Bidirectional path tracing Trace paths starting from the light AND the eye Different ways to generate the same path  Multiple importance sampling L E

72. Example scene We want a full global illumination solution! (without waiting too long) Classical ray tracing (30 sec)Bidirectional path tracing (20 hrs)

73. Contributions General techniques to construct better, more robust global illumination algorithms –full global illumination –Monte Carlo algorithms –multi-pass algorithms On robust Monte Carlo algorithms for multi-pass global illumination

74. Radiosity & stochastic ray tracing E Use radiosity solution at end points D|G|S LD * G|S LD * (G|S)(D|G|S) * E  LX*E Full global illumination but drawbacks of stoch. ray tracing  Combine with bidirectional path tracing

75. Path footprint estimation Beam tracing, cone tracing, … –difficult intersection tests –difficult to handle general reflection/refraction Extend to finite size Heckbert ‘84, Amanatides ‘84, …

76. Path footprint estimation Partial derivatives: Footprint estimated from a single path –General reflection/refraction –No change in intersection tests Path differentials

77. Path differentials: technique Path = function of (random) variables D D’ D’ = f(D, u,v) = f(u 0, u 1,…, u,v) (random) variables Path differentials –small neighborhood in domain  small neighborhood around a vertex –partial derivatives & first order Taylor approximation Domain 0 1 1 1

78. Application: texture filtering Filter textures locally –reduces noise caused by texture variation –fast box filter over the estimated footprint Extends Igehy ‘99

79. Filtering over footprint (4 s/p)No filtering (4 s/p) Application: texture filtering