On robust Monte Carlo algorithms for multi-pass global illumination Frank Suykens – De Laet 17 September 2002

Overview Introduction –Realistic image synthesis –Global illumination Algorithms for global illumination Contributions –Weighted multi-pass methods –Path differentials –Density control for photon maps Conclusion

Overview Introduction –Realistic image synthesis –Global illumination Algorithms for global illumination Contributions –Weighted multi-pass methods –Path differentials –Density control for photon maps Conclusion

Realistic image synthesis Goal: Compute images that appear to an observer as real photographs Which one is real?

Realistic image synthesis Applications –Architecture –Movie industry –Lighting design –Computer games –Archeology –Product design –…

Realistic image synthesis Scene description Light Transport Simulation Compute illumination Image

Scene description Geometry Materials Light sources Camera / Eye Position, size, … (e.g., CAD)

Scene description Geometry Materials Light sources Camera / Eye Diffuse paint, glass, metal, …  BSDF

Materials: BSDF Bidirectional scattering distribution function (reflection & transmission)  x  Fraction of incoming radiance L(x   ) that is scattered into the direction θ

BSDF Components Diffuse (D)Glossy (G)Specular (S) Diffuse, glossy and specular: (D|G|S) = X

Scene description Geometry Materials Light sources Camera / Eye Position, brightness, spotlight, …

Scene description Geometry Materials Light sources Camera / Eye Position, viewing angle, …

Realistic image synthesis Scene description Light Transport Simulation Compute illumination Image Geometry Materials Light sources Camera/Eye

Compute illumination For every pixel: how much light passes through? Account for all possible paths from light to eye!  Global illumination Light Transport Simulation

Global illumination Mathematical basis for light transport Outgoing radiance L in x in direction θ ? x  L  Rendering equation Light Transport Simulation

Rendering equation =+ Radiance x  L Integration over all directions BSDF Unknown incoming radiance x  LeLe Self emitted radiance  LrLr Reflected (& refracted) radiance x Light Transport Simulation Recursive

Realistic image synthesis Scene description Light Transport Simulation Compute illumination Image Geometry Materials Light sources Camera/Eye Global illumination Rendering equation

Overview Introduction –Realistic image synthesis –Global illumination Algorithms for global illumination Contributions –Weighted multi-pass methods –Path differentials –Density control for photon maps Conclusion

Example scene Specular refraction Caustics Indirect caustics Indirect illumination Many different illumination features: We want a full global illumination solution!

Algorithms for global illumination Computation: Numerical integration –Monte Carlo integration Algorithms –Image space algorithms Stochastic ray tracing Particle tracing Bidirectional path tracing –Object space algorithms Radiosity

Monte Carlo integration Estimate integrals by random sampling –draw a number of random samples –average their contribution  estimate of integral Statistical errors  Noise in images Convergence: More samples, less noise

Stochastic ray tracing Trace paths starting from the eye 9 paths/pixel L E Monte Carlo integration

Particle tracing Trace paths starting from the light 9 paths/pixel L E Pattanaik ’92, Dutré ’93

Bidirectional path tracing Trace paths starting from the light AND the eye L E Lafortune ’93, Veach ’94

Comparison Same computation time (± 5 min.) Stochastic ray tracing (9 samples per pixel) Particle tracing (9 samples per pixel) Bidirectional path tracing (4 samples per pixel)

Radiosity methods Object space method Diffuse surfaces only View independent Galerkin radiosity

Overview Introduction –Realistic image synthesis –Global illumination Algorithms for global illumination Contributions –Weighted multi-pass methods –Path differentials –Density control for photon maps Conclusion

Overview Introduction –Realistic image synthesis –Global illumination Algorithms for global illumination Contributions –Weighted multi-pass methods –Path differentials –Density control for photon maps Conclusion

Multi-pass methods Combine different algorithms Separate light transport –Based on BSDF components –Different algorithms  different illumination –Preserve strengths of individual algorithms Regular expressions (e.g., LD *, LX * E ) –derive path evaluation from regular expression

Radiosity & stochastic ray tracing LD * (G|S)X * E  LX*E E D|G|S LD * G|S Full global illumination but drawbacks of stoch. ray tracing  Combine with bidirectional path tracing 1. Radiosity 2. Stochastic ray tracing Use radiosity solution at end points

Multi-pass configuration ++ BPTUse weighting Rad + SR L(G|S)X*E LD(G|S)X*E + LDE ??? Self-emitted light Direct diffuse Indirect diffuse LDD + (G|S)X*E + LDD + E

Weighting instead of separation –allow overlapping transport between different algorithms –weight individual paths  automatic ‘separation’ Technique –General Monte Carlo variance reduction technique –Constraints, weighting heuristics Weighted multi-pass methods

Results (unweighted)  Bidirectional path tracingRadiosity + stoch. ray tracing LD(G|S)X*E + LDE

Results (weighted) + Bidirectional path tracingRadiosity + stoch. ray tracing LD(G|S)X*E + LDE

Final result BPT only Radiosity + Stoch. RT Weighted combination Radiosity + Stoch. RT and Bidirectional path tracing

Conclusion: WMP Multi-pass methods –separation: path evaluation from regular expression –weighting: each path is weighted individually  automatic ‘separation’ General technique Robust combination of bidirectional path tracing and radiosity

Overview Introduction –Realistic image synthesis –Global illumination Algorithms for global illumination Contributions –Weighted multi-pass methods –Path differentials –Density control for photon maps Conclusion

Path differentials Idea –Many algorithms trace paths –A path is infinitely thin: no neighborhood information –Knowledge about ‘region of influence’ or ‘footprint ’ would be useful in many applications: bias-noise trade-off Footprint definition Path differentials

Path footprint Path = function of random variables –direction sampling, light source sampling, …

Path footprint Variables change  path perturbation

Path footprint Set of path perturbations  footprint

Path differentials Partial derivatives –approximate perturbations –combine into footprint (first order Taylor approx.) –footprint estimate from a single path!

Applications Path differentials widely applicable –Any Monte Carlo path sampling algorithm Texture filtering Hierarchical particle tracing radiosity Importance maps

Application: hierarchical radiosity Particle tracing radiosity L Trace light paths Each hit contributes to the illumination of the element In which level should the particle contribute? Path differentials: size of footprint  size of element Small elements  noise Large elements  blur fixed hierarchical

Application: hierarchical radiosity Fixed size (large) Fixed size (small) Path differentials

Application: hierarchical radiosity Fixed size (large) Fixed size (small) Path differentials

Conclusion: Path differentials New, robust technique to compute path footprint Handles general BSDFs, complex geometry Many applications in global illumination

Overview Introduction –Realistic image synthesis –Global illumination Algorithms for global illumination Contributions –Weighted multi-pass methods –Path differentials –Density control for photon maps Conclusion

Photon mapping Popular 2-pass global illumination algorithm Jensen ’96, … 1. Particle tracing trace light paths

1. Particle tracing trace light paths record all hitpoints Photon mapping Popular 2-pass global illumination algorithm  Set of photons: ‘Photon map’ Jensen ’96, …

Photon mapping Density of photons  radiance estimate Photon hits Radiance estimate

Photon mapping: second pass Global map: indirect visualization Caustic map: direct visualization Global map Caustic map Final image 2. Stochastic ray tracing indirect direct

Photon mapping examples

Photon mapping Advantages –efficient, full global illumination –robust (photon map independent of geometrical complexity) Difficulties –many photons  a lot of memory! –how many photons needed?  Density control

Density control Only store photons when more photons are needed –choose target density –new photon hit: target density reached? No  store photon Yes  redistribute photon power among neighbors

Density control Target density?  Importance maps Path differentials can be used! Trace ‘importons’ from eye  importance map OverviewViewpoint Target density Error analysis

Results: photon map construction Actual density of photon map Radiance estimate No density control, photons Density control, photons

Results: final image No density control, photons With density control, photons No visible difference with 1/7 th of the photons

Conclusion: Density control Fewer photons: memory efficient Global & Caustic map Important step towards error control

Overview Introduction –Realistic image synthesis –Global illumination Algorithms for global illumination Contributions –Weighted multi-pass methods –Path differentials –Density control for photon maps Conclusion

Conclusion General techniques to construct better, more robust global illumination methods –Weighted multi-pass methods –Path differentials –Density control for photon maps Wide applicability (general scenes, other algorithms) Future work: –improved techniques –more applications RenderPark: our freely available global illumination software (

Future work Improved techniques –Path differentials: second order derivatives –Error control for photon maps More applications –Weighted multi-pass & photon maps –Path differentials New techniques

Light transport simulation For every pixel: how much light passes?

Weighted multi-pass methods 1.Separation –derived directly from regular expressions 2.Weighting –weight overlapping transport –automatically preserves strengths –extension of multiple importance sampling

Path footprint Region of influence? Distance to neighboring paths?

Regular expressions Convenient representation of partial light transport Examples –LD * E : paths with one or more diffuse reflections  radiosity solution –L(D|G|S) * E = LX * E : paths including all BSDF components  full global illumination

Multi-pass example Radiosity preprocess (LD * )+ stochastic ray tracing ((G|S) * E)  LD * ELD * (G|S) * E

Path footprint

Bidirectional path tracing Trace paths starting from the light AND the eye Different ways to generate the same path  Multiple importance sampling L E

Weighting instead of separation Extension of multiple importance sampling –combine paths of different length General Monte Carlo variance reduction technique –constraints, weighting heuristics Weighted multi-pass methods L E L E LD Rad + SR BPT

Bidirectional path tracing Trace paths starting from the light AND the eye Different ways to generate the same path  Multiple importance sampling L E

Example scene We want a full global illumination solution! (without waiting too long) Classical ray tracing (30 sec)Bidirectional path tracing (20 hrs)

Contributions General techniques to construct better, more robust global illumination algorithms –full global illumination –Monte Carlo algorithms –multi-pass algorithms On robust Monte Carlo algorithms for multi-pass global illumination

Radiosity & stochastic ray tracing E Use radiosity solution at end points D|G|S LD * G|S LD * (G|S)(D|G|S) * E  LX*E Full global illumination but drawbacks of stoch. ray tracing  Combine with bidirectional path tracing

Path footprint estimation Beam tracing, cone tracing, … –difficult intersection tests –difficult to handle general reflection/refraction Extend to finite size Heckbert ‘84, Amanatides ‘84, …

Path footprint estimation Partial derivatives: Footprint estimated from a single path –General reflection/refraction –No change in intersection tests Path differentials

Path differentials: technique Path = function of (random) variables D D’ D’ = f(D, u,v) = f(u 0, u 1,…, u,v) (random) variables Path differentials –small neighborhood in domain  small neighborhood around a vertex –partial derivatives & first order Taylor approximation Domain

Application: texture filtering Filter textures locally –reduces noise caused by texture variation –fast box filter over the estimated footprint Extends Igehy ‘99

Filtering over footprint (4 s/p)No filtering (4 s/p) Application: texture filtering