Real-time Global Illumination with precomputed probe Baek Jun Kim KAIST (Korea Advanced Institute of Science and Technology) 1

Review Ambient Occlusion Rendering with Neural Networks (20184230 김세훈) 1) Deep shading (2D frame based learning) 2) DRAO (3D geometry based learning)

Global Illumination Direct illumination only Global illumination (indirect illumination)

Ray Tracing Ray Tracing In Real-time? To implement Global Illumination In Real-time? Too costly to compute at interactive rates Even several hours for a image How to realize Real-time Ray Tracing Precomputation … and others

Papers Real-Time Global Illumination using Precomputed Light Field Probes Real-time Global Illumination by Precomputed Local Reconstruction from Sparse Radiance Probes

Real-Time Global Illumination using Precomputed Light Field Probes Morgan McGuire, Michael Mara, Derek Nowrouzezahrai, and David Luebke. In I3D 2017. 11.

Summary Precomputing Probe Radiance, normal, and distance map Ray casting using Probe Intersection testing with distance map Shading with Probes 1) Spatial-Temporal Radiance Denoising (BRDF factorization) 2) Irradiance with Pre-Filtered Visibility (Geometry aware)

Real-Time GI using Precomputed Light Field Probes A scene computed in 42ms (212fps) Let me show its previous works briefly: Image Based Lighting Voxel Cone Tracing

Image-based Lighting Cube map (Environment mapping) In this paper, every probe get its cube map.

Voxel Cone Tracing Using Sparse Voxel Octree Optimized in modern GPU Hierarchical

Precomputed Light Field Probes Rasterization to render cube map Octahedral parameterization for efficiency Two-level MIPmap hierarchy (heuristic)

Light Field Probe Ray Tracing How to use precomputed probe? We can trace a ray! Procedure Selecting a probe Single-probe tracing routine Selecting fall-back probes

Selecting a Probe The probe whose center lies closest to the ray Ray

Single-probe tracing routine

Intersection test Intersection test by comparing depth map value in the probe and radial distance from the probe’s center Ray Depth map in the probe Surface Probe

Intersection test (2) If depth map value is smaller than radial distance, the ray has either hit a surface or pass behind the surface Ray Depth map in the probe Surface Probe

Intersection test (3) By using normal map in probe, we can distinguish between HIT and behind the surface Ray Depth map in the probe Surface Probe

Selecting fall-back probes If unable to find a definitive hit or miss, pick another probe

Shading with Light Field Probes Now we can trace a ray with probes, then? 3 Global Illumination Algorithm Compute indirect lighting using MC & Importance Sampling Unbiased, but slow We have to reduce the number of ray samples Spatial-Temporal Radiance Denoising Irradiance with Pre-Filtered Visibility

Spatial-Temporal Radiance Denoising Factorize BRDF into Lambertian term and Glossy Term Indirect illumination pass traces one ray sampled from each of them and writes the result to each buffer For Lambertian reflection, we compute incident irradiance instead of radiance

Spatial-Temporal Radiance Denoising (2) Lambertian reflections is spatially slower- varying Use of wider filter Glossy reflection is spatially faster-varying Use of narrower filter

Irradiance with Pre-Filtered Visibility How to compute irradiance? Precompute irradiance map Interpolate irradiance map at eight probes within the cube of the intersection Reduce number of texture operation → really fast Limitation: Lighting and darkness leaking

Irradiance with Pre-Filtered Visibility (2) Geometry-aware approach While precomputing irradiance map, we also compute radial distance map At runtime, we use the radial distance map for interpolating irradiance

Summary Precomputing Probe Radiance, normal, and distance map Ray casting using Probe Intersection testing with distance map Shading with Probes 1) Spatial-Temporal Radiance Denoising (BRDF factorization) 2) Irradiance with Pre-Filtered Visibility (Geometry aware)

Real-Time Global Illumination using Precomputed Light Field Probes Ari Silvennoinen and Jaakko Lehtinen. ACM Transactions on Graphics, November 2017.

Summary To get indirect illumination, combined two methods Precomputed Light Transport Interpolation Factorize Global Transport into (1) Local Transport and (2) Probe Transport

Real-time GI by Precomputed Local Reconstruction from Sparse Radiance Probes GI = Direct illumination + Indirect illumination How to get indirect illumination?

How to get indirect illumination? In this paper, Precomputed light transport Direct-to-Indirect transfer [Hašan et al. 2006] Interpolation method Interpolated irradiance and bandlimited radiance [E.g. Ward 1992]

Direct-to-Indirect transfer [Hašan et al. 2006] Precomputed radiance transfer method Precompute transfer matrix that map direct illumination into indirect illumination

Interpolation method Interpolation from radiance probes are spatial interpolation weight ω p2 p1 x

Limitations a) Direction mismatch b) Visibility mismatch

Visibility-Aware Interpolation Ψ for direction Vi(ω) for Visibility

Spherical harmonics is a kind of Laplace transform of Sphere Probe Radiance High-resolution angular sampling of the probes’ radiance is expensive Replace it by Spherical Harmonics form λ: probe radiance vector Y: SH basis function (basis func. index j) Spherical harmonics is a kind of Laplace transform of Sphere

Local transport Operator Local transport Operator transforms the probe radiance vector λ to interpolated incident radiance at x as a function of the continuous direction ω

Precomputed Transport transport coefficients ϕij

Factorized transport Factorize global transport into local transport and probe transport Interpolate with probes

Local Transport Precomputed local transport matrix: accounts for near field effects between the receiver and the mutually visible surfaces seen by the probes Vi(ω) makes matrix

Probe Transport Captures radiance in a sparse set of probes at runtime

Scene Example

Scene Exmaple (2)

Implementation - Precomputation Probe locations and radii: Every surface point is under the support of at least one probe Every surface point is visible from at least one probe Use greedy algorithm Cluster Receivers into a vector

Implementation - Runtime Computing Probe Transport Direct illumination light map is updated every frame

Result

Result

Summary To get indirect illumination, combined two methods Precomputed Light Transport Interpolation Factorize Global Transport into (1) Local Transport and (2) Probe Transport

Pros and Cons Real-Time Global Illumination using Precomputed Light Field Probes Dynamic Camera Static and Dynamic Environment Real-time Global Illumination by Precomputed Local Reconstruction from Sparse Radiance Probes Only Static Environment Dynamic Light Source

Quiz In Real-Time Global Illumination using Precomputed Light Field Probes, which map of the probe used to test intersection during Light Field Probe Ray Tracing? (1) Shadow map (2) Distance map (3) Irradiance map (4) Radiance map In Real-time Global Illumination by Precomputed Local Reconstruction from Sparse Radiance Probes, global transport is factorized into 2 parts: precomputed ( ) transport and ( ) transport which is computed at runtime (1) probe, local (2) local, probe (3) SH, SVD (4) SVD, SH