Efficient Irradiance Normal Mapping Ralf Habel, Michael Wimmer Institute of Computer Graphics and Algorithms Vienna University of Technology

Ralf Habel 1 Motivation Combining Light Mapping and Normal Mapping Also know as: Radiosity Normal Mapping Directional Light Mapping Spherical Harmonics Light Mapping Popular in Games Half-Life 2, Halo 3 … Cheap and good looking: Normal maps can be reused Per vertex/per texel light map pipeline Fast and trivial evaluation

Ralf Habel 2 Motivation Light mapped

Ralf Habel 3 Motivation Irradiance normal mapped

Ralf Habel 4 Motivation Irradiance normal mapped no albedo

Ralf Habel 5 Introduction Goal: Represent irradiance on all surfaces for all possible directions (S x Ω) Allows illumination to be stored sparsely similar to light mapping Local variation is transported by normal maps Representation: Environment maps (piecewise linear) Basis function sets (Spherical Harmonics) Evaluation: Look up/calculate irradiance value in normal direction

Ralf Habel 6 Irradiance Environment Maps Ramamoorthi et al. 2001: Spherical Harmonics up to the quadratic band (RGB: 27 coefficients) is enough for an accurate representation (avg. error < 3%). 9 RGB textures containing SH coefficients  Irradiance over all directions is a low frequency signal Can we do better? Only hemispherical signal (Ω + ) needed on opaque surfaces Other basis functions than Spherical Harmonics?

Ralf Habel 7 Hemispherical bases Set of functions defined over the hemisphere (Ω + ) Desired attributes for irradiance: No discontinuities for smooth interpolation Orthonormality: simplifies projections and other calculations (just like in Euclidian space) Band structure for LOD/increasing accuracy (like Spherical Harmonics) Not important: Locality High-frequency behavior

Ralf Habel 8 Half-Life 2 Basis Consists of 3 orthonormal cosine lobes (linear SRBFs) Orthonormal over Ω + Equivalent to Directional occlusion (one general cosine lobe) Linear Spherical Harmonics band normed on Ω + All require 3 coefficients and are linear No quadratic terms

Ralf Habel 9 Hemispherical bases General orthonormal hemispherical bases: Hemispherical Harmonics [Gautron et al. 04] Makhotkin Basis [Makhotkin 96]  All basis functions are 0 or constant on border of Ω + due to generation through shifting  Non-polynomial Zernike Basis [Koenderink 96]  Different band structure: 1,2,3..instead of 1,3,5..  Non-polynomial

Ralf Habel 10 Creating Directional Irradiance We need irradiance on all surface points in all Ω + directions: Convolution with diffuse kernel far too expensive in Cartesian coordinates Tens of millions of convolutions Instead: Spherical Harmonics as an intermediate basis [Ramamoorthi 01, Basri and Jacobs 00]

Ralf Habel 11 Creating Directional Irradiance Create radiance estimate in precomputation From photon mapping, path tracing, shadow mapping… In tangent space (for tangent space normal maps) Expand radiance into Spherical Harmonics by integrating against SH basis functions:

Ralf Habel 12 Creating Directional Irradiance Perform diffuse convolution directly in SH to get Using Funk-Hecke Theorem, diffuse convolution is carried out by scaling SH coefficients in each band l with a l : a 0 = 1, a 1 = 2/3, a 2 = ¼, a 3 = 0, a 4 = -1/24 There is never a cubic contribution in an SH irradiance signal All l >=4 are very small  This is why SH up to the quadratic band is so efficient for irradiance!

Ralf Habel 13 H-Basis We would like something similar to SH on Ω + Polynomial As fast as SH to evaluate Same interpolation behavior Orthonormal on Ω + Targeted for irradiance representation Take a close look at SH functions and polynomial Hilbert space to derive basis functions

Ralf Habel 14 H-Basis SH functions that are symmetric to the z-axis are orthogonal on the hemisphere as well: Y 0 0,Y 1 -1,Y 1 1,Y 2 -2,Y 2 2 Renormed to Ω +

Ralf Habel 15 H-Basis SH functions that are symmetric to the z-axis are orthogonal on the hemisphere as well: Y 0 0,Y 1 -1,Y 1 1,Y 2 -2,Y 2 2 Renormed to Ω +

Ralf Habel 16 H-Basis Apply shifting to Y 1 0 (cos θ = 2 cos θ -1) Similar to Hemispherical Harmonics/ Makhotkin basis

Ralf Habel 17 H-Basis  Results in Ω + orthonormal polynomial basis with 1 constant, 3 linear and 2 quadratic basis functions  There is a mathematical rigorous derivation!

Ralf Habel 18 H-Basis Band structure allows to use only the constant+linear functions (H 4 ) or all six (H 6 ) similar to SH

Ralf Habel 19 SH to H-Basis Directional irradiance signals are calculated in SH Project SH coefficient vector into H-Basis with matrix multiplication: Sparse due to closeness to SH Both bases are polynomial No loss due to change in used function space

Ralf Habel 20 Bases Comparison Visual/perceptual comparison of all bases Replace H-Basis with any other In “very bad case” lighting situation  All basis functions are contributing SH comparison is least-square hemispherically projected [Sloan 03] Makes optimal use of SH on Ω + Shown with increasing number of coefficients Only few are shown See paper for all of them

Ralf Habel 21 Bases Comparison: 3 Coefficients Ground truth Half-Life 2 (not how the game evaluates) Zernike 2 bands

Ralf Habel 22 Bases Comparison: 4 Coefficients Ground truth H 4 SH 2 bands (Ω + projected) Makhotkin 2 bands (Artefacts at border)

Ralf Habel 23 Bases Comparison: 6/9 Coefficients Ground truth H 6 (6 coefficients) SH 3 bands (9 coefficients)

Ralf Habel 24 Bases Comparison Integrated Mean Square Error averaged over random irradiance signals 6 coefficients is enough for a numerically accurate representation What about difference between H 4 and H 6 ?

Ralf Habel 25 H-Basis Comparison H coefficients

Ralf Habel 26 H-Basis Comparison H coefficients

Ralf Habel 27 Conclusion H-Basis is very efficient and very simple solution for hemispherical irradiance signals 4 coeffs. for perceptually accurate representation Probably sufficient for almost all practical cases 6 coeffs. for numerically accurate representation Some lighting situations may benefit from 6 coeffs. Orthonormality : Shader LOD (functions are delocalized) Easy expansion of other low frequency signals

Ralf Habel 28 Future Work There is a general mathematical description and derivation similar to Spherical Harmonics H-Basis is a special case Efficient generating procedures Clarify correlations to SH Other hemispherical signals Visibility? BRDFs?

Ralf Habel 29 Thanks for your attention