1. Normal Mapping for Precomputed Radiance Transfer
Peter-Pike Sloan
Microsoft

2. Inspiration
[McTaggart04] Half-Life 2 “Radiosity Normal Mapping”

3. Related Work
[Willmott99] Vector irradiance formulation of radiosity (for accelerating computation)
[Tabellion04] Lighting model similar to above (dominant light direction)
[Good2005] “Spherical Harmonic Light Maps”

4. Goal
Develop lightweight techniques to decouple normal variation (ala HL2)
From parameterized models of lighting (PRT)
For rigid objects
Non goals
Modeling “local GI” of the bumps [Sig05]
Masking effects
Glossy materials

5. PRT
Now that we’ve seen a demo we are going to define some of the terms we will use for the rest of the talk.
We use the abbreviation “PRT” for precomputed radiance transfer.
In PRT, the source radiance function represents the distant lighting that will illuminate the object.
It is a spherical function represented by a vector of coefficients with respect to a given basis.

6. PRT
At every point over the surface of the object, this source radiance function is attenuated due to shadowing and increased due to inter-reflections.
We call the result transferred incident radiance.
It represents the local environment lighting each point “p”.

7. PRT
This transferred incident radiance function can be integrated against the surface material properties to produce the exit radiance emanating from that point.
Exit radiance is a spherical function that is parameterized by view direction.

8. Bi-Scale Radiance Transfer
In order to get that global transport we introduce Bi-scale radiance transfer.
It uses PRT to model macro-scale transport – the source radiance is transformed into a local lighting vector at a coarse sampling over the object. This is then interpolated and used to light an RTT that models the meso-scale effects.

9. Bi-Scale Radiance Transfer
l : vector = source radiance spherical function
Mp : 25x25 transfer matrix at point p (source → transferred incident)
q(xp) : ID map (2D → 2D, maps RTT patch over surface)
b(x,v) : RTT (4D → 25D, tabulated over small spatial patch)
Mathematically, source radiance L is transferred to incident radiance using the transfer matrix Mp.
An ID map “q” defined over the whole surface indexes into a small RTT “b”.
The resulting RTT sample is then dotted with the transferred incident radiance vector, generating exit radiance “e”.
This essentially applies macro-scale transferred radiance to the meso-scale RTT.
applies macro-scale transferred radiance to meso-scale RTT

10. Bi-Scale Radiance Transfer
l : vector = source radiance spherical function
Mp : 25x25 transfer matrix at point p (source → transferred incident)
q(xp) : ID map (2D → 2D, maps RTT patch over surface)
b(x,v) : RTT (4D → 25D, tabulated over small spatial patch)
Mathematically, source radiance L is transferred to incident radiance using the transfer matrix Mp.
An ID map “q” defined over the whole surface indexes into a small RTT “b”.
The resulting RTT sample is then dotted with the transferred incident radiance vector, generating exit radiance “e”.
This essentially applies macro-scale transferred radiance to the meso-scale RTT.
applies macro-scale transferred radiance to meso-scale RTT

11. Bi-Scale Radiance Transfer
l : vector = source radiance spherical function
Mp : 25x25 transfer matrix at point p (source → transferred incident)
q(xp) : ID map (2D → 2D, maps RTT patch over surface)
b(x,v) : RTT (4D → 25D, tabulated over small spatial patch)
Mathematically, source radiance L is transferred to incident radiance using the transfer matrix Mp.
An ID map “q” defined over the whole surface indexes into a small RTT “b”.
The resulting RTT sample is then dotted with the transferred incident radiance vector, generating exit radiance “e”.
This essentially applies macro-scale transferred radiance to the meso-scale RTT.
applies macro-scale transferred radiance to meso-scale RTT

12. Limitations Expensive Add constraints
Stores 64 “response vectors” that are 9-36D (x3 for spectral)
Parallax mapping cheaper way of getting masking
Local radiance is too much data (9 – 36 x 3) for low res textures/per-vertex
Add constraints
Just model normal variation
Diffuse only

13. Diffuse + Normal Maps Quadratic SH [Ramamoorthi2001]
Distant Lighting Environment

14. Distant Radiance to Transferred Incident Radiance
Diffuse + Normal Maps
Quadratic SH [Ramamoorthi2001]
Distant Radiance to Transferred Incident Radiance
In local frame

15. Diagonal Convolution Matrix
Diffuse + Normal Maps
Quadratic SH [Ramamoorthi2001]
Diagonal Convolution Matrix
Clamped cosine kernel

16. Irradiance Environment Map
Diffuse + Normal Maps
Quadratic SH [Ramamoorthi2001]
Irradiance Environment Map

17. Irradiance Environment Map
Diffuse + Normal Maps
Quadratic SH [Ramamoorthi2001]
Irradiance Environment Map

18. Evaluate SH basis with normal
Diffuse + Normal Maps
Quadratic SH [Ramamoorthi2001]
Evaluate SH basis with normal

19. Concerns Lot of data at low res
9xO2 matrices (x3 with color bleeding)
Can compress using CPCA [Sig03]
Too much data passed from low res to high res
Irradiance emap (27 numbers, 7 interpolators)
Alternatives
Project into analytic basis
Separable approximation

20. Project into new basis (fewer rows)
Analytic Basis
Project into new basis (fewer rows)

21. Shifted Associated Legendre Polynomials
[Gautron2005]

22. Half-Life 2 Basis

23. Comparison
PRT
Gold Standard
HL2
SAL

24. Comparison
PRT
Gold Standard
HL2
SAL

25. Normal Mapping for PRT
Use same ideas as BRDF factorization [Kautz and McCool1999]
Another approach, that we will discuss here is to use the same ideas that are used for BRDF factorization, which Jaakko talked about earlier.
That is build a matrix “A” sampling the convolved lighting environment over a hemisphere of normals. The rows represent the normal directions, the columns the lighting environment.
Bi-linear basis functions over hemisphere (4 non-zero)
Matrix, rows “normal directions” columns quadratic
SH light Aij equals evaluating convolved light basis
function “j” in normal direction “i”

26. Normal Mapping for PRT
This is just trying to give more intuition to the matrix A.
You have some sampling of the hemisphere, each row corresponds to a sample.
The columns correspond to illumination from the 9 quadratic SH basis functions.
A coefficient in the matrix represents the integral of a cosine kernel in the given direction against the lighting environment. Not that the lighting environment should be clamped to the hemisphere before this integral happens.
Bi-linear basis functions are used on the unit disk, and then mapped to the hemisphere to generate a value at any point on the hemisphere (so there are at most 4 non-zero entries for a given normal.)

27. Normal Mapping for PRT Compute SVD of A
Nx9 matrix (each column is a “normal basis” texture)
Then you compute the singular value decomposition of this matrix and you get 3 terms.
A matrix U, where each column represents a “normal basis” texture.
A diagonal matrix S (the singular values)
And a matrix Vt,which is 9x9 (for quadratic SH.)
9x9 diagonal matrix (singular values)
9x9 matrix

28. Normal Mapping for PRT Old equation New equation
This generates a new equation that simply replaces the evaluation of the quadratic SH in the normal direction.

29. Normal Mapping for PRT Use first M singular values MxO2 matrix
M channel “normal direction” texture
Then instead of using the full matrix, just use the first M singular values.

30. Pixel Shader StandardSVDPS( VS_OUT In, out float3 rgb : COLOR ) {
float2 Normal = tex2D(NormalSampler, In.TexCoord);
float2 vTex = Normal*0.5 + float2(0.5,0.5);
float4 vU = tex2D(USampler,vTex);
rgb.r = dot(In.cR,vU);
rgb.g = dot(In.cG,vU);
rgb.b = dot(In.cB,vU);
rgb *= tex2D(AlbedoSampler, In.TexCoord); }

31. Comparison

32. Demo

33. Conclusions Lightweight form of normal mapping for PRT
Inspired by Half-Life 2
For static objects, diffuse only
HL2 basis and separable basis seem to be best
Experiment with CPCA more [Sig03]
Integrate with other techniques
Parallax mapping for masking
Ambient Occlusion for local effects

34. Acknowledgments Gary McTaggart for HL2 images Shanon Drone for Models
Paul Debevec for Light Probes