Spherical Harmonics in Actual Games Tom Forsyth Muckyfoot Productions Ltd tomf@muckyfoot.com

What they are… Representation of an image over a sphere Something like a cube-map Not like a cube-map: Frequency-space representation Spherical equivalent of post-Fourier-transform data A bit like a JPEG, but a 2D sphere, not a 2D plane Continuous, rotationally invariant No sharp edges, no rotational “lumpiness” Easy for vector units to manipulate

What they are not… Not an algorithm Not Precomputed Radiance Transform Just a data representation Not Precomputed Radiance Transform That is an algorithm that happens to use SHs Not actually that complex Just the maths language barrier Read the Ramamoorthi & Hanrahan paper – it makes sense eventually!

The Basics “Looking up a direction in the cube-map” A set of N basis functions – used by all SHs Components of the direction being looked up 1, X, Y, Z, XY, YZ, ZX, X2-Y2, 1-3Z2,… A set of N weights – this stores the data Each weight is usually an RGB colour Multiply each basis function by its weight And sum – result is the looked-up value Number of basis functions goes on for ever, but first 9 is good enough for diffuse lighting Weights are the equivalent of “pixels” – they store the actual image data, but they’re in frequency space.

Features of Harmonics Smooth over the sphere No seams or “hot spots” Easy to reconstruct in vector unit Rotation loses no data, has no aliasing Rotate then convert to SH is identical to convert to SH then rotate Basis functions used are orthonormal To add two SHs, add their weights To multiply two SHs, multiply their weights Explain verbally what “orthonormal” means –integral of any two different basis functions is 0 (“orthogonal”), integral of any basis function squared is 1 (“normal”) , which means during calculations lots of scary maths falls out to 0 or 1 and means that the add and mul Just Works.

Irradiance Concept Ignore SH for now – think of cubemaps Representing incoming distant lights Representing the diffuse cosine response Cosine response = max(0,N.L), summed for all lights Only normals matter, not positions Draw response to single light on cubemap The “picture” of a lit sphere Add together each light’s cubemap To shade any object, look up normal on the resulting cubemap

Irradiance Concept Get some better art! 

Irradiance Concept

Irradiance Concept

Using SH Final cubemap is smooth, low frequency Encode as SH weights Feed weights to vertex unit Vertex unit shades normals with SH 9 RGB values to feed to vertex unit rather than a cubemap 9 weights represents diffuse shading with maximum 6% error Works on hardware without cubemaps, and is faster even on hardware that does.

Using SH Better But wait! Why use a cubemap at all? Go from light description to SH weights Adding two SHs = add weights together So: For each light, find the SH weights Sum weights Feed final weights to vertex unit No cubemaps anywhere!

CPU Code per Object per Light void AddLightToSH ( Colour *sharm, Colour &colour, Vector &dirn ) { sharm[0] += colour * fConst1; sharm[1] += colour * fConst2 * dirn.x; sharm[2] += colour * fConst2 * dirn.y; sharm[3] += colour * fConst2 * dirn.z; sharm[4] += colour * fConst3 * (dirn.x * dirn.z); sharm[5] += colour * fConst3 * (dirn.z * dirn.y); sharm[6] += colour * fConst3 * (dirn.y * dirn.x); sharm[7] += colour * fConst4 * (3.0f * dirn.z * dirn.z - 1.0f); sharm[8] += colour * fConst5 * (dirn.x * dirn.x - dirn.y * dirn.y); }

Vertex Processor Code const Colour sharm[9]; Colour LightNormal ( Vector &dirn ) { Colour colour = sharm[0]; colour += sharm[1] * dirn.x; colour += sharm[2] * dirn.y; colour += sharm[3] * dirn.z; colour += sharm[4] * (dirn.x * dirn.z); colour += sharm[5] * (dirn.z * dirn.y); colour += sharm[6] * (dirn.y * dirn.x); colour += sharm[7] * (3.0f * dirn.z * dirn.z - 1.0f); colour += sharm[8] * (dirn.x * dirn.x - dirn.y * dirn.y); return colour; }

In Practice Find brightest n lights (n=1 to 3 usually) Shade those conventionally Specular, bumpmapped, self-shadowing, etc Rest of lights accumulated into SH Last real light fades between real and SH Provides smooth transition as object moves

Gotchas Only vertex diffuse lighting Only directional lights No bumpmaps (PS2.0 shaders can!) No specular, only diffuse So do brightest 2 lights normally Only directional lights Point lights must be approximated Discontinuities on abutting objects (walls, etc)

Cool Stuff Take (HDR) skybox, convert to SH offline Add to all computed SHs Perfect sunset lighting Artists can tweak by drawing on bitmaps! Presampled “radiosity” Pick some sample points in environment Render cubemaps at those points, convert to SH At runtime, for any object, linearly interpolate between nearest sample point SHs. Gives lighting off walls, through doorways, etc

Demo Compare many real lights to just 2 Too dark – lights are discarded. Compare many real lights to 2 + ambient Too washed out Compare many real lights to just SH Pretty good – slight differences Compare many lights to 2 real + SH Excellent, plus bumpmapping works

Questions

Conclusions SH is excellent for images on a sphere SH irradiance makes lights cheap Standard lighting has per-vertex cost Using SHs converts it to a per-object cost Per-vertex cost is constant Artists can put down far more lights And the game is still faster! (10-20% on PS2)

References Ramamoorthi & Hanrahan Peter-Pike Sloan Robin Green “An Efficient Representation for Irradiance Environment Maps” Peter-Pike Sloan Siggraph 2002 & 2003 papers Robin Green “Spherical Harmonic Lighting: The Gritty Details” Google for “spherical harmonic irradiance” Without “irradiance” you get lots of chemists