Frequency Domain Normal Map Filtering Charles Han Bo Sun Ravi Ramamoorthi Eitan Grinspun Columbia University

Normal Mapping (Blinn 78)

Normal Mapping (Blinn 78) Specify surface normals

Normal Mapping

A Problem… Multiple normals per pixel Undersampling Filtering needed ?

Supersampling Correct results Too slow

MIP mapping Pre-filter Normals do not interpolate linearly Blurring of details

Comparison supersampled MIP mapped

Representation a single vector is not enough how do we represent multiple surface normals?

Previous Work Gaussian Distributions –(Olano and North 97) –(Schilling 97) –(Toksvig 05) Mixture Models –(Fournier 92) –(Tan, et.al. 05) 3D Gaussian 2D covariance matrix 1D Gaussian mixture of Phong lobes mixture of 2D Gaussians no general solution

Our Contributions Theoretical Framework –Normal Distribution Function (NDF) –Linear averaging for filtering –Convolution for rendering –Unifies previous works New normal map representations –Spherical harmonics –von Mises-Fisher Distribution Simple, efficient rendering algorithms

Normal Distribution Function (NDF) Describes normals within region Defined on the unit sphere Integrates to one Extended Gaussian Image (Horn 84)

Normal Distribution Function NDF normal map

Normal Distribution Function NDF normal map

Normal Distribution Function NDF normal map

Normal Distribution Function NDF normal map

NDF Filtering normal map

NDF Filtering normal map

NDF Filtering NDF averaging is linear Store NDFs in MIP map

Rendering rendered image normal, pixel value lightsBRDF Radially symmetric BRDFs Lambertian: Blinn-Phong: Torrance-Sparrow: Factored:

Supersampling supersampled image samples Effective BRDF

samples Effective BRDF NDF,

Spherical Convolution Form studied in lighting –(Basri and Jacobs 01) –(Ramamoorthi and Hanrahan 01) Effective BRDF = convolution of NDF & BRDF

NDF Spherical Convolution Effective BRDF BRDF

Previous Work Gaussian Distributions –Olano and North (97) –Schilling (97) –Toksvig (05) Mixture Models –Fournier (92) –Tan, et.al. (05) Our Work 3D Gaussian 2D covariance matrix 1D Gaussian mixture of Phong lobes mixture of 2D Gaussians NDF representations spherical harmonics von Mises-Fisher mixtures

Spherical Harmonics Analogous to Fourier basis Convolution formula:

BRDF Coefficients Arbitrary BRDFs Cheaply represented –Analytic: compute in shader –Measured: store on GPU Easily changed at runtime

NDF Coefficients Store in MIP mapped textures Finest-level NDFs are delta functions, so: Use standard linear filtering

Effective BRDF Coefficients Product of NDF, BRDF coefficients Proceed as usual

Limitations Storage cost of NDF –One texture per coefficient –O( ) cost Limited to low frequencies

von Mises-Fisher Distribution (vMF) Spherical analogue to Gaussian Desirable properties –Spherical domain –Distribution function –Radially symmetric more concentratedless concentrated

Mixtures of vMFs NDF number of vMFs

Expectation Maximization (EM) From machine learning Used in (Tan et.al. 05) Fit model parameters to data data NDF model vMF Mixture EM

Rendering Convolution –Spherical harmonic coefficients –Analytic convolution formula Extensions to EM –Aligned lobes (Tan et.al. 05) –Colored lobes NDFrendered image

Conclusion Summary –Theoretical Framework –New NDF representations –Practical rendering algorithms Future directions –Offline rendering, PRT –Further applications for vMFs –Shadows, parallax, inter-reflections, etc.

Thanks! Tony Jebara, Aner Ben-Artzi, Peter Belhumeur, Pat Hanrahan, Shree Nayar, Evgueni Parilov, Makiko Yasui, Denis Zorin, and nVidia.