1. PRT Summary

2. Motivation for Precomputed Transfer better light integration and light transport –dynamic, area lights –shadowing –interreflections in real-time point light area light area lighting, no shadows area lighting, shadows

3. Precomputed Radiance Transfer (PRT) represent lighting using spherical harmonics low frequency = few coefficients n=4n=9n=25 n=676original

4. Tabulating PRT Lighting Basis i Lighting Basis i+1 Lighting Basis i+2 illuminate store coefs on surface...... PRT is a linear operator (matrix) per surface point maps incident lighting to exit radiance

5. Self-Transfer Results (Diffuse) No Shadows/Inter Shadows Shadows+Inter No Shadows/Inter Shadows Shadows+Inter

6. Self-Transfer Results (Glossy) No Shadows/Inter Shadows Shadows+Inter No Shadows/Inter Shadows Shadows+Inter

7. PRT Terminology

11. PRT as a Linear Operator l : light vector (in source basis) M p : source-to-exit transfer matrix e p : exit radiance vector (in exit basis) y(v p ) : exit basis evaluated in direction v p e p (v p ) : exit radiance in direction v p

12. PRT Special Case: Diffuse Objects transfer vector rather than matrix independent of view (constant exit basis) matrix is row vector previous work uses different light bases image relighting [PRT02]SH [Xi03]Directional [Ng03]Haar [Ashikhmin02]Steerable

13. PRT Special Case: Surface Light Fields transfer vector rather than matrix frozen lighting environment matrix is column vector [Miller98] [Nishino99] [Wood00] [Chen02] [Matusik02]

14. Factoring PRT (BRDFs) T p : source → transferred incident radiance R p : rotate to local frame B : integrate against BRDF [Westin92] y ( v p ) e p : evaluate exit radiance at v p

15. Hemispherical Projection exit radiance is defined over hemisphere, not sphere spherical harmonics not orthogonal over hemisphere how to project hemispherical functions using SH? –naïve projection assumes “underside” is zero –least squares projection minimizes approximation error see appendix

16. Factoring PRT (BRDFs) TechniqueLightBExitBNote [Sloan02] SH Phong [Kautz02] SHDirArb [Lehtinen03] SHDirLsq [Matusik02] Dir IBR

17. Extending PRT to BSSRDFs already handled by original equation use [Jensen02], only multiple scattering (matrix with only 1 row) mix with “conventional” BRDF

18. Problems With PRT Big matrices at each surface point –25-vectors for diffuse, x3 for spectral –25x25-matrices for glossy –at ~50,000 vertices Slows glossy rendering (4hz) –Frozen View/Light can increase performance –Not as GPU friendly Limits diffuse lighting order –Only very soft shadows

19. Compression Goals Decode efficiently –As much on the GPU as possible –Render compressed representation directly Increase rendering performance –Make non-diffuse case practical Reduce memory consumption –Not just on disk

20. Compression Example Surface is curve, signal is normal

21. Compression Example Signal Space

22. VQ Cluster normals

23. VQ Replace samples with cluster mean

24. PCA Replace samples with mean + linear combination

25. CPCA Compute a linear subspace in each cluster

26. CPCA Clusters with low dimensional affine models How should clustering be done? Static PCA –VQ, followed by one-time per-cluster PCA –optimizes for piecewise-constant reconstruction Iterative PCA –PCA in the inner loop, slower to compute –optimizes for piecewise-affine reconstruction

27. Static vs. Iterative

28. Related Work VQ+PCA [Kambhatla94] (static) VQPCA [Khambhatla97] (iterative) Mixture PC [Dony95] (iterative) More sophisticated models exist –[Brand03], [Roweis02] –Mapping to current GPUs is challenging Variable storage per vertex Partitioning is more difficult (or requires more passes)

29. Equal Rendering Cost VQPCACPCA

30. Rendering with CPCA

31. Constant per cluster – precompute on the CPU Rendering is a dot product Compute linear combination of vectors Only depends on # rows of M

32. Non-Local Viewer Assume: v p constant across object (distant viewer) Rendering independent of view & light orders - linear combination of colors

33. Rendering = + +

34. Overdraw 1 1 1 3 3 2 2 2 2 2 2 2 2 2 2 2 2 2 faces belong to 1-3 clusters OD = 1  face drawn once OD = 2  face drawn 2x OD = 3  face drawn 3x coherence optimization: reclassification superclustering

35. Pixel Shader TextureConstants GPU Dataflow Vertices Vertex Shader Exit Rad.

36. Demo

37. Results Model#PtsSPCAIPCAFPS Buddha49.9k3m30s1h51m27 BuddhaSS49.9k6m12s4h32m27 Bird Anis48.7k6m34s3h43m45 Bird Diff48.7k43s3m26s227 Head50k4m20s2h12m58.5 All examples have 25x25 matrices, 256 clusters, 8 PCA vectors

38. Conclusions CPCA works in “signal space”, not “surface space” uses affine subspace per-cluster compresses PRT well is used directly without “blowing out” signal requires small, uniform state storage provides –faster rendering –higher-frequency lighting

39. Future Work time-dependent and parameterized geometry higher-frequency lighting combination with bi-scale rendering better signal continuity

40. Questions? DirectX SDK for PRT available soon. Jason Mitchell, Hugues Hoppe, Jason Sandlin, David Kirk Stanford, MPI for models