1. View-Dependent Precomputed Light Transport Using Nonlinear Gaussian Function Approximations Paul Green 1 Jan Kautz 1 Wojciech Matusik 2 Frédo Durand 1 MIT CSAIL 1 MERL 2 ACM Symposium in Interactive 3D Graphics 2006 View-Dependent Precomputed Light Transport Using Nonlinear Gaussian Function Approximations

2. Geometry & Viewpoint All-Frequency Lighting Rendered Frame Reflectance Interactive 6D Relighting

3. Goal: 6D Relighting  High-quality view-dependent effects Sharp highlights  Spatially varying BRDFs  Allow rendering w/ large environment maps (e.g. 6x256x256)  Without paying a prohibitive data storage price

4. Applications  Games  Architectural Visualization

5. Outline  Background / Previous Work PRT Nonlinear Approximation  Our Representation Rendering Fitting Results  Conclusion

6. Courtesy of Sloan et al. 2003 Exit Radiance Distant Radiance Incident Radiance Shadowing Inter-reflection Transport function maps distant light to incident light Can Include BRDF if outgoing direction ω o is fixed Precomputed Light Transport

7. Courtesy of Sloan et al. 2003  Radiance L o at point p along direction is weighted sum of distant radiance L i Outgoing Radiance Transport Vector Distant Radiance (Environment Map) Light Transport

8. Example Transport Function Environment Map BRDF Weighted Incident Radiance Exit Radiance (outgoing color) It’s a Dot Product Between Lighting and Transport Vectors!!

9. Light Transport  Data Size Problem Many GB’s of data Rendering is slow  6x64x64 cubemap ~24,000 mults/vert  Can reduce size in different basis: Spherical Harmonics Wavelets Zonal Harmonics

10. PRT with Spherical Harmonics  Precomputed Radiance Transfer [Sloan et al 02,03] Low Order Spherical Harmonics Soft Shadows and Low Frequency Lighting Not Suitable For Highly Glossy Materials Not Practical For High Frequency Lighting Image from slides by Ng et al.2003

11. Nonlinear Wavelet PRT  Nonlinear Wavelet Lighting Approximation [Ng et al 03] Haar Wavelets Nonlinear Approximation All Frequency Lighting Fixed View For Arbitrary BRDFs Image from slides by Ng et al.2003

12. Separable PRT  Factor BRDF into product of view-only and light-only functions [Liu et al 04, Wang et al 04]  Nonlinear Wavelet Approximation [Ng et al 03]  Need factorization per BRDF  Very specular materials still require many coefficients Liu et al 04

13. Our Goal: 6D Relighting  High quality view-dependent effects  Representation of transport that enables Spatially varying BRDFs Arbitrary highlight scale compact storage High-res environment maps (e.g. 6x256x256)  Sparse view sampling  Requires high-quality interpolation Over view directions Over mesh triangles

14. Outline  Background / Previous Work PRT Nonlinear Approximation  Our Representation Rendering Fitting Results  Conclusion

15. Factoring Transport Represent with SH or Wavelets View-independent (diffuse) View-dependent Nonlinear Gaussian Function Approximation ? Per view

16. Our Nonlinear Representation - mean- std. deviation Sum of N isotropic Gaussians

17. Previous work: Nonlinear Wavelet  Nonlinear: Approximating basis set depends on input  Truncate small coefficients  Effect of coefficients is still linear Linear: First 8 Coefficients SSE = 140.2 Nonlinear: 8 Largest Coefficients SSE = 25.1

18. Our solution: Even More Nonlinear  We don’t start from linear basis  No Truncation of coefficients  Nonlinear parameter estimation  Parameters have nonlinear effects SSE = 5.61 Nonlinear sum of 2 Gaussians Nonlinear: 8 Largest Coefficients SSE = 25.1

19.  Arbitrary freq bandwidth  Accurate approx w/small storage  Good interpolation  Good visual quality Advantages of sum of Gaussians original N = 1 N = 2 Haar 70 terms

20. Examples p

21. Rendering Approximated Transport Lighting ?

22. Gaussian Pyramid Larger σ Pre-convolve environment with Gaussians of varying sizes Only done Once Can start with large cubemap e.g. 6x256x256

23. Rendering Tri-linear lookup in Gaussian Pyramid Exit Radiance (outgoing color) Approximated Transport Lighting

24. Rendering Novel Views  Precomputed Transport Functions for sparse set of outgoing directions  Naïve solution: (Gouraud Shading) Interpolate Outgoing Radiance  Cross-fading artifacts p ?

25. Better: Interpolate Parameters  Interpolate Gaussian parameters Mean, Std. Dev, Weights  Analogous to Phong vs Gouraud shading p ? t=0t=1t=0.5

26. Per-Pixel Interpolation

27. Interpolation Drawbacks  Visibility may not interpolate correctly But is usually plausible  Correspondences Makes fitting more difficult View-independent View-dependent

28. Data Fitting  Precompute Raw Transport Data  Solve large scale nonlinear optimization problem For each vertex  For each view Fit Gaussians to transport data p

29. Nonlinear Optimization Objective has terms for  Fitting the Data  Regularization Angular smoothness and correspondences Spatial smoothness and correspondences original N = 2 N=1N=1 - p

30. Error Plots

31. Results  6x256x256 Environment Maps  Single Gaussian  2.8 GHz P4  1GB RAM  Nvidia 6800 Ultra  Software screen capture

32. Contributions  New parametric representation of PLT Compact Storage High Quality Interpolation of parameters Sparse View Sampling Efficient Rendering Spatially varying BRDFs original N = 2Haar

33. Questions?