1. Efficient Streaming of 3D Scenes with Complex Geometry and Complex Lighting Romain Pacanowski and M. Raynaud X. Granier P. Reuter C. Schlick P. Poulin INRIA Bordeaux University

2. Global illumination (indirect lighting) Increases realism of synthetic images Very long to compute unless using interactive/real-time techniques Motivation Global illumination for remote visualization systems

3. All lighting computations done on client Low data transfer requirements Rendering speed depends on scene geometric complexity  Motivation Client approach

4. Pre/compute indirect illumination Stream the indirect illumination to the client BUT: How to avoid an overhead transfer time proportional to the size of the geometry ? Need for an illumination representation not correlated to the geometry Motivation Server approach

5. Stochastic methods [ Purcell03,Gautron05, …] Fast but not real time  Depend on geometry  Radiosity methods [ Keller97, Segovia07 ] [ Dachsbacher07 ]: scene depth dependent  [ Laine07 ]: Real time Visual quality depends on geometric accuracy  Not suited for streaming context Previous Work Interactive/Real time global illumination

6. Concept: encode light transport effects in a structure [ Sloan02,Wang04,Pan07 ] Real time even with dynamic scenes Huge data size  Direct-to-indirect transfer [ Pellacini07 ] Data size is dependent on geometry complexity Previous Work Precomputed radiance transfer approaches

7. Most closely related to our work 3D regular grid [ Mitchell06 ] Irradiance values at vertices Geometric dependency of irradiance  Storage cost increases Previous Work Irradiance Volumes [Greger97]

8. New structure for indirect illumination Geometry independent GPU friendly Streaming technique for our lighting structure Client/Server visualization system Independent streaming of geometry and lighting Direct illumination on the client side Our Method Overview

9. Indirect Lighting Representation Overview Regular 3D grid 6 irradiance vectors at each vertex Directional interpolation To reconstruct irradiance for any normal Spatial interpolation Easily compressed GPU friendly

10. Indirect Lighting Representation Irradiance vector Irradiance Materials Reflected Radiance

11. Indirect Lighting Representation Irradiance vector directional interpolation

12. Colored irradiance vector for direction : 3x3 matrix Compression: Direction + Color If : no artefacts are introduced Indirect Lighting Representation GPU : Irradiance vector compression

13. Color 32 bits R9_G9_B9_E5 GPU compatible format RGBE [ Ward91 ] Direction XYZ: 24 bits (3x8 bits) (θ, ϕ ): 2x8 bits ([ Jensen96 ]) Quantization used to reduce the transfer size Indirect Lighting Representation GPU : Irradiance vector quantization

14. Regular grid 12x3D Textures 6 for direction 6 for color Format GL_RGB16F_ARB 6 texture fetches per pixel Native trilinear interpolation Indirect Lighting Representation GPU issues

15. Server Precomputes and stores Illumination grids LOD for 3D objects Stores : Materials Planar Surfaces Our Remote Visualization System Overview Client CPU processes: Geometry Lighting (Push-Pull) Direct Transfer Streaming

16. Geometry, and then Lighting Lighting, and then Geometry Interleave Geometry and Lighting Our Remote Visualization System Streaming strategies

17. Initialization: 8 corners Each client request: N samples per slice Not yet received data vertices Holes in data = black spots Our Remote Visualization System Irradiance vector grid streaming

18. Our Remote Visualization System Push-Pull : filling holes in the grid 2. 3D Hierarchical hole filling (PUSH) 1. 3D Hierarchy construction (PULL) 3.For each completed level => Pyramidal Filter

19. Our Remote Visualization System Push-Pull : Results Without Push-Pull With Push-Pull and Filtering

20. Adaptation of [ Melax98,Gueziec99 ] techniques Vertex split to get a multiresolution mesh Streaming : Vertices Vertex Indices Vertex lookup tables Mesh is globally updated Our Remote Visualization System Geometry streaming

21. Our remote system: Server: Intel Q6600 with 4GB RAM Client: Nvidia 8800GTX Network: Wifi 802.11g Results Independence of geometry and lighting

22. Results Streaming geometry with constant illumination

23. Results Streaming illumination with constant geometry

24. Results Interleave streaming of geometry and illumination

25. Results Transfer time for indirect illumination

26. Results Transfer time for indirect illumination

27. Results Transfer time for geometry

28. New structure to represent indirect lighting: 3D regular grid with irradiance vectors at vertices GPU friendly Small memory footprint and short transfer time overhead Independent of geometric complexity Easily integrated with geometry streaming Conclusion Summary

29. Server side Precomputation to fit cluster architectures On-line precomputation Fast update mechanism for dynamic 3D scenes Local recomputation in regions of important changes Client side: reducing the process time New push-pull process (GPU) Future Work

30. Questions ? Thank you for your attention