Efficient Streaming of 3D Scenes with Complex Geometry and Complex Lighting Romain Pacanowski and M. Raynaud X. Granier P. Reuter C. Schlick P. Poulin INRIA Bordeaux University

Global illumination (indirect lighting) Increases realism of synthetic images Very long to compute unless using interactive/real-time techniques Motivation Global illumination for remote visualization systems

All lighting computations done on client Low data transfer requirements Rendering speed depends on scene geometric complexity  Motivation Client approach

Pre/compute indirect illumination Stream the indirect illumination to the client BUT: How to avoid an overhead transfer time proportional to the size of the geometry ? Need for an illumination representation not correlated to the geometry Motivation Server approach

Stochastic methods [ Purcell03,Gautron05, …] Fast but not real time  Depend on geometry  Radiosity methods [ Keller97, Segovia07 ] [ Dachsbacher07 ]: scene depth dependent  [ Laine07 ]: Real time Visual quality depends on geometric accuracy  Not suited for streaming context Previous Work Interactive/Real time global illumination

Concept: encode light transport effects in a structure [ Sloan02,Wang04,Pan07 ] Real time even with dynamic scenes Huge data size  Direct-to-indirect transfer [ Pellacini07 ] Data size is dependent on geometry complexity Previous Work Precomputed radiance transfer approaches

Most closely related to our work 3D regular grid [ Mitchell06 ] Irradiance values at vertices Geometric dependency of irradiance  Storage cost increases Previous Work Irradiance Volumes [Greger97]

New structure for indirect illumination Geometry independent GPU friendly Streaming technique for our lighting structure Client/Server visualization system Independent streaming of geometry and lighting Direct illumination on the client side Our Method Overview

Indirect Lighting Representation Overview Regular 3D grid 6 irradiance vectors at each vertex Directional interpolation To reconstruct irradiance for any normal Spatial interpolation Easily compressed GPU friendly

Indirect Lighting Representation Irradiance vector Irradiance Materials Reflected Radiance

Indirect Lighting Representation Irradiance vector directional interpolation

Colored irradiance vector for direction : 3x3 matrix Compression: Direction + Color If : no artefacts are introduced Indirect Lighting Representation GPU : Irradiance vector compression

Color 32 bits R9_G9_B9_E5 GPU compatible format RGBE [ Ward91 ] Direction XYZ: 24 bits (3x8 bits) (θ, ϕ ): 2x8 bits ([ Jensen96 ]) Quantization used to reduce the transfer size Indirect Lighting Representation GPU : Irradiance vector quantization

Regular grid 12x3D Textures 6 for direction 6 for color Format GL_RGB16F_ARB 6 texture fetches per pixel Native trilinear interpolation Indirect Lighting Representation GPU issues

Server Precomputes and stores Illumination grids LOD for 3D objects Stores : Materials Planar Surfaces Our Remote Visualization System Overview Client CPU processes: Geometry Lighting (Push-Pull) Direct Transfer Streaming

Geometry, and then Lighting Lighting, and then Geometry Interleave Geometry and Lighting Our Remote Visualization System Streaming strategies

Initialization: 8 corners Each client request: N samples per slice Not yet received data vertices Holes in data = black spots Our Remote Visualization System Irradiance vector grid streaming

Our Remote Visualization System Push-Pull : filling holes in the grid 2. 3D Hierarchical hole filling (PUSH) 1. 3D Hierarchy construction (PULL) 3.For each completed level => Pyramidal Filter

Our Remote Visualization System Push-Pull : Results Without Push-Pull With Push-Pull and Filtering

Adaptation of [ Melax98,Gueziec99 ] techniques Vertex split to get a multiresolution mesh Streaming : Vertices Vertex Indices Vertex lookup tables Mesh is globally updated Our Remote Visualization System Geometry streaming

Our remote system: Server: Intel Q6600 with 4GB RAM Client: Nvidia 8800GTX Network: Wifi g Results Independence of geometry and lighting

Results Streaming geometry with constant illumination

Results Streaming illumination with constant geometry

Results Interleave streaming of geometry and illumination

Results Transfer time for indirect illumination

Results Transfer time for indirect illumination

Results Transfer time for geometry

New structure to represent indirect lighting: 3D regular grid with irradiance vectors at vertices GPU friendly Small memory footprint and short transfer time overhead Independent of geometric complexity Easily integrated with geometry streaming Conclusion Summary

Server side Precomputation to fit cluster architectures On-line precomputation Fast update mechanism for dynamic 3D scenes Local recomputation in regions of important changes Client side: reducing the process time New push-pull process (GPU) Future Work

Questions ? Thank you for your attention