1. IIIT Hyderabad Real-time Streaming and Rendering of Terrains Soumyajit Deb (Microsoft Research India, Banglore), Shiben Bhattacharjee, Suryakant Patidar, P. J. Narayanan (CVIT, IIIT Hyderabad)

2. IIIT Hyderabad Why Stream terrains? Central Storage of Data Bulky models, important data Support Heterogeneous Clients Easy updates Low client wait time Dynamism Model Size: 540 million samples Uncompressed Size: Over 350MB Time to transfer on 512kbps: Over 1 hour Wait time for streaming: Less than 1 minute

3. IIIT Hyderabad Challenges No Temporal Partitioning Low Bandwidth and Changing Latency Varied client types Dynamic Entities Client Synchronization Collaboration between users High bandwidth 3 mbps Low bandwidth 256 kbps Central Server Low-end Client High-end Client

4. IIIT Hyderabad Content Serving Video/Movies FilesMusic Photo Album ServingStreaming Faithful Reproduction Real-time Delivery

5. IIIT Hyderabad Previous Work Terrain Rendering 1.Geometry Clipmaps - Lossaso, Hoppe (Siggraph 04) 1 2.ROAM - Duchaineau et al (VIS 97) 2 3.Strip Masks – Pouderoux, Marvie (CGIT Australia 2005) 3 4.Terrain Geomorphing – Wagner (Shader X2 2003) 4 3 2 1 4

6. IIIT Hyderabad Previous Work… Geometry Streaming 1.ARTE - Schneider, Martin (Computers & Graphics 2003) 1 2.Streaming Remote Walkthroughs - Teler (EG01) 2 3.Protected 3D Rendering - Levoy et al (Siggraph 04) 3 4.Windows Live Local 4 and Google Earth/Maps Design of a Geometry Server. Deb & Narayanan. ICVGIP 2004 Streaming Terrain Rendering. Deb, Narayanan & Bhattacharjee. Sketch, SIGGRAPH 2006 3 4 1 2

7. IIIT Hyderabad Requirements Provide best frame-rate in all situations Adapt to the client capabilities –Computation & rendering power, memory, etc. Adapt to the connection parameters –Bandwidth, latency, fluctuations Adapt to user motion Handle dynamic objects Transparent to user –Remotely served objects are just like local ones –Modify them, mix them.

8. IIIT Hyderabad Client & Network: Summary Connection Bandwidth LowMediumHigh Low Low LoD Low to medium LoD Low to high LoD Medium Low LoD Medium LoD Medium to high LoD High Low LoD Medium LoD High LoD Client Rendering Capability

9. IIIT Hyderabad Resources, Issues and Solutions Client Capabilities Network Performance Highest quality rendering Interactive Frame-rates Latency Immunity Progressive Refinement Level of Detail Latency dependent Prediction and Prefetching Client Side Visibility Culling Optimal use Of Bandwidth Multiple Height map Resolutions Client side Caching Speed based optimization Compression

10. IIIT Hyderabad Terrain Streaming: Overview Server and Client modules exchange data User program connects to client module Server Module Client Module Scene Database Client Module Client Module Terrain Renderer Terrain Renderer Terrain Renderer Network Client 1 Client 2 Client 3

11. IIIT Hyderabad Terrain Representation Terrain represented by equal sized tiles For a 2 n x2 n terrain, m discrete LODs m ≤ n Lower LODs obtained by dropping alternative heights or averaging them SxSx SxSx SxSx SxSx l=0 l=1l=2l=3

12. IIIT Hyderabad View Frustum Culling Terrains don’t require 3D VFC. Project view frustum on base plane and find tiles inside bounding box For a tile B t Level of Detail L L = floor (d t / l) Baseline L=0 L=1 L=2 Rejected Tile l Projection of View Frustum dtdt

13. IIIT Hyderabad Smooth LOD transition To avoid popping up artifacts, we use smooth LOD transition by blending the two LODs with a factor The height of a point h is defined by: h = h (2i,2j) l (1 - α) + h (i,j) (l+1) α For a tile B t Level of Detail L and blending factor α L = floor (d t / l) α = frac (d t / l) Baseline L=0 L=1 L=2 Rejected Tile l Projection of View Frustum dtdt

14. IIIT Hyderabad Terrain Streaming Basics Tile Transmission –Multiple LODs –Residue Compression Tile Selection –Potentially Visible Tile detection Object Selection –Object LOD is selected based on anchored tile LOD

15. IIIT Hyderabad Performance Optimization Speculative Prefetching – Prefetch tiles on predicted path of motion Client side Caching – Cache tiles around viewer’s vicinity. We utilize a Least Potentially Visible scheme for cache replacement

16. IIIT Hyderabad Dynamic Entities Dynamic Entities change form/position/appearance Dynamic VE – addition and deletion of entities Multiple client sync to maintain consistent state Use Lazy Updates

17. IIIT Hyderabad Client side modifications Client created annotations Stored at server, treated as dynamic entities Potential for sharing annotations across users

18. IIIT Hyderabad Transmission Data Reduction Original Model 350 MB Visibility Detection 20 MB Client Side Caching 0.8 MB PTC Compression 300KB LOD Optimization 2 MB Typical Numbers for each request.

19. IIIT Hyderabad Demo/Video

20. IIIT Hyderabad Graphs Clients with Lower capability and/or network bandwidth receive less data. The highest detail tiles contribute substantially to data rate

21. IIIT Hyderabad Graphs Both clients maintain steady frame rate in both bandwidth settings. Quality Factor improves with higher available bandwidth

22. IIIT Hyderabad Conclusions and Future Work Efficient terrain streaming over low bandwidths Support for dynamic entities and environments Supports collaboration between clients In Future we will support in real-time in place terrain editing and deformations Use programmable GPU for high performance at clients end

23. IIIT Hyderabad Th4nk y0u

24. IIIT Hyderabad Geometry Server: Highlights Server Module: –Visibility computation, LoD selection, client book-keeping, dynamic object updation Client Module: –Server interface, user program interface, caching, LPV visibility, user motion prediction Transparent Serving: –Open SceneGraph objects, add/modify remote object to local scenegraph. Dynamic Objects: –Server informs changes, lazy updation

25. IIIT Hyderabad Server-side Communication module LOD Optimizer Speed based Optimizer Texture Optimizer Server-side Communication Network Module Client Tracker Client Notifier Scene Database Renderer Control Program User Interface Network Module Client Interfaces Predictor Client Cache Vertex & Texture Visibility Culler Server ModuleClient Module User Program Server-side Processing Module Geometry Server Geometry Server Design

26. IIIT Hyderabad Quantifying System Performance Walkthrough performance is directly proportional on achieved quality and frame rate. Network performance is directly proportional to bandwidth utilization. We define τ qual and τ net empirically. We use simple linear models with scope for improved analysis in future.

27. IIIT Hyderabad LPV Scheme A B C D H E F G I Y Axis X Axis GHDIFBCEA H F C D G I A B E XvXrXl Yv Yr Yl GHDIFBCEA H F C D G I A B E Yv Yr X-List Y-List

28. IIIT Hyderabad Requirements Content optimization based upon available bandwidth and client capabilities High quality rendering Adaptability to changing latencies Progressive refinement / graceful degradation Continuous connection monitoring

29. IIIT Hyderabad Tile Stitching Smooth transition between level l and l+1 Finally, we stitch the corners of each tile to remove discontinuity