1. Streaming QSplat: A Viewer for Networked Visualization of Large, Dense Models Szymon Rusinkiewicz Marc Levoy Stanford University

2. Motivation (1) Range scanning makes it possible to create 3D models of interesting, complex objectsRange scanning makes it possible to create 3D models of interesting, complex objects Fast networking is becoming common, even at homeFast networking is becoming common, even at home How to stream high-resolution 3D models across the net?

3. Motivation (2) Under what conditions has streaming been studied?Under what conditions has streaming been studied? Low speed (e.g. modem) (topological surgery, Metacreations, RealityWave) Maximum compression at any cost High speed (e.g. local disk) (architectural walkthrough, terrain flythrough) H ide presence of streaming Medium speed (e.g. DSL) M anaging user interaction

4. Streaming QSplat Based on QSplat, a multiresolution point rendering systemBased on QSplat, a multiresolution point rendering system Low costLow cost – No special preprocessing for streaming – No server support – use standard web server – Low CPU and graphics requirements for client Flexible enough to allow exploration of user interface issuesFlexible enough to allow exploration of user interface issues

5. Streaming QSplat Review of QSplat data structure and rendering algorithmReview of QSplat data structure and rendering algorithm Modifying algorithm for network streamingModifying algorithm for network streaming User interface:User interface: – Color coding – Download order – Magnifying glass

6. Review of QSplat An interactive viewer for large models (10 8 – 10 9 samples)An interactive viewer for large models (10 8 – 10 9 samples) Fast startup and progressive loadingFast startup and progressive loading Maintains interactive frame rateMaintains interactive frame rate Compact data structureCompact data structure Fast preprocessingFast preprocessing

7. QSplat Data Structure Key observation: a single bounding sphere hierarchy can be used forKey observation: a single bounding sphere hierarchy can be used for – Hierarchical frustum and backface culling – Level of detail control – Splat rendering

8. QSplat Node Structure Position and Radius Tree Structure Normal Width of Cone of Normals Color (Optional) 13 bits3 bits14 bits2 bits16 bits 6 bytes

9. QSplat Rendering Algorithm Traverse hierarchy recursivelyTraverse hierarchy recursively if (node not visible) Skip this branch else if (leaf node) Draw a splat else if (size on screen < threshold) Draw a splat else Traverse children Hierarchical frustum / backface culling Point rendering Adjusted to maintain desired frame rate Level of detail control

10. Addition of Network Streaming if (node not visible) Skip this branch else if (leaf node) Draw a splat else if (size on screen < threshold) Draw a splat else if (any child not present) Draw a splat Place children on request queue else Traverse children

11. Addition of Network Streaming New data structures:New data structures: – Availability mask – which nodes or groups of nodes are present on the client – Request queue for parts of the model New thread:New thread: – Make requests for regions of the file – Listen for responses – Update availability mask

12. Availability Mask What granularity?What granularity? – Small granularity permits requests for precisely the visible regions – Large granularity necessary for memory efficiency and to avoid overhead of many tiny requests – Current implementation uses 1k blocks

13. Request Queue Prioritized queuePrioritized queue – priorities determine download order Cleared on every frameCleared on every frame – View-dependent streaming

14. Color Coding Color coding for visualization of relative resolution present in different areasColor coding for visualization of relative resolution present in different areas

15. Demo – St. Matthew 3D scan of 2.7 meter statue at 0.25 mm3D scan of 2.7 meter statue at 0.25 mm 102,868,637 points102,868,637 points File size: 644 MBFile size: 644 MB Preprocessing time: 1 hourPreprocessing time: 1 hour Network bandwidth: limited to 384 kbpsNetwork bandwidth: limited to 384 kbps

16. Choice of Streaming Order Choices for streaming order:Choices for streaming order: 1.Level within hierarchy 2.Size of node in world coordinates 3.Projected screen-space size 4.Projected y coordinate 5.Combination of #4 and #5 Coarse ordering based on quantized log(splat size)Coarse ordering based on quantized log(splat size) Fine ordering based on projected y coordinateFine ordering based on projected y coordinate

17. Magnifying Glass Provide tools for flexible control over download orderProvide tools for flexible control over download order – Example: “magnifying glass”

18. Conclusions Streaming QSplat permits interactive display of large, complex models across a networkStreaming QSplat permits interactive display of large, complex models across a network Given today’s network speeds, need more research on user interaction techniques for streaming 3D dataGiven today’s network speeds, need more research on user interaction techniques for streaming 3D data