Streaming Compressed 3D Data on the Web using JavaScript and WebGL Université de Lyon, INSA-Lyon LIRIS UMR CNRS 5205 Guillaume Lavoué Florent Dupont Velvet Laurent Chevalier Web3D San Sebastian, Spain

2 3D data, an emerging multimedia content Visualization of 3D content through the Web is of interest for many industries. Video games, SciVis, CAD, E-commerce, online advertisement, Medical imaging, architecture, cultural heritage… Introduction

3 But…few web applications handle 3D data Why ? Huge data Low bandwith Heterogeneous devices Web browser An efficient web visualization system need to:   Remove/decrease the latency  Compression  Adapt the data to different network and devices  Levels of details / streaming capability  Comply with web browser constraints  Low complexity Introduction

430 Ko5Mo 4 A solution for streaming 3D graphics in a browser Two main components:  A dedicated progressive compression algorithm for 3D graphics.  A multi-thread JavaScript implementation of the associated decompression and streaming. Introduction Transmission Streamed decoding

5 V1 = (x 1, y 1, z 1 ) V2 = (x 2, y 2, z 2 ) V3 = (x 3, y 3, z 3 ) V4 = (x 4, y 4, z 4 ) …. F1 = (1, 3, 4) F2 = (1, 2, 4) F3 = (2, 3, 4) F4 = (1, 2, 3) …. Geometry Connectivity Attributes C1 = (r 1, g 1, b 1 ) C2 = (r 2, g 2, b 2 ) C3 = (r 3, g 3, b 3 ) C4 = (r 4, g 4, b 4 ) …. The 3D data we consider Introduction

Single-rate techniques Progressive techniques Recall on 3D Mesh compression 6 Compression rate Compressed Stream Compressed Stream Compression rate Quality of the intermediate Levels Of Details Introduction

7 Outline  Related Works  Web-based progressive compression  Web integration using WebGL and JavaScript  Experiments and comparisons Introduction – Related Works – Compression – Web integration – Experiments

8 Related works Progressive compression of 3D meshes [Hoppe 1996][Taubin et al. 1998][Cohen-Or et al. 1999][Pajarola et Rossignac, 2000][Alliez and Desbrun 2001][Gandoin and Devillers 2002][Peng and Kuo 2005 ][Valette et al. 2009][Peng et al. 2010][Cirio et al. 2010][Ahn et al. 2011][Kim et al.2001][Lee et al. 2012][Maglo et al. 2012] × do not comply with web constraints Remote 3D visualization on the Web  ParaViewWeb [Jomier et al. 2011]  ShareX3D [Jourdain et al. 2008], COVISE [Niebling, Kopecki 2010] × 3D data in ASCII format  huge size Compressed format for web delivery  X3Db, OpenCTM [Geelnard 2009], Google webgl-loader [Chun 2012], X3DOM BinaryGeometry [Behr et al. 2012]   Adaptive Quad Patches [Gobbetti et al. 2012] × 3D rendering on the server side × no streaming × constraints on the inputs Introduction – Related Works – Compression – Web integration – Experiments

9 Web-based 3D mesh compression A progressive compression scheme for 3D meshes Encoding 430 Ko5 Mo Decoding Compressed stream  Algorithm from [Alliez, Desbrun 2001] [Lee, Lavoué, Dupont, 2012]  Connectivity (C), geometry (G) and color (Cl) are encoded Introduction – Related Works – Compression – Web integration – Experiments.X3D.P3DW

Iterative simplification What information do I need to encode to reverse the process ? 10 The encoding process Introduction – Related Works – Compression – Web integration – Experiments Encoding Decoding

11 Encoded information  Encoding connectivity changes: Valences of removed vertices + null patch codes. 3 bits per value. (average of 10 bits/vertex).  Encoding vertex geometry: Global uniform quantization (Q bits) + simple binary encoding (X,Y,Z).  Encoding vertex color: RGB to L.a.b then 8 bits quantization. We have simplified existing algorithms [Alliez 2001] [Lee 2012]  No prediction, no arithmetic coding  simple quantization.  We loose some performance for gaining simplicity and speed. Introduction – Related Works – Compression – Web integration – Experiments

12 Implementation of the decompression We have implemented an Half-Edge data structure  Fast adjacency queries  Allow topological operations ( (create_center_vertex, join_facets, split_face…)  Based upon Three.js  Polyhedron_HalfEdge.js Decoding Topological and geometrical operations on the 3D mesh Introduction – Related Works – Compression – Web integration – Experiments

13 Implementation of the decompression Five main JavaScript components  The binary reader:  It decodes the connectivity, geometry and color information from the binary P3DW file.  Streamed reading and decoding (responseText property of the XHR).  Nothing on the server side.  The base mesh initialiser which constructs the base mesh.  The LoD decompressor which constructs the next level of details starting from already decompressed ones.  The rendering and user interaction management (Three.js + WebGL).  A multi-thread implementation (using Web Workers, Array Buffers). Minimization of the garbage collection Introduction – Related Works – Compression – Web integration – Experiments

14 Experiments and comparisons Compression ratio Model (#vertices)OFFZIPP3DW Neptune (250k)19,1846,788 (2.8)1,509 (12.7) Tank (160k)12,7622,980 (4,3) 1,390 (9.2) Venus (100k)7,1822,701 (2.7)609 (11.8) Monkey (50k)5,1051,856 (2.8)430 (11.9) Casting (5k) (3)28 (11.9) File size (kB) and associated reduction factor (in parenthesis) Model (#vertices)OFFP3DWX3DOMUTF8GZIP UTF8 Bunny (35k)2, Happy Buddha (540k)41,6234, Introduction – Related Works – Compression – Web integration – Experiments

15 Experiments and comparisons Quality of the levels of details Introduction – Related Works – Compression – Web integration – Experiments

16 Experiments and comparisons Decompression time Introduction – Related Works – Compression – Web integration – Experiments Model (#vertices)10%50%100% Neptune (250k) Tank (160k) Venus (100k) Monkey (50k) Casting (5k) Linear behavior Between 20k and 30k vertices per second Decompression time (in seconds, for a 2GHz laptop)

1500 ms3000 ms 6000 ms 17 Experiments and comparisons Comparison with concurrent state of the art Happy Buddha model (500K vertices) - 5 Mbit/s Introduction – Related Works – Compression – Web integration – Experiments Our method Google webgl-loader [Chun 2012] X3DOM BinaryGeometry [Behr et al. 2012] 800 ms

Conclusion 18 Contributions  A technical solution for remote streaming and visualization of compressed 3D content on the web.  Complex geometry processing operations are possible in JavaScript. Perspectives  Improving compression performance (prediction, entropy coding).  Extension to generic 3D models (non manifold, textures).  Integrating copyright protection or meta data hiding (watermarking).