1. Binary Compression Rates for ASCII Formats Martin IsenburgJack Snoeyink University of North Carolina at Chapel Hill

2. “Geometry compression does not require binary.” “This enables full conformance between ASCII and binary X3D.” Take this home: “We can store compressed geometry as ASCII and achieve benchmark compression rates.”

3. Overview Motivation Benchmark Compressor Storing Bits as ASCII Results More Motivation Demo Summary

4. Motivation

5. Why ASCII Scenes? authors like text-based 3D – wide acceptance – read, understand, modify, and save scene with any text editor – Web3D Developer survey: “very important” no binary standard rather readable ASCII than binary

6. Why Compression? authors want compressed 3D – smaller files – faster download – Web3D Developer survey: “very important” data-heavy nodes – audio / images / video – geometry

7. Approaches to Compression binary formats: – proprietary compression schemes – automatically “protects” the content – Shockwave3D, Viewpoint, Virtue3D, Cult3D... gzipped text-based formats: – human readable ASCII content – WRL, X3D, OBJ, PLY, SMF, OFF…

8. Compression Standards software to read, edit, and write content is available and widely used binary compression standards: – audio data:MP3 – image data: JPEG, GIF – movie data:MPEG – geometry data:?

9. Why no Standard yet? ( 1 )  structure of 3D data is complex audio:sequence of numbers image: block of numbers video: sequence of blocks of numbers geometry:vertex positions + triangles (or polygons ?) + texcoords (1, 2, 4, 8 layers ?) + normals (smoothing groups ?) + colors (per-face ? per-vertex ?) + bones (1, 2, 3 attachments ?)

10. Why no Standard yet? ( 2 )  3D data is much less used audio:everybody image: everybody video: everybody geometry:anybody ?  3D data is not a consumable product on its own  profitability issues my mom, dad, sister, aunt, uncle, grandma, friends, neighbors, … gamers, researchers

11. Geometry compression & X3D long wanted feature ( `96 ) – compression requires binary VRML – Compressed Binary Format workgroup – binary VRML + 5 new compressed nodes but … CBF proposal refused ( `98 ) – problem-child: the binary specification second call for BF proposals ( `00 ) – unanswered

12. Compression without Binary  Web3D 2002 paper – eliminate binary requirement – only requires specification changes where it matters – does not interfere with how authors publish content – does not affect (overall) readability Coding Polygon Meshes as Compressable ASCII [ Isenburg & Snoeyink, 02 ]

13. Design Objective work with Shout3D API pure Java viewer ( plugin-less ) fast, light-weight decoding – low computational complexity – small size for decoder class be worthwhile less than 400 lines of code 5,381 bytes 1 : 6

14. ASCII coder binary coder How good is the compression? – compare to benchmark coder – oops … New Design Objective: >>> Maximum Compression <<< – produce ASCII output – compress with “gzip -9” – be as good as benchmark

15. Benchmark Compressor

16. Connectivity Coder – codes connectivity as sequence of vertex degrees Geometry Coder – codes geometry as sequence of corrective vectors Triangle Mesh Compression [ Touma & Gotsman, 98 ]  only handles fully triangulated meshes  no support for texture coordinates

17. Polygon Mesh Compressor Compressing Polygon Mesh Connectivity with Degree Duality Prediction [ Isenburg, 02 ] Compressing Polygon Mesh Geometry with Parallelogram Prediction [ Isenburg & Alliez, 02 ] handling non-triangular meshes: Compressing the Property Mapping of Polygon Meshes [ Isenburg & Snoeyink, 01 ] Compressing Texture Coordinates with Selective Linear Predictions [ Isenburg & Snoeyink, 03 ] handling textured meshes:

18. It’s available! a working Web implementation Java applet based on node set of Shout3D WRL, OBJ, PLY, OFF, SMF, … compress any model online

19. lion wolf raptor … horse cat dog average 42.6 18.4 21.5 20.3 22.5 21.5 55% 60% 62% 101% 77% 61% 74% Binary vs. ASCII binary scene difference ASCII 66.2 29.4 34.9 40.9 39.9 34.6 [ results from Web3D’02 paper ]

20. Mesh Compression Process grow region + store symbols

21. Why is binary so much better? Because it … is binary? stores symbols in binary? uses better predictive encoding scheme? compresses symbols with entropy coding? allows to store a bit-stream? No! Not quite! Yes! Sort of. Yes, but …

22. Entropy Coders for a symbol sequence of t types # of type t pi =pi =  i = 1 t Entropy = p i log 2 ( ) bits pipi 1 # total 2.0 bits 1.3 bits 0.2 bits

23. Contents of Binary File an array of bits [ 0 0 1 1 0 1 0 0 1 1 1 1 0 … 1 0 0 0 1 0 1 0 ] six floats x min =-1.2323y min =-2.1324z min =-1.7123 x max =2.6731y max =0.8372z max =3.4971 an integer precision = 12

24. Shape { appearance Appearance { material Material { diffuseColor 1.5 0 } } geometry IndexedFaceSet { creaseAngle 0.9 coord Coordinate { point [ -0.0715 1.7609... -0.4479 -1.5153 1.5304 ] } coordIndex [ 7 6 209 204 -1 4 … 4577 4223 4222 -1 ] } } triceratops.wrl

25. Shape { appearance Appearance { material Material { diffuseColor 1.5 0 } } geometry CompressedIndexedFaceSet { creaseAngle 0.9 code [ 0 0 1 1 1 0 1 1 0 1 1 1 0 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 1 1 1 0 1 1 1 0 … 1 0 0 0 1 1 1 0 1 1 0 1 1 0 1 1 0 ] bits 12 box [ -1.2323 -2.1324 -1.7123 2.6731 0.8372 3.4971 ] } } triceratops_compressed.wrl

26. Shape { appearance Appearance { material Material { diffuseColor 1.5 0 } } geometry CompressedIndexedFaceSet { creaseAngle 0.9 code “ AQg+N9gOd+Rhh+lXdfzKEyz0CCZUJHLs3+ fWpsTQFxortsk8XW8j7jEZwL … vXS7GVbHHUuX ” bits 12 box [ -1.2323 -2.1324 -1.7123 2.6731 0.8372 3.4971 ] } } triceratops_compressed.wrl

27. Storing Bits as ASCII

28. Design Goals 1.resulting ASCII should be “safe” other possibilities: 2 …. 96 chars obvious choices: –uuencode –base64 –Hexbin represent binary files with a set of 64 ASCII characters

29. Design Goals 1.resulting ASCII should be “safe” 2.conversion should be simple efficient implementation by using 2 k ASCII characters –value of k bits –look-up table [ 0 0 1 1 0 1 0 0 1 1 1 1 0 1 … ] LT = { ‘A’, ‘B’, ‘C’, ‘D’, ‘E’, ‘F’, ‘G’, ‘H’, … …, ‘+’, ‘=’ } LT [ 6 ] LT [ 19 ] …

30. Design Goals 1.resulting ASCII should be “safe” 2.conversion should be simple 3.ASCII representation should be compact expansion factor: 8 / log 2 (n) number of ASCII chars 32 chars= 60.0 % 64 chars= 33.0 % 96 chars= 21.5 %

31. Design Goals 1.resulting ASCII should be “safe” 2.conversion should be simple 3.ASCII representation should be compact 4.size of ASCII representation of bits after “gzipping” should be close to original

32. Results

33. fish.wrl Shape { appearance Appearance { material Material { modulateTextureWithDiffuse true diffuseColor 1 1 1 } texture ImageTexture { url fish.jpg } } geometry IndexedFaceSet { creaseAngle 0.9 coord Coordinate { point [ -0.0715 4.7609 6.3930... -0.4479 -4.5153 4.5304 ] } coordIndex [ 7 6 209 204 -1 4 217 … 4577 -1 4577 4223 4222 -1 ] texCoord TextureCoordinate { point [ 0.3735 0.9441 0.3289 … 0.2666 0.4990 0.1082 ] } texCoordIndex [ 0 1 2 3 -1 4 5 6 7 … 4311 -1 4311 4293 4683 -1 ] } }

34. fish_compressed.wrl Shape { appearance Appearance { material Material { modulateTextureWithDiffuse true diffuseColor 1 1 1 } texture ImageTexture { url fish.jpg } } geometry CompressedIndexedFaceSet { creaseAngle 0.9 code “AQg+N9gOd+Rhh+lXdfzK20Eyz0CCZUJHMK8EKLs3QrP+ fWpsTQFxo4dPVVZ0pGK720hVQw3a9xXrtsk8XW4d8j7jEDfP7pC hVQw3a9xXrtPTsk8XW8j7jEDfP7pCgEofTPoTnfUYfTWzzAbc9eh Fji6UL0ZwL … vXS7GVbHHUuX“ bits 12 box [ -1.9725 -7.6861 -10 1.9725 7.6861 10 ] bits_tex 10 box_tex [ 0.0146 0.0749 0.9896 0.9834 ] } }

35. lion wolf raptor … horse cat dog average IFS scene ratio comp IFS 1,360 569 586 749 791 586 57 25 29 27 30 29 1:24 1:23 1:20 1:28 1:26 1:20 1:25 resulting file sizes [ without gzipping the text file ]

36. lion wolf raptor … horse cat dog average IFS scene ratio comp IFS 442 183 200 266 267 186 43 19 22 21 23 22 1:10 1:9 1:13 1:11 1:8 1:11 resulting file sizes [ with gzipping the text file ]

37. binary lion42.643.21.3 % wolf18.418.71.8 % raptor21.521.91.7 % … horse 20.3 20.6 1.7 % cat 22.5 22.8 1.6 % dog21.521.91.7 % average 1.7 % scene difference ASCII Binary vs. ASCII

38. More Motivation

39. Scene.x3d “statue” IFS T “mona lisa” IMG binary ASCII binary Compression ASCII “frame” IFS T Scene.bx3d “living room” “frame” “mona lisa” IFS T IMG “statue” IFST asc2binbin2asc “living room” IFS.gz bin2asc ? Scene.bx3d “living room” “frame” “mona lisa” IFS T IMG “statue” CIFST compress 12 bits Scene.x3d “living room” “frame” “mona lisa” IFS T IMG “statue” CIFST compress 12 bits asc2bin.gz

40. Complete Conformance compressed nodes exist in ASCII and binary going back and forth between the two is a very simple mapping same decompression algorithm used – no matter if compressed node stored in ASCII or binary

41. Demo > local web <localweb

42. Summary

43. Summary ( 1 ) completely independent things: – ASCII format – binary format – use of compressed geometry nodes store compressed nodes in ASCII as compact as if stored in binary ( after gzipping ) two alternative representations author’s decision at publishing time

44. Summary ( 2 ) provide “bit-field” in ASCII formats same decompression algorithm for ASCII and binary version full conformance between ASCII and binary version of X3D --- --- including compressed nodes  design/decide compression now without waiting for a binary format !!!

45. Thank you.

47. Example XML node CompressedIndexedFaceSet : IndexedFaceSet { MFbyte [out] code NULLIC SFint32 [out] bit [ 1..24 ]IC SFint32 [out] bit_tex [ 1..16 ]IC SFint32 [out] bit_col [ 1..16 ]IC SFint32 [out] bit_nor [ 1..16 ]IC SFvec6f [out] box [ 1..24 ]IC SFvec4f [out] box_tex [ 0 0 1 1 ]IC SFvec6f [out] box_col [ 0 0 0 1 1 1 ] IC SFbool [] valid [ false ] IC }