1. XML Compression Aslam Tajwala Kalyan Chakravorty

2. Overview Motivation for XML Compression Techniques for achieving XML compression XMill XMill Architecture

3. Why Compress XML? Structured nature of XML makes it understandable to humans, Downside: XML is verbose –Each non-empty element tag must end with a matching closing tag -- data –Ordering of tags is often repeated in a document (e.g. multiple records)

4. Why Compress XML?: 2 XML documents are text-based: well-known compression schemes such as Huffman and LZ can be easily applied Can gain a significant savings from compression, due to highly structured nature of XML XML is being used more frequently in real-time applications (e.g. web service-based e-commerce applications); increasing interest in finding ways to reduce overall size of XML documents

5. Using Huffman/LZ Usually some degree of repetition in an XML document (multiple occurrences of tags, attribute or data values) Compression schemes like Huffman and LZ can use this repetition to achieve some degree of compression

6. Using Huffman/LZ: 2 Many existing (and efficient) implementations of these algorithms are readily available (e.g. gzip) Downside is that these techniques aren’t fully capable of exploiting the structure of XML to achieve greater compression

7. Huffman Encoding Example ACDABA Since these are 6 characters, this text is 6 bytes or 48 bits long tree is build that replaces the symbols by shorter bit sequences. In this particular case, the algorithm would use the following substitution table: A=0, B=10, C=110, D=111 01101110100 (ACDABA = 11 bits)

8. LZ77 Example( Dictionary Based Compressors) Lempel-Ziv 77 algorithm Dictionary is a portion of encoded sequence The encoder examines the input sequence through a sliding window The window consists of two parts: –a search buffer that contains a portion of the recently encoded sequence, and –a look-ahead buffer that contains the next portion of the sequence to be encoded.

9. XMill (Liefke and Suciu, 2000) Relies heavily on zlib, the compression library used in gzip Also defines a few data type specific compressors; user-defined compressors can be added using SCAPI (Semantic Compressor API) During compression, each XML tag is examined to see which compression technique(s) should be applied

10. XML Compression View XML as a tree Separate the tree structure and what is stored in leaves Save the tree structure so that it can be restored The compressed file may or may not remember the tree structure breadfruit tree

11. XMill: Compression Strategy XMill applies 3 principles during compression: –Separate structure (element tags and attribute names) from data –Group related data items in a single container; compress each container separately –Apply appropriate semantic compressors to each container

12. XMill – Separating Structure From Content Start tags and attribute names are dictionary-encoded (as T1, T2, etc.) End tags replaced with ‘/’ token Data values replaced with their container number

13. XMill – Separating Structure From Content 2 Homer Simpson Frank Grimes Dictionary T1 =>Employees T2 => Employee T3 => @id Structure Container T1 T2 T3 C3 / C4 / T2 T3 C3 / C4 / / C3 1 2 C4 Homer Simpson Frank Grimes

14. XMill: Container Expressions Users can override default settings using the container expression language –Specify container membership –Specify which semantic compressor(s) are applied for each container E.g. to indicate all ‘Name’ and ‘Location’ tags should be grouped in the same container: xmill –p //(Name | Location) employees.xml

15. XMill: Semantic Compressors CompressorDescription tDefault Text Compressor (gzipped only) uCompressor for positive integers (binary encoded using 1 – 4 bytes) iCompressor for integers u8Compressor for positive integers < 256 diDifferential compressor for integers

16. XMill: Semantic Compressors 2 CompressorDescription rlRun-length encoder (store single copy of a sequence, its length, and repetition count) eEnumeration encoder (dictionary) “…”Constant compressor – outputs nothing: used to check that current token is a specified constant value

17. XMill: Semantic Compressors 3 Text compressor is applied to each element by default User can add other instructions via command line: xmill –p //price=>i file.xml Applies integer compressor to each occurrence of ‘price’ element in file.xml

18. XMill Architecture (1/3)

19. XMill Architecture (2/3) SAX Parser –sends tokens to the path processor. Path Processor –determines how to map data values to containers. Semantic Compressors –compresses the input and copies it to the container – in the memory window. –E.x. binary encoding of integers, differential compressors. When the window is filled, all containers are gzipped, stored on disk, and the compression resumes.

20. Performance Evaluation (1/2)

21. Performance Evaluation (2/2)

22. References XMill:An efficent Compressor for XML Data XGrind:A query friendly compressor www.cs.washington.edu/homes/ suciu/COURSES/590DS/19compression.ppt

23. Questions ?