1. Web Algorithmics Dictionary-based compressors

2. LZ77 Algorithm’s step: Output Advance by len + 1 A buffer “window” has fixed length and moves aacaacabcaaaaaa Dictionary (all substrings starting here) aacaacabcaaaaaa c ac ac

3. LZ77 Decoding Decoder keeps same dictionary window as encoder. Finds substring and inserts a copy of it What if l > d? (overlap with text to be compressed) E.g. seen = abcd, next codeword is (2,9,e) Simply copy starting at the cursor for (i = 0; i < len; i++) out[cursor+i] = out[cursor-d+i] Output is correct: abcdcdcdcdcdce

4. Lempel-Ziv Algorithms Keep a “dictionary” of recently-seen strings. The differences are: How the dictionary is stored How it is extended How it is indexed How elements are removed LZ-algos are asymptotically optimal, i.e. their compression ratio goes to H(S) for n   !! No explicit frequency estimation

5. You find this at: www.gzip.org/zlib/

6. LZ77 Optimizations used by gzip LZSS: Output one of the following formats (0, position, length) or (1,char) Typically uses the second format if length < 3. Special greedy: possibly use shorter match so that next match is better Hash Table for speed-up searches on triplets Triples are coded with Huffman’s code

7. Web Algorithmics Some special compressors Spatial vs Temporal Locality

8.  code for integer encoding x > 0 and Length =  log 2 x  +1 e.g., 9 represented as.   code for x takes 2  log 2 x  +1 bits (ie. factor of 2 from optimal) Length-1 Optimal for Pr(x) = 1/2x 2, and i.i.d integers

9. It is a prefix-free encoding… Given the following sequence of  coded integers, reconstruct the original sequence: 0001000001100110000011101100111 8 63 597

10. Streaming compression Still you need to determine and sort all terms…. Can we do everything in one pass ? Move-to-Front (MTF): As a freq-sorting approximator As a caching strategy As a compressor Run-Length-Encoding (RLE): FAX compression

11. Move to Front Coding Transforms a char sequence into an integer sequence, that can then be var-length coded Start with the list of symbols L=[a,b,c,d,…] For each input symbol s 1) output the position of s in L 2) move s to the front of L Properties: Exploit temporal locality, and it is dynamic X = 1 n 2 n 3 n… n n  Huff = O(n 2 log n), MTF = O(n log n) + n 2 There is a memory

12. MTF: how good is it ? Encode the integers via  -coding: |  (i)| ≤ 2 * log i + 1 Put  in the front and consider the cost of encoding: By Jensen’s: No much worse than Huffman...but it may be far better

13. Run Length Encoding (RLE) If spatial locality is very high, then abbbaacccca => (a,1),(b,3),(a,2),(c,4),(a,1) In case of binary strings  just numbers and one bit Properties: Exploit spatial locality, and it is a dynamic code X = 1 n 2 n 3 n… n n  Huff(X) = O(n 2 log n) > Rle(X) = O( n (1+log n) ) There is a memory

14. Web Algorithmics Burrows-Wheeler Transform

15. The big (unconscious) step...

16. p i#mississi p p pi#mississ i s ippi#missi s s issippi#mi s s sippi#miss i s sissippi#m i i ssippi#mis s m ississippi # i ssissippi# m The Burrows-Wheeler Transform (1994) Let us given a text T = mississippi# mississippi# ississippi#m ssissippi#mi sissippi#mis sippi#missis ippi#mississ ppi#mississi pi#mississip i#mississipp #mississippi ssippi#missi issippi#miss Sort the rows # mississipp i i #mississip p i ppi#missis s FL T

17. A famous example Much longer...

18. Compressing L seems promising... Key observation: l L is locally homogeneous L is highly compressible Algorithm Bzip : Œ Move-to-Front coding of L  Run-Length coding Ž Statistical coder  Bzip vs. Gzip: 20% vs. 33%, but it is slower in (de)compression !

19. BWT matrix #mississipp i#mississip ippi#missis issippi#mis ississippi# mississippi pi#mississi ppi#mississ sippi#missi sissippi#mi ssippi#miss ssissippi#m #mississipp i#mississip ippi#missis issippi#mis ississippi# mississippi pi#mississi ppi#mississ sippi#missi sissippi#mi ssippi#miss ssissippi#m How to compute the BWT ? ipssm#pissiiipssm#pissii L 12 11 8 5 2 1 10 9 7 4 6 3 SA L[3] = T[ 7 ] We said that: L[i] precedes F[i] in T Given SA and T, we have L[i] = T[SA[i]-1]

20. How to construct SA from T ? # i# ippi# issippi# ississippi# mississippi pi# ppi# sippi# sissippi# ssippi# ssissippi# 12 11 8 5 2 1 10 9 7 4 6 3 SA Elegant but inefficient Obvious inefficiencies:  (n 2 log n) time in the worst-case  (n log n) cache misses or I/O faults Input: T = mississippi#

21. p i#mississi p p pi#mississ i s ippi#missi s s issippi#mi s s sippi#miss i s sissippi#m i i ssippi#mis s m ississippi # i ssissippi# m # mississipp i i #mississip p i ppi#missis s FL Take two equal L’s chars How do we map L’s onto F’s chars ?... Need to distinguish equal chars in F... Rotate rightward their rows Same relative order !! unknown A useful tool: L  F mapping

22. T =.... # i #mississip p p i#mississi p p pi#mississ i s ippi#missi s s issippi#mi s s sippi#miss i s sissippi#m i i ssippi#mis s m ississippi # i ssissippi# m The BWT is invertible # mississipp i i ppi#missis s FL unknown 1. LF-array maps L’s to F’s chars 2. L[ i ] precedes F[ i ] in T Two key properties: Reconstruct T backward: i p p i InvertBWT(L) Compute LF[0,n-1]; r = 0; i = n; while (i>0) { T[i] = L[r]; r = LF[r]; i--; }

23. RLE0 = 03141041403141410210 An encoding example T = mississippimississippimississippi L = ipppssssssmmmii#pppiiissssssiiiiii Mtf = 020030000030030200300300000100000 Mtf = [i,m,p,s] # at 16 Bzip2-output = Arithmetic/Huffman on |  |+1 symbols...... plus  (16), plus the original Mtf-list (i,m,p,s) Mtf = 030040000040040300400400000200000 Alphabet |  |+1 Bin(6)=110, Wheeler’s code

24. You find this in your Linux distribution