1. IKI 10100: Data Structures & Algorithms Ruli Manurung (acknowledgments to Denny & Ade Azurat) 1 Fasilkom UI Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 Huffman Coding

2. 2 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 Data Compression Huffman Coding Outline

3. 3 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 Compression Process: Encoding: raw  compressed Decoding: compressed  raw Types of compression Lossy: MP3, JPG Lossless: ZIP, GZ Compression Algorithm: RLE: Run Length Encoding Lempel-Ziv Huffman Encoding Performance of compression depends on file types.

4. 4 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 Huffman Compression “If a woodchuck could chuck wood!” 32 char  8 bit = 256 bits 13 distinct characters  4 bit Compressed code: 128 bits Variable length string of bits to further improve compression. Using prefix codes Main idea: Frequently occurring letters: short representation. Infrequent letters: long representations.

5. 5 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 i a u e 85 3 3 Huffman Encoding: Comparison a = 00  16 bits i = 01  10 bits u = 10  6 bits e = 11  6 bits Total : 42 bits

6. 6 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 Huffman Encoding: Comparison i 5 a 8 u 3 e 3 6 11 19 a = 0  8 bits i = 10  10 bits u = 110  9 bits e = 111  9 bits Total: 36 bits

7. 7 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 ! a l u d k w If h c o space Huffman Encoding

8. 8 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 13 6 3 4 9 2 32 19 10 7 4 2 !:1 a:1 l:1 u:3 d:3 k:2w:2 I:1 f:1 h:2 c:5 o:5 space:5 Huffman Encoding

9. 9 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 Huffman Encoding (freq) ! = 0000 (1) I= 10000 (1) a = 00010 (1) f= 10001 (1) l= 00011 (1) h= 1001 (2) u= 001 (3) c= 101 (5) d= 010 (3) space= 110 (5) k= 0110 (2) o= 111 (5) w= 0111 (2) Cost:  d i * f i = 111 bits = 44%  256 bits

10. 10 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 Huffman Encoding: steps c 5 o 5 5 u 3 d 3 w 2 k 2 a 1 l 1 f 1 I 1 h 2 ! 1 2

11. 11 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 Huffman Encoding: steps c 5 o 5 5 u 3 d 3 w 2 k 2 f 1 I 1 h 2 ! 1 al 2 2

12. 12 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 Huffman Encoding: steps c 5 o 5 5 u 3 d 3 w 2 k 2 I 1 h 2 al 2 f! 2 3

13. 13 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 c 5 o 5 5 u 3 d 3 w 2 k 2 h 2 al 2 I f! 3 4 Huffman Encoding: steps

14. 14 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 c 5 o 5 5 u 3 d 3 k 2 al 2 I f! 3 4 wh 4 Huffman Encoding: steps

15. 15 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 Huffman Encoding: steps c 5 o 5 5 u 3 d 3 I f! 3 4 wh k al 4 6

16. 16 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 Huffman Encoding: steps c 5 o 5 5 f! 3 4 w h k al 4 u d 6 I

17. 17 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 c 5 o 5 5 f! 3 4 w h k al 4 u d 6 I 7 Huffman Encoding: steps

18. 18 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 Huffman Encoding: steps c 5 o 55 4 wh u d 6 f! k al I 7 9

19. 19 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 o 55 u d 6 f! k al I 7 c wh 9 10 Huffman Encoding: steps

20. 20 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 u d 6 f! k al I 7 c wh 9 o 10 13 Huffman Encoding: steps

21. 21 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 c wh 9 o 10 u d f! k al I 13 19 Huffman Encoding: steps

22. 22 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 u d f! k al I 13 c wh o 19 32 Huffman Encoding: steps

23. 23 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 Huffman Encoding: steps u d f! k al I c wh o Total: 111 bits

24. 24 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 How do we implement this? Use a priority queue! Maintain a forest of trees Each element in the queue is a root node Order them by character frequency count Algorithm: Add all unique characters as single node tree Call dequeue twice, merge trees, enqueue result Repeat until only 1 tree left!

25. 25 Ruli Manurung (Fasilkom UI)IKI10100: Lecture3 rd May 2007 Huffman encoding uses character frequency information to compress a file. The most frequent character gets a shorter prefix code, and vice versa. Summary