1. Huffman Coding Yancy Vance Paredes

2. Outline Background Motivation Huffman Algorithm Sample Implementation Running Time Analysis Proof of Correctness Application

3. Background Lossless compression where around 20% to 90% of savings in space Developed by David A. Huffman Published in 1952

4. Motivation /1 Let’s say we want to store the string: go go gophers (13 characters) How do we usually do it? – ASCII – 7 bits + 1 more bit 13 * 8 bits = 104 bits – Reduce it? 8 unique characters: g, o, p, h, e, r, s, space Instead of 8 bits, we can lower it to 3 bits 13 * 3 bits = 39 bits We saved 65 bits!

5. Motivation /2 What if we lessen the number of bits for frequent characters? CharacterFrequencyCodeTotal Bits g3006 o3016 p11003 h110104 r110114 e111004 s111014 Space21116

6. Motivation /3 The total number of bits used is lowered to 37 – Prefix Code Easy to encode and decode 0001111000111100011001010110010111101

7. Motivation /4 How do we decode? – 0 means go LEFT – 1 means go RIGHT How to decode the following? 0001111000111100011001010110010111101

8. How to Decode? 0001111000111100011001010110010111101 CharacterCode g00 o01 p100 h1010 r1011 e1100 s1101 Space111

9. Huffman Algorithm A greedy algorithm Constructs an optimal prefix code – Huffman code HUFFMAN(C) n = |C| Q = C for i = 1 to n-1 allocate a new node z z.left = x = EXTRACT_MIN(Q) z.right = y = EXTRACT_MIN(Q) z.freq = x.freq + y.freq INSERT(Q,z) return EXTRACT_MIN(Q)

10. Sample Implementation See program demo

11. Running Time Analysis Assume that Q is implemented as a min heap (priority queue) Building the Q takes O(n) The for loop executes n-1 times – The heap operations contribute O(lg n) – Thus, the loop contributes O(n lg n) – Total running time is O(n lg n)!

12. Proof of Correctness /1 Show that the problem of determining an optimal prefix code exhibits the following properties: – Greedy choice – Optimal substructure

13. Proof of Correctness /2 To compute the cost of a tree:

14. Proof of Correctness /3 Greedy choice

15. Proof of Correctness /4 Optimal substructure

16. Application Commonly used as the back-end of some multimedia codecs – JPEG, MP3

17. Summary Background Motivation Huffman Algorithm Sample Implementation Running Time Analysis Proof of Correctness Application

18. References Chapter 13: Greedy Algorithm