Abdullah Aldahami ( ) April 6,

2  Huffman Coding is a simple algorithm that generates a set of variable sized codes with the minimum average size.  Huffman codes are part of several data formats as ZIP, MPEG and JPEG.  The code is generated based on the estimated probability of occurrence.  Huffman coding works by creating an optimal binary tree of nodes, that can be stored in a regular array.

3  The method starts by building a list of all the alphabet symbols in descending order of their probabilities (frequency of appearance).  It then construct a tree from bottom to top.  Step by step, the two symbols with the smallest probabilities are selected; added to the top.  When the tree is completed, the codes of the symbols are assigned.

4  Example: circuit elements in digital computations  Summation of frequencies (Number of events) is 40 CharacterFrequency i6 t5 space4 c3 e3 n3 u2 l2 CharacterFrequency m2 s2 a2 o2 r1 d1 g1 p1

5  Example: circuit elements in digital computations r1d1g1p1 2 u2l2m2 2 a2o2s c3e3n3‘ 4 7 t5 7 i

6  So, the code will be generated as follows: CharacterFrequencyCode Length Total Length i t space c e n u l CharacterFrequencyCode Length Total Length m s a o r d g p  Total is 154 bits with Huffman Coding compared to 240 bits with no compression

7 Input Symbol it‘ ’cenul Probability P(x) Output Code Code length (in bits) (L i ) Weighted path length L i ×P(x) Optimality Probability budget (2 -L i ) 1/8 1/16 1/81/32 Information of a Message I(x) = – log 2 P(x) Entropy H(x) =-P(x) log 2 P(x) Entropy is a measure defined in information theory that quantifies the information of an information source. The measure entropy gives an impression about the success of a data compression process.

8 Input Symbol msaordgp Sum Probability P(x) = 1 Output Code Code length (in bits) (L i ) Weighted path length L i ×P(x) Optimality Probability budget (2 -L i ) 1/321/16 1/64 1/32 = 1 Information of a Message I(x) = – log 2 P(x) Entropy H(x) =-P(x) log 2 P(x) Bit/sym The sum of the probability budgets across all symbols is always less than or equal to one. In this example, the sum is equal to one; as a result, the code is termed a complete code. Huffman coding approaches the optimum on 98.36% = (3.787 / 3.85) *100

9  Static probability distribution (Static Huffman Coding)  Coding procedures with static Huffman codes operate with a predefined code tree, previously defined for any type of data and is independent from the particular contents.  The primary problem of a static, predefined code tree arises, if the real probability distribution strongly differs from the assumptions. In this case the compression rate decreases drastically.

10  Adaptive probability distribution (Adaptive Huffman Coding)  The adaptive coding procedure uses a code tree that is permanently adapted to the previously encoded or decoded data.  Starting with an empty tree or a standard distribution.  This variant is characterized by its minimum requirements for header data, but the attainable compression rate is unfavourable at the beginning of the coding or for small files.

11