1. Representing Sets (2.3.3) Huffman Encoding Trees (2.3.4)
Lecture 11
Representing Sets (2.3.3)
Huffman Encoding Trees (2.3.4)
מבוא מורחב - שיעור 11

2. The Abstract Data Type Set
A set is a collection of distinct items. The following are allowable set operations.
(empty-set? set)
(make-empty-set)
(element-of-set? x set)
(adjoin-set x set)
(union-set s1 s2)
(intersection-set s1 s2)
Contract: the operations have the usual meaning of set operations.
Set  Boolean
 Set
Item × Set  Boolean
Item × Set  Set
Set × Set  Set
מבוא מורחב - שיעור 11

3. Implementing Sets Must decide on a Representation
How a set is represented.
Then must write an implementation
Write the code of the operations.
Each operation can be written separately
We will take a look at several alternative representations/implementations
מבוא מורחב - שיעור 11

4. Version 1: Unordered List
Representation: a set is represented as a list. No duplicates are allowed.
Implementation:
(define (empty-set? set)(null? set))
(define (make-empty-set)'())
מבוא מורחב - שיעור 11

5. Version 1: element-of-set?
(define (element-of-set? x set)
(cond ((null? set) #f)
((equal? x (car set)) #t)
(else (element-of-set? x (cdr set)))))
equal? : Like eq? for symbols. Compares the contents of lists and trees. Can be applied for numbers and strings.
(eq? (list 'a 'b) (list 'a 'b))
(equal? (list 'a 'b) (list 'a 'b))
 #f
 #t
מבוא מורחב - שיעור 11

6. Version 1: Adjoin-set (define (adjoin-set x set)
(if (element-of-set? x set)
set
(cons x set)))
מבוא מורחב - שיעור 11

7. Version 1: Intersection
(define (intersection-set set1 set2)
(cond ((or (null? set1) (null? set2)) '())
((element-of-set? (car set1) set2)
(cons (car set1)
(intersection-set (cdr set1) set2)))
(else (intersection-set (cdr set1) set2))))
Or is this better?
(define (intersection-set set1 set2)
(cond ((or (null? set1) (null? set2)) '())
((element-of-set? (car set1) set2)
(adjoin-set (car set1)
(intersection-set (cdr set1) set2)))
(else (intersection-set (cdr set1) set2))))
מבוא מורחב - שיעור 11

8. Version 1: Union Is the alternative better this time?
(define (union-set set1 set2)
(cond ((null? set1) set2))
((not (element-of-set? (car set1) set2))
(cons (car set1)
(union-set (cdr set1) set2)))
(else (union-set (cdr set1) set2))))
Is the alternative better this time?
(define (union-set set1 set2)
(cond ((null? set1) set2))
(else
(adjoin-set (car set1)
(union-set (cdr set1) set2)))))
מבוא מורחב - שיעור 11

9. Version 1: Time Complexity
Suppose a set contains n elements
Element-of-set
Adjoin-set
Intersection-set
Union-set
(n)
(n)
(n2)
(n2)
מבוא מורחב - שיעור 11

10. Version 2: Ordered List
Representation: a set (of numbers) is represented as an ordered list without duplicates.
(define (element-of-set? x set)
(cond ((null? set) #f)
((= x (car set)) #t)
((< x (car set)) #f)
(else (element-of-set? x (cdr set)))))
Time complexity: (n)
You will implement adjoin-set yourself.
empty-set? and make-empty-set are the same.
מבוא מורחב - שיעור 11

11. Version 2: Intersection
intersection-set from version 1 will work here.
(define (intersection-set set1 set2)
(cond ((or (null? set1) (null? set2)) '())
((element-of-set? (car set1) set2)
(cons (car set1)
(intersection-set (cdr set1) set2)))
(else (intersection-set (cdr set1) set2))))
But, its complexity is (n2). Can we do it better ?
מבוא מורחב - שיעור 11

12. Version 2: Better Intersection
(define (intersection-set set1 set2)
(if (or (null? set1) (null? set2))
'()
(let ((x1 (car set1)) (x2 (car set2)))
(cond ((= x1 x2)
(cons x1
(intersection-set (cdr set1)
(cdr set2))))
((< x1 x2)
(intersection-set (cdr set1) set2))
(else
(intersection-set set1 (cdr set2)))))))
מבוא מורחב - שיעור 11

13. Version 2: Intersection Example
set set intersection
( ) ( ) (1
(3 7 9) (4 6 7) (1
(7 9) (4 6 7) (1
(7 9) (6 7) (1
(7 9) (7) (1
(9) () (1 7)
Time and space  (n)
Union -- similar
מבוא מורחב - שיעור 11

14. Complexity unordered ordered empty-set? make-empty-set element-of-set
adjoin-set
intersection-set
union-set
(1)
(1)
(1)
(1)
(n)
(n)
(n)
(n)
(n2)
(n)
(n2)
(n)
מבוא מורחב - שיעור 11

15. Version 3: Binary Trees Binary Search: Lion in the desert
מבוא מורחב - שיעור 11

16. Version 3: Binary Trees
Store the elements in the nodes of a binary tree.
The values in the left subtree of a node v are all smaller than the value stored at v.
The values in the right subtree of a node v are all larger than the value stored at v. 7 3 1 5 9 12
A possible representation of the set {1,3,5,7,9,12} :
מבוא מורחב - שיעור 11

17. Version 3: Binary Trees A set has many representations: 7 3 1 5 9 12 3
Height= (log n)
Balanced Tree
Unbalanced Tree
מבוא מורחב - שיעור 11

18. Version 3: Representation
(define (make-tree entry left right)
(list entry left right))
(define (entry tree)
(car tree))
(define (left-branch tree)
(cadr tree))
(define (right-branch tree)
(caddr tree)) 7 3 1 5 9 12
מבוא מורחב - שיעור 11

19. Version 3: Element-of-set
(define (element-of-set? x set)
(cond ((null? set) #f)
((= x (entry set)) true)
((< x (entry set))
(element-of-set? x (left-branch set)))
(else
(element-of-set? x (right-branch set)))))
Complexity: (h), where h is the height of the tree.
If tree is balanced, then h  log(n)
In the worst case, h  n
מבוא מורחב - שיעור 11

20. Version 3: Adjoin-set
(define (adjoin-set x set)   (cond ((null? set) (make-tree x '() '()))         ((= x (entry set)) set)         ((< x (entry set))          (make-tree (entry set)                      (adjoin-set x (left-branch set))                     (right-branch set)))         (else          (make-tree (entry set)                     (left-branch set)                     (adjoin-set x (right-branch set))))))
Complexity: (h), where h is the height of the tree.
מבוא מורחב - שיעור 11

21. If a tree is roughly balanced, then h  log(n).
Complexity
We omit the trivial operations.
unordered
ordered
trees
Element-of-set
Adjoin-set
Intersection-set
Union-set
(n)
(n)
(h)
(n)
(n)
(h)
(n2)
(n)
(n)
(n2)
(n)
(n)
If a tree is roughly balanced, then h  log(n).
Main challenge: Keep the trees roughly balanced. (More on this next term in “Data Structures”.)
מבוא מורחב - שיעור 11

22. Random trees are fairly balanced
(define (rand-tree n range)
(if (= n 0) '()
(adjoin-set (random range)
(rand-tree (- n 1) range))))
(define (height tree)
(if (null? tree) 0
(+ 1 (max (height (left-branch tree))
(height (right-branch tree))))))
(height (rand-tree )) 
(height (rand-tree )) 
~ 22
~ 31
average over several runs.
height of a balanced tree: ~ 10 , ~13 resp.
מבוא מורחב - שיעור 11

23. Huffman encoding trees
מבוא מורחב - שיעור 11

24. Data Transmission “sos” Bob Alice We wish to send information
efficiently from Alice to Bob
Morse code not necessarily the most
efficient you could think of
מבוא מורחב - שיעור 11

25. Fixed Length Codes Represent data as a sequence of 0’s and 1’s
Sequence: BACADAEAFABBAAAGAH
A fixed length code (ASCII):
A B C D 011
E F G H 111
Encoding of sequence:
The Encoding is 18x3=54 bits long. Can we make the encoding shorter?
מבוא מורחב - שיעור 11

26. Variable Length Code
Make use of frequencies. Frequency of A=8, B=3, others 1.
A B C D 1011
E F G H 1111
Example: BACADAEAFABBAAAGAH
42 bits (20% shorter)
Morse code is a variable length code. We can encrypt more efficiently if we give frequent symbols E is the most frequent word and so is represented by a single dot. Top decode, we can use a special separator code as in Morse code.
But how do we decode?
מבוא מורחב - שיעור 11

27. Prefix code  Binary tree
Prefix code: No codeword is a prefix of any other codeword
A B C D 1011
E F G H 1111 A B C D E F G H 1
מבוא מורחב - שיעור 11

28. Decoding Example 10001010 10001010 B 10001010 BA 10001010 BAC 1 A B CF G H 1 B BA
BAC
מבוא מורחב - שיעור 11

29. Abstract representation of code trees
Constructors:
make-leaf Construct a leaf
make-code-tree - Construct a code tree
Predicates:
leaf? Is leaf?
Selectors:
left-branch Select left branch
right-branch - Select right branch
symbol-leaf the symbol attched to leaf
מבוא מורחב - שיעור 11

30. Decoding a Message (define (decode bits tree)
(define (decode-one bits current-branch)
(if (null? bits)
'()
(let ((next-branch
(choose-branch (car bits) current-branch)))
(if (leaf? next-branch)
(cons (symbol-leaf next-branch)
(decode-one (cdr bits) tree))
(decode-one (cdr bits) next-branch)))))
(decode-one bits tree))
(define (choose-branch bit branch)
(cond ((= bit 0) (left-branch branch))
((= bit 1) (right-branch branch))
(else (error "bad bit -- CHOOSE-BRANCH" bit))))
מבוא מורחב - שיעור 11

31. Huffman Tree = Optimal Length CodeB C D E F G H 1 8 3
Before we can describe the algorithm for generating optimal codes, lets talk a little bit about our representation.
Optimal: no code has better weighted average length
מבוא מורחב - שיעור 11

32. Representation 17 A 9 5 4 B 2 2 2 C D E F G H {A,B,C,D,E,F,G,H} 83
{C,D}
{E,F}
{G,H} C D E F G H 1 1 1 1 1 1
מבוא מורחב - שיעור 11

33. Representation (Cont.)
(define (make-leaf symbol weight)
(list 'leaf symbol weight))
(define (leaf? object)
(eq? (car object) 'leaf))
(define (symbol-leaf x) (cadr x))
(define (weight-leaf x) (caddr x))
מבוא מורחב - שיעור 11

34. Representation (Cont.)
(define (make-code-tree left right)
(list left
right
(append (symbols left) (symbols right))
(+ (weight left) (weight right))))
(define (left-branch tree) (car tree))
(define (right-branch tree) (cadr tree))
מבוא מורחב - שיעור 11

35. Representation (Cont.)
(define (symbols tree)
(if (leaf? tree)
(list (symbol-leaf tree))
(caddr tree)))
(define (weight tree)
(weight-leaf tree)
(cadddr tree)))
מבוא מורחב - שיעור 11

36. Huffman’s Algorithm
Build tree bottom-up, so that lowest weight leaves are farthest from the root.
Repeatedly:
Find two trees of lowest weight.
merge them to form a new tree whose weight is the sum of their weights.
מבוא מורחב - שיעור 11

37. Construction of Huffman tree17
{A,B,C,D,E,F,G,H} 9
{B,C,D,E,F,G,H} 5
{B,C,D} 4
{E,F,G,H} 2
{C,D} 2
{E,F} 2
{G,H} A B C D E F G H 8 3 1
מבוא מורחב - שיעור 11

38. Construction of Huffman Tree
Initial leaves {(A 8) (B 3) (C 1) (D 1) (E 1) (F 1) (G 1) (H 1)}
Merge {(A 8) (B 3) ({C D} 2) (E 1) (F 1) (G 1) (H 1)}
Merge {(A 8) (B 3) ({C D} 2) ({E F} 2) (G 1) (H 1)}
Merge {(A 8) (B 3) ({C D} 2) ({E F} 2) ({G H} 2)}
Merge {(A 8) (B 3) ({C D} 2) ({E F G H} 4)}
Merge {(A 8) ({B C D} 5) ({E F G H} 4)}
Merge {(A 8) ({B C D E F G H} 9)}
Final merge {({A B C D E F G H} 17)}
מבוא מורחב - שיעור 11

39. Construction of Huffman tree
(generate-huffman-tree '((A 8) (B 3) (C 1) (D 1)
(E 1) (F 1) (H 1) (G 1))
((leaf a 8) ((((leaf g 1) (leaf h 1) (g h) 2) ((leaf f 1) (leaf e 1) (f e) 2) (g h f e) 4) (((leaf d 1) (leaf c 1) (d c) 2) (leaf b 3) (d c b) 5) (g h f e d c b) 9) (a g h f e d c b) 17)
left-branch
right-branch
symbols
weight
מבוא מורחב - שיעור 11

40. Construction of Huffman tree
(define (generate-huffman-tree pairs)
(successive-merge (make-leaf-set pairs)))
Sort pairs
(define (make-leaf-set pairs)
(if (null? pairs)
'()
(let ((pair (car pairs)))
(adjoin-set (make-leaf (car pair)
(cadr pair))
(make-leaf-set (cdr pairs))))))
Ordered insert
Set of leaves, represented as lists ordered by weight.
מבוא מורחב - שיעור 11

41. Construction of Huffman tree
Need a variation of the ordered adjoin-set (since was written only for sets of numbers)
(define (adjoin-set x set)
(cond ((null? set) (list x))
((< (weight x) (weight (car set)))(cons x set))
(else (cons (car set)
(adjoin-set x (cdr set))))))
(define (successive-merge trees)
(if (null? (cdr trees))
(car trees)
(let ((smallest (car trees))
(2smallest (cadr trees))
(rest (cddr trees)))
(successive-merge
(adjoin-set
(make-code-tree smallest 2smallest)
rest)))))
מבוא מורחב - שיעור 11

42. Summary
We saw today that even a seemingly simple abstract concept like a set, when implemented on a computer, can give rise to many implementations, each with a different complexity.
מבוא מורחב - שיעור 11