1. Princeton University COS 423 Theory of Algorithms Spring 2002 Kevin Wayne Binary and Binomial Heaps These lecture slides are adapted from CLRS, Chapters 6, 19.

2. 2 Priority Queues Supports the following operations. n Insert element x. n Return min element. n Return and delete minimum element. n Decrease key of element x to k. Applications. n Dijkstra's shortest path algorithm. n Prim's MST algorithm. n Event-driven simulation. n Huffman encoding. n Heapsort. n...

3. 3 Priority Queues in Action PQinit() for each v  V key(v)   PQinsert(v) key(s)  0 while (!PQisempty()) v = PQdelmin() for each w  Q s.t (v,w)  E if  (w) >  (v) + c(v,w) PQdecrease(w,  (v) + c(v,w)) Dijkstra's Shortest Path Algorithm

4. 4 Dijkstra/Prim 1 make-heap |V| insert |V| delete-min |E| decrease-key Priority Queues make-heap Operation insert find-min delete-min union decrease-key delete 1 Binary log N 1 N 1 Binomial log N 1 Fibonacci * 1 1 log N 1 1 1 Relaxed 1 1 log N 1 1 1 Linked List 1 N N 1 1 N is-empty11111 Heaps O(|E| + |V| log |V|)O(|E| log |V|)O(|V| 2 )

5. 5 Binary Heap: Definition Binary heap. n Almost complete binary tree. – filled on all levels, except last, where filled from left to right n Min-heap ordered. – every child greater than (or equal to) parent 06 14 78 18 81 7791 45 5347 64 84 99 83

6. 6 Binary Heap: Properties Properties. n Min element is in root. n Heap with N elements has height =  log 2 N . 06 14 78 18 81 7791 45 5347 64 84 99 83 N = 14 Height = 3

7. 7 Binary Heaps: Array Implementation Implementing binary heaps. n Use an array: no need for explicit parent or child pointers. – Parent(i) =  i/2  – Left(i) = 2i – Right(i) = 2i + 1 06 14 78 18 81 7791 45 5347 64 84 99 83 1 2 3 4 56 7 8910 11 12 13 14

8. 8 Binary Heap: Insertion Insert element x into heap. n Insert into next available slot. n Bubble up until it's heap ordered. – Peter principle: nodes rise to level of incompetence 06 14 78 18 81 7791 45 5347 64 84 99 83 42 next free slot

9. 9 Binary Heap: Insertion Insert element x into heap. n Insert into next available slot. n Bubble up until it's heap ordered. – Peter principle: nodes rise to level of incompetence 06 14 78 18 81 7791 45 5347 64 84 99 83 42 swap with parent

10. 10 Binary Heap: Insertion Insert element x into heap. n Insert into next available slot. n Bubble up until it's heap ordered. – Peter principle: nodes rise to level of incompetence 06 14 78 18 81 7791 45 4247 64 84 99 83 42 53 swap with parent

11. 11 Binary Heap: Insertion Insert element x into heap. n Insert into next available slot. n Bubble up until it's heap ordered. – Peter principle: nodes rise to level of incompetence n O(log N) operations. 06 14 78 18 81 7791 42 4547 64 84 99 83 53 stop: heap ordered

12. 12 Binary Heap: Decrease Key Decrease key of element x to k. n Bubble up until it's heap ordered. n O(log N) operations. 06 14 78 18 81 7791 42 4547 64 84 99 83 53

13. 13 Binary Heap: Delete Min Delete minimum element from heap. n Exchange root with rightmost leaf. n Bubble root down until it's heap ordered. – power struggle principle: better subordinate is promoted 06 14 78 18 81 7791 42 4547 64 84 99 83 53

14. 14 Binary Heap: Delete Min Delete minimum element from heap. n Exchange root with rightmost leaf. n Bubble root down until it's heap ordered. – power struggle principle: better subordinate is promoted 53 14 78 18 81 7791 42 4547 64 84 99 83 06

15. 15 Binary Heap: Delete Min Delete minimum element from heap. n Exchange root with rightmost leaf. n Bubble root down until it's heap ordered. – power struggle principle: better subordinate is promoted 53 14 78 18 81 7791 42 4547 64 84 99 83 exchange with left child

16. 16 Binary Heap: Delete Min Delete minimum element from heap. n Exchange root with rightmost leaf. n Bubble root down until it's heap ordered. – power struggle principle: better subordinate is promoted 14 53 78 18 81 7791 42 4547 64 84 99 83 exchange with right child

17. 17 Binary Heap: Delete Min Delete minimum element from heap. n Exchange root with rightmost leaf. n Bubble root down until it's heap ordered. – power struggle principle: better subordinate is promoted n O(log N) operations. 14 18 78 53 81 7791 42 4547 64 84 99 83 stop: heap ordered

18. 18 Binary Heap: Heapsort Heapsort. n Insert N items into binary heap. n Perform N delete-min operations. n O(N log N) sort. n No extra storage.

19. 19 H1H1 H2H2 14 78 18 81 7791 11 6253 64 84 99 41 Binary Heap: Union Union. n Combine two binary heaps H 1 and H 2 into a single heap. n No easy solution. –  (N) operations apparently required n Can support fast union with fancier heaps.

20. 20 Priority Queues make-heap Operation insert find-min delete-min union decrease-key delete 1 Binary log N 1 N 1 Binomial log N 1 Fibonacci * 1 1 log N 1 1 1 Relaxed 1 1 log N 1 1 1 Linked List 1 N N 1 1 N is-empty11111 Heaps

21. 21 Binomial Tree Binomial tree. n Recursive definition: B k-1 B0B0 BkBk B0B0 B1B1 B2B2 B3B3 B4B4

22. 22 Binomial Tree Useful properties of order k binomial tree B k. n Number of nodes = 2 k. n Height = k. n Degree of root = k. n Deleting root yields binomial trees B k-1, …, B 0. Proof. n By induction on k. B0B0 B1B1 B2B2 B3B3 B4B4 B1B1 BkBk B k+1 B2B2 B0B0

23. 23 Binomial Tree A property useful for naming the data structure. n B k has nodes at depth i. B4B4 depth 2 depth 3 depth 4 depth 0 depth 1

24. 24 Binomial Heap Binomial heap. Vuillemin, 1978. n Sequence of binomial trees that satisfy binomial heap property. – each tree is min-heap ordered – 0 or 1 binomial tree of order k B4B4 B0B0 B1B1 55 45 32 30 24 2322 50 483117 44 82910 6 37 318

25. 25 Binomial Heap: Implementation Implementation. n Represent trees using left-child, right sibling pointers. – three links per node (parent, left, right) n Roots of trees connected with singly linked list. – degrees of trees strictly decreasing from left to right 50 483117 44 10 6 37 318 29 6 37 318 48 3150 10 4417 heap 29 Leftist Power-of-2 HeapBinomial Heap Parent Left Right

26. 26 Binomial Heap: Properties Properties of N-node binomial heap. n Min key contained in root of B 0, B 1,..., B k. n Contains binomial tree B i iff b i = 1 where b n  b 2 b 1 b 0 is binary representation of N. n At most  log 2 N  + 1 binomial trees. n Height   log 2 N . B4B4 B0B0 B1B1 55 45 32 30 24 2322 50 483117 44 82910 6 37 318 N = 19 # trees = 3 height = 4 binary = 10011

27. 27 Binomial Heap: Union Create heap H that is union of heaps H' and H''. n "Mergeable heaps." n Easy if H' and H'' are each order k binomial trees. – connect roots of H' and H'' – choose smaller key to be root of H H'' 55 45 32 30 24 2322 50 483117 44 82910 6 H'

28. 28 Binomial Heap: Union 0011 1001+ 0111 11 1 1 0 1 19 + 7 = 26 55 45 32 30 24 2322 50 483117 44 82910 6 37 318 41 33 28 15 25 712 +

29. 29 Binomial Heap: Union 55 45 32 30 24 2322 50 483117 44 82910 6 37 318 41 33 28 15 25 712 +

30. 30 Binomial Heap: Union 55 45 32 30 24 2322 50 483117 44 82910 6 37 3 41 33 28 15 25 7 + 12 18 12

31. 31 55 45 32 30 24 2322 50 483117 44 82910 6 37 3 41 33 28 15 25 7 + 12 18 25 37 7 3 18 12 18 12

32. 32 55 45 32 30 24 2322 50 483117 44 82910 6 37 3 41 33 28 15 25 7 12 + 18 25 37 7 3 41 283325 37 157 3 18 12 18 12

33. 33 55 45 32 30 24 2322 50 483117 44 82910 6 37 3 41 33 28 15 25 7 + 18 12 41 283325 37 157 3 12 18 25 37 7 3 41 283325 37 157 3 18 12

34. 34 55 45 32 30 24 2322 50 483117 44 82910 6 37 3 41 33 28 15 25 7 + 18 12 41 283325 37 157 3 12 18 25 37 7 3 41 283325 37 157 3 55 45 32 30 24 2322 50 483117 44 82910 6 18 12

35. 35 Binomial Heap: Union Create heap H that is union of heaps H' and H''. n Analogous to binary addition. Running time. O(log N) n Proportional to number of trees in root lists  2(  log 2 N  + 1). 0011 1001+ 0111 11 1 1 0 1 19 + 7 = 26

36. 36 3 37 618 55 45 32 30 24 2322 50 483117 44 82910 H Binomial Heap: Delete Min Delete node with minimum key in binomial heap H. n Find root x with min key in root list of H, and delete n H'  broken binomial trees n H  Union(H', H) Running time. O(log N)

37. 37 Binomial Heap: Delete Min Delete node with minimum key in binomial heap H. n Find root x with min key in root list of H, and delete n H'  broken binomial trees n H  Union(H', H) Running time. O(log N) 55 45 32 30 24 2322 50 483117 37 618 44 82910 H H'

38. 38 3 37 618 55 x 32 30 24 2322 50 483117 44 82910 H Binomial Heap: Decrease Key Decrease key of node x in binomial heap H. n Suppose x is in binomial tree B k. n Bubble node x up the tree if x is too small. Running time. O(log N) n Proportional to depth of node x   log 2 N . depth = 3

39. 39 Binomial Heap: Delete Delete node x in binomial heap H. n Decrease key of x to - . n Delete min. Running time. O(log N)

40. 40 Binomial Heap: Insert Insert a new node x into binomial heap H. n H'  MakeHeap(x) n H  Union(H', H) Running time. O(log N) 3 37 618 55 45 32 30 24 2322 50 483117 44 82910 H x H'

41. 41 Binomial Heap: Sequence of Inserts Insert a new node x into binomial heap H. If N =.......0, then only 1 steps. If N =......01, then only 2 steps. If N =.....011, then only 3 steps. If N =....0111, then only 4 steps. Inserting 1 item can take  (log N) time. If N = 11...111, then log 2 N steps. But, inserting sequence of N items takes O(N) time! n (N/2)(1) + (N/4)(2) + (N/8)(3) +...  2N n Amortized analysis. n Basis for getting most operations down to constant time. 50 483117 44 2910 3 37 6 x

42. 42 Priority Queues make-heap Operation insert find-min delete-min union decrease-key delete 1 Binary log N 1 N 1 Binomial log N 1 Fibonacci * 1 1 log N 1 1 1 Relaxed 1 1 log N 1 1 1 Linked List 1 N N 1 1 N is-empty11111 Heaps just did this