1. Heaps and Priority Queues1 2 65 79

2. 2 What is a heap? A heap is a binary tree storing keys at its internal nodes and satisfying the following properties: Heap-Order: for every internal node v other than the root, key(v)  key(parent(v)) Complete Binary Tree: is a binary tree in which every level, except possibly the last, is completely filled, and all nodes are as far left as possible. 2 65 79 The last node of a heap is the rightmost internal node of depth h  1 last node

3. Heaps and Priority Queues3 Height of a Heap Theorem: A heap storing n keys has height O(log n) Proof: (we apply the complete binary tree property) Let h be the height of a heap storing n keys Since there are 2 i keys at depth i  0, …, h  2 and at least one key at depth h  1, we have n  1  2  4  …  2 h  2  1 Thus, n  2 h  1, i.e., h  log n  1 1 2 2h22h2 1 keys 0 1 h2h2 h1h1 depth

4. Heaps and Priority Queues4 We can use a heap to implement a priority queue We store an item (key, element) at each internal node We keep track of the position of the last node For simplicity, we show only the keys in the pictures (2, Sue) (6, Mark)(5, Pat) (9, Jeff)(7, Anna)

5. Heaps and Priority Queues5 Insertion into a Heap For any element in array position i, the left child is in position 2*i, the right child is in the cell after the left child ( 2*i+1). Thus, the parent is in i/2. Heap-order-property: key(v)  key(parent(v)): To keep the smallest element at root, for every node v, the key of the parent of v is smaller than or equal to the key of v. A CB EDF I H J G ABCDEFGHIJ 01234567891011

6. Heaps and Priority Queues6 Insertion into a Heap insertItem() function of the priority queue ADT corresponds to the insertion of a key k to the heap The insertion algorithm consists of three steps 1. Find the insertion node z (the new last node) 2. Store k at z and expand z into an internal node 3. Restore the heap- order property: key(v)  key(parent(v)) 2 65 79 insertion node 2 65 79 1 z z

7. Heaps and Priority Queues7 Upheap Algorithm After the insertion of a new key k, the heap-order property may be violated Algorithm upheap restores the heap-order property by swapping k along an upward path from the insertion node (percolate up) Upheap terminates when the key k reaches the root or a node whose parent has a key smaller than or equal to k Since a heap has height O(log n), upheap runs in O(log n) time 2 1 5 79 6 z 1 2 5 79 6 z

8. Heaps and Priority Queues8 Removal from a Heap removeMin() function of the priority queue ADT corresponds to the removal of the root key from the heap The removal algorithm consists of three steps 1. Replace the root key with the key of the last node w 2. Compress w and its children into a leaf 3. Restore the heap-order property (discussed next) 2 65 79 last node w 7 65 9 w

9. Heaps and Priority Queues9 Downheap (percolate down) After replacing the root key with the key k of the last node, the heap-order property may be violated Algorithm downheap restores the heap-order property by swapping key k along a downward path from the root Downheap terminates when key k reaches a leaf or a node whose children have keys greater than or equal to k Since a heap has height O(log n), downheap runs in O(log n) time 7 65 9 w 5 6 7 9 w

10. Heaps and Priority Queues10 Updating the Last Node The insertion node can be found by traversing a path of O(log n) nodes Go up until a left child or the root is reached If a left child is reached, go to the right child Go down left until a leaf is reached Similar algorithm for updating the last node after a removal

11. Heaps and Priority Queues11 5 12 3423753555335489. root removeMin()  Down Heap Bubbling

12. Heaps and Priority Queues12 3423753555335489. root

13. Heaps and Priority Queues13 12 3423753555335489. root

14. Heaps and Priority Queues14 89 3423753555335412. root

15. Heaps and Priority Queues15 23 3489753555335412. root

16. Heaps and Priority Queues16 23 3433753555895412. root

17. Heaps and Priority Queues17 23 3433753555895412 14. root

18. Heaps and Priority Queues18 23 3433143555895412 75. root

19. Heaps and Priority Queues19 14 3433233555895412 75. root

20. Heaps and Priority Queues20 Heap-Sort Consider a priority queue with n items implemented by means of a heap the space used is O(n) functions insertItem and removeMin take O(log n) time methods size, isEmpty, minKey, and minElement take time O(1) time Using a heap-based priority queue, we can sort a sequence of n elements in O(n log n) time The resulting algorithm is called heap-sort Heap-sort is much faster than quadratic sorting algorithms, such as insertion-sort and selection-sort

21. Heaps and Priority Queues21 Implementation using an array of pointers

22. Heaps and Priority Queues22 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 0

23. Heaps and Priority Queues23 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 1 5

24. Heaps and Priority Queues24 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 2 5 14

25. Heaps and Priority Queues25 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 3 5 1434

26. Heaps and Priority Queues26 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 4 5 1434 3

27. Heaps and Priority Queues27 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 4 5 1434 3 Parent of “3” node is determined by subtracting the index of that node by 1 and dividing by 2. In this case its (3-1)/2 = 1

28. Heaps and Priority Queues28 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 4 5 1434 3 Parent of “3” node is determined by subtracting the index of that node by 1 and dividing by 2. In this case its (3-1)/2 = 1 Compare parent with child.

29. Heaps and Priority Queues29 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 4 5 3 34 14 Parent of “3” node is determined by subtracting the index of that node by 1 and dividing by 2. In this case its (3-1)/2 = 1 Compare parent with child. Move the child up by one level.

30. Heaps and Priority Queues30 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 4 5 3 34 14 Parent of “3” node is determined by subtracting the index of that node by 1 and dividing by 2. In this case its (1-1)/2 = 0

31. Heaps and Priority Queues31 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 4 5 3 34 14 Parent of “3” node is determined by subtracting the index of that node by 1 and dividing by 2. In this case its (1-1)/2 = 0 Compare parent and child.

32. Heaps and Priority Queues32 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 4 3 5 34 14 Parent of “3” node is determined by subtracting the index of that node by 1 and dividing by 2. In this case its (1-1)/2 = 0 Compare parent and child. Move child up one level.

33. Heaps and Priority Queues33 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 4 3 5 34 14

34. Heaps and Priority Queues34 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 5 3 5 34 14 4

35. Heaps and Priority Queues35 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 5 3 5 34 14 4 Parent of “4” node is determined by subtracting the index of that node by 1 and dividing by 2. In this case its (4-1)/2 = 1

36. Heaps and Priority Queues36 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 5 3 5 34 14 4 Parent of “4” node is determined by subtracting the index of that node by 1 and dividing by 2. In this case its (4-1)/2 = 1 Compare parent and child.

37. Heaps and Priority Queues37 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 5 3 4 34 14 5 Parent of “4” node is determined by subtracting the index of that node by 1 and dividing by 2. In this case its (4-1)/2 = 1 Compare parent and child. Move child up one level.

38. Heaps and Priority Queues38 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 5 3 4 34 14 5 Parent of “4” node is determined by subtracting the index of that node by 1 and dividing by 2. In this case its (4-1)/2 = 1 Compare parent and child. Move child up one level.

39. Heaps and Priority Queues39 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 5 3 4 34 14 5 Parent of “4” node is determined by subtracting the index of that node by 1 and dividing by 2. In this case its (1-1)/2 = 0

40. Heaps and Priority Queues40 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 5 3 4 34 14 5 Parent of “4” node is determined by subtracting the index of that node by 1 and dividing by 2. In this case its (1-1)/2 = 0 Compare parent and child.

41. Heaps and Priority Queues41 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 5 3 4 34 14 5 Parent of “4” node is determined by subtracting the index of that node by 1 and dividing by 2. In this case its (1-1)/2 = 0 Compare parent and child. Leave child alone.

42. Heaps and Priority Queues42 Now remove the element with the smallest value

43. Heaps and Priority Queues43 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 5 3 4 34 14 5 Now to “pop” off the root node

44. Heaps and Priority Queues44 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 5 4 34 14 5 Now to “pop” off the root node. 3.object

45. Heaps and Priority Queues45 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 5 4 34 14 Now to “pop” off the root node. Move “tail” object to root. 3.object 5

46. Heaps and Priority Queues46 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 4 4 34 14 Now to “pop” off the root node. Move “tail” object to root. Change the “tail” index 3.object 5

47. Heaps and Priority Queues47 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 4 4 34 14 Now to “pop” off the root node. Move “tail” object to root. Change the “tail” index Compare root with two children 3.object 5

48. Heaps and Priority Queues48 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 4 4 34 14 Now to “pop” off the root node. Move “tail” object to root. Change the “tail” index Compare root with two children If greater, swap with smallest of the children 3.object 5

49. Heaps and Priority Queues49 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 4 5 34 14 Now to “pop” off the root node. Move “tail” object to root. Change the “tail” index Compare root with two children If greater, swap with smallest of the children 3.object 4

50. Heaps and Priority Queues50 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 4 5 34 14 Now to “pop” off the root node. Move “tail” object to root. Change the “tail” index Compare root with two children If greater, swap with smallest of the children Compare swapped object with children at indices 1*2+1=3 and 1*2+2=4 3.object 4

51. Heaps and Priority Queues51 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 4 5 34 14 Now to “pop” off the root node. Move “tail” object to root. Change the “tail” index Compare root with two children If greater, swap with smallest of the children Compare swapped object with children at indices 1*2+1=3 and 1*2+2=4 Node is smaller than both: stop 3.object 4

52. Heaps and Priority Queues52 0 1 2 3 4 5 6 7 8 9 10 11 12.root = 0.tail = 4 5 34 14 Now to “pop” off the root node. Move “tail” object to root. Change the “tail” index Compare root with two children If greater, swap with smallest of the children Compare swapped object with children at indices 1*2+1=3 and 1*2+2=4 Node is smaller than both: stop Return the object 4

53. Implementation in C - HeapStruct 53 #define MinPQSize (10) #define MinData (-32767) typedef int ElementType; struct HeapStruct { int Capacity; int Size; ElementType *Elements; }; typedef struct HeapStruct *PriorityQueue;

54. Implementation in C - Initialize 54 PriorityQueue Initialize( int MaxElements ) { PriorityQueue H; if( MaxElements < MinPQSize ) Error( "Priority queue size is too small" ); H = malloc( sizeof( struct HeapStruct ) ); if( H ==NULL ) FatalError( "Out of space!!!" ); /* Allocate the array plus one extra for sentinel */ H->Elements = malloc((MaxElements + 1)*sizeof(ElementType)); if( H->Elements == NULL ) FatalError( "Out of space!!!" ); H->Capacity = MaxElements; H->Size = 0; H->Elements[ 0 ] = MinData; return H; }

55. 55 int IsEmpty( PriorityQueue H ) { return H->Size == 0; } int IsFull( PriorityQueue H ) { return H->Size == H->Capacity; } Implementation in C – IsEmpty, IsFull

56. 56 Implementation in C – Insert /* H->Element[ 0 ] is a sentinel */ void Insert( ElementType X, PriorityQueue H ) { int i; if( IsFull( H ) ) { Error( "Priority queue is full" ); return; } for( i = ++H->Size; H->Elements[ i / 2 ] > X; i /= 2 ) H->Elements[ i ] = H->Elements[ i / 2 ]; H->Elements[ i ] = X; }

57. 57 Implementation in C – DeleteMin ElementType DeleteMin( PriorityQueue H ) { ElementType MinElement; if( IsEmpty( H ) ) { Error( "Priority queue is empty" ); return H->Elements[0]; } MinElement = H->Elements[ 1 ]; H->Elements[ 1 ] = H->Elements[ H->Size-- ]; percolateDown( H, 1 ); return MinElement; }

58. 58 Implementation in C – percolateDown void percolateDown( PriorityQueue H, int hole ) { int child; ElementType tmp = H->Elements[ hole ]; for( ; hole * 2 Size; hole = child ) { /* Find smaller child */ child = hole * 2; if (child != H->Size && H->Elements[child+1] Elements[child]) child++; /* Percolate one level */ if( tmp > H->Elements[child] ) H->Elements[ hole ] = H->Elements[ child ]; else break; } H->Elements[ hole ] = tmp; }

59. Building a Heap Sometimes it is required to construct it from an initial collection of items O(NlogN) in the worst case. But insertions take O(1) on the average. Hence the question: is it possible to do any better? 59

60. buildHeap Algorithm General Algorithm Place the N items into the tree in any order, maintaining the structure property. Call buildHeap 60 void buildHeap( PriorityQueue H, int N ) { int i; for( i = N / 2; i > 0; i-- ) percolateDown( H, i ); }

61. Heaps and Priority Queues61 Merging Two Heaps We are given two heaps and a key k We create a new heap with the root node storing k and with the two heaps as subtrees We perform downheap to restore the heap- order property 7 3 58 2 64 3 58 2 64 2 3 58 4 6 7

62. Heaps and Priority Queues62 We can construct a heap storing n given keys in using a bottom-up construction with log n phases In phase i, pairs of heaps with 2 i  1 keys are merged into heaps with 2 i  1  1 keys Bottom-up Heap Construction 2 i  1 2i112i11

63. Heaps and Priority Queues63 Example 1516 12476 2023 25 1516 5 124 11 76 27 2023

64. Heaps and Priority Queues64 Example (contd.) 25 1516 5 124 11 96 27 2023 15 25 16 4 12 5 6 9 11 23 20 27

65. Heaps and Priority Queues65 Example (contd.) 7 15 2516 4 125 8 6 911 23 2027 4 15 2516 5 12 7 6 8 911 23 2027

66. Heaps and Priority Queues66 Example (end) 4 15 2516 5 127 10 6 8 911 23 2027 5 15 2516 7 12 10 4 6 8 911 23 2027