1. 1 HEAPS & PRIORITY QUEUES Array and Tree implementations

2. 2 Priority queue A stack is first in, last out A queue is first in, first out A priority queue is least-first-out The “smallest” element is the first one removed

3. 3 The definition of “smallest” is up to the programmer (for example, you might define it by implementing Comparator or Comparable )

4. 4 Priority queue If there are several “smallest” elements, the implementer must decide which to remove first Remove any “smallest” element (don’t care which) Remove the first one added

5. 5 A priority queue is an: ADT where items are removed based on their priority (highest-to-lowest), regardless of the order they ARE inserted in the structure

6. 6 A priority queue ADT PriorityQueue(): Methods void add(Comparable o): void add(Comparable o): inserts o into the priority queue Comparable removeLeast(): Comparable removeLeast(): removes and returns the least element Comparable getLeast(): Comparable getLeast(): returns (but does not remove) the least element isEmpty(): boolean isEmpty(): returns true iff empty int size(): int size(): returns the number of elements void clear(): void clear(): discards all elements

7. 7 Evaluating implementations When we choose a data structure, usage patterns it is important to look at usage patterns If we load an array once and do thousands of searches on it, we want to make searching fast— so we would probably sort the array

8. 8 When we choose a data structure, it is important to look at usage patterns few searches If we load a huge array and expect to do only a few searches, we probably don’t want to spend time sorting the array

9. 9 Evaluating implementations uses of a queue (including a priority queue), For almost all uses of a queue (including a priority queue), remove everything that we add we eventually remove everything that we add

10. 10 TIMING Hence, analyze a priority queue, when we analyze a priority queue, neither “add” nor “remove” is more important— timing for we need to look at the timing for (“add + remove”)

11. 11 Array implementations A priority queue could be implemented as an unsorted array (with a count of elements) Adding an element would take O(1) time (why?) Removing an element would take O(n) time (why?) Hence, (adding and removing) an element takes O(n) time inefficient representation This is an inefficient representation – a BigO(N)

12. 12 Array implementations A priority queue could be implemented as a sorted array (again, with a counter of elements) Adding an element would take O(n) time (why?) Removing an element would take O(1) time (why?) So adding and removing) an element takes O(n) time Again, this is very inefficient

13. 13 Linked list implementations A priority queue could be implemented as an – unsorted linked list O(1) Adding an element would take O(1) time (why?) Removing an element would take (N) time (why?) Again, inefficient!

14. 14 SORTED LINKED LIST A priority queue could be implemented as a sorted linked list Adding an element Adding an element would take O(n) time (why?) Removing an element Removing an element would take O(1) time (why?) inefficient algorithm Again, an inefficient algorithm

15. 15 Binary tree implementations A priority queue could be represented as a – balanced binary search tree Insertion and removal could destroy the balance We would need an algorithm to rebalance the binary tree Good rebalancing algorithms O(log n) time complicated Good rebalancing algorithms require O(log n) time, but are complicated

16. 16 Heap Implementaton The concepts of heaps is used to refer to memory in a computer, E.g. the part of the memory that provides dynamic memory allocation ( get it as you need it) “Heap” is also used as a name for a special kind of binary tree

17. 17 Heap is a complete binary tree A heap is a complete binary tree with values stored in its nodes such that no child has a value bigger than the value of its parent – (though it can be equal)

18. 18 Therefore we would like to find a better algorithm. We want to: Insert and Remove with a Big O of one

19. 19 HEAPS & Priority Queues A binary tree represented as a heap facilitates the operation to find the maximum element: “it is always in the root of the tree “ Because finding the node with the maximum value in a heap is a BigO(I) A HEAP is the most efficient structure for implementing priority queues

20. 20 HeapsHeaps A heap is a certain kind of complete binary tree. When a complete binary tree is built, its first node must be the root. When a complete binary tree is built, its first node must be the root. Root

21. 21 HeapsHeaps Complete binary tree. Right child The Second and third Nodes are always the left and right child of the root. The Second and third Nodes are always the left and right child of the root. Left child

22. 22 HeapsHeaps Complete binary tree. The next nodes always fill the next left-to-right. level from left-to-right. The next nodes always fill the next left-to-right. level from left-to-right.

23. 23 HeapsHeaps Complete binary tree.

24. 24 HeapsHeaps A heap is a certain kind of complete binary tree. Each node in a heap contains a key that can be compared to other nodes' keys. Each node in a heap contains a key that can be compared to other nodes' keys. 19 4222127 23 45 35

25. 25 To implement a heap as a priority queue A node has the heap property if it is Equal to or greater than as its children OR ( since smaller numbers represent higher priorities) if it is smaller than or equal to its children ( since smaller numbers represent higher priorities) 12 83 Heap: Yellow node has the heap property 3 812 Priority queue: Yellow node has the heap property - less than its children

26. 26 Heaps A heap is a certain kind of complete binary tree. This example is a max heap. The "heap property" requires that each node's key is >= to the keys of its children The "heap property" requires that each node's key is >= to the keys of its children 19 4222127 23 45 35

27. 27 Priority Queues A heap can easily implement a priority queue remove the element of highest priority? but what happens with we remove the element of highest priority? How do we re-arrange the tree?

28. 28 Removing from a heap There are two things we need to consider when rearranging : We want to end up with a heap which means that The tree has to be complete – ALL HEAPS ARE COMPLETE TREES!

29. 29 To remove an element at top of tree : Remove the element at location 0 to location 0, decrement Move the element in the lastIndex to location 0, decrement lastIndex (lastIndex is at the end of the heap) Reheap the new root node Reheap the new root node (the one now at location 0 ) down-heap bubbling or percolating down This is called down-heap bubbling or percolating down Down-heap bubbling Down-heap bubbling requires O(log n) time to remove an element

30. 30 Removing the Top of a Heap (max heap) ¶Move the last node onto the root. 19 4222135 23 45 42 27

31. 31 Removing the Top of a Heap (max heap) ¶Move the value in the last node into the root. 19 4222135 23 27 42

32. 32 Removing the Top of a Heap ¶Move the last node onto the root. ·Push the out-of-place node downward, ·swapping with its larger child ·until the new node reaches an acceptable location. 19 4222135 23 27 42

33. 33 Removing the Top of a Heap ¶Move the last node onto the root. ·Push the out-of-place node downward, acceptable location. ·Push the out-of-place node downward, swapping with its larger child until the new node reaches an acceptable location. 19 4222135 23 42 27

34. 34 Removing the Top of a Heap ¶Move the last node onto the root. ·Push the out-of-place node downward, swapping with its larger child until the new node reaches an acceptable location. ·Note that we discard half of the tree with each move downward 19 4222127 23 42 35

35. reheapification downward. We stop when: keys <= to the out-of- place node, or The children all have keys <= to the out-of- place node, or The node reaches the leaf. The node reaches the leaf. The process of pushing the node downward is called: reheapification downward. 19 4222127 23 42 35

36. 36 Priority Queues Another Example : How to delete a node from a Priority Queue Since it is queue - first in first out, we remove the first one The Root

37. 37 An Example of a MAX Heap The root has largest value 23 1612 79135 2 1 3

38. 38 Deleting a node from the heap a heap priority queue We use a heap to implement a priority queue next" element ( the root the "next" element ( the root ) in the queue was removed (pop operation). We end up with a root with no value 1612 79135 2 1 3 Root has no value

39. 39 Deleting a node from the heap 1612 79135 2 1 3 The obvious question is: which node can we use to replace this one? The obvious question is: which node can we use to replace this one?

40. 40 Deleting a node from the heap 1612 79135 2 1 3 If we want this tree to stay complete the rightmost element in the bottommost level is the obvious choice If we want this tree to stay complete the rightmost element in the bottommost level is the obvious choice

41. 41 Deleting a node from the heap 2 1612 79135 1 3 We now have a problem: We have a complete tree but this is not a heap! WHY??? This can be solved by applying systematic swap of nodes. We now have a problem: We have a complete tree but this is not a heap! WHY??? This can be solved by applying systematic swap of nodes.

42. 42 Deleting a node from the heap 2 1612 79135 1 3 We systematically swap the root with the larger of the children nodes until no more swaps can be done We systematically swap the root with the larger of the children nodes until no more swaps can be done

43. 43 Deleting a node from the heap 16 212 79135 1 3

44. 44 Deleting a node from the heap 16 1312 7925 1 3

45. 45 Deleting a node from the heap 16 1312 7925 1 3 The tree has restored its heap property! The tree has restored its heap property!

46. 46 Insert operation Another example - the heap below Recall that while delete causes pairwise swaps from the root to the bottom, insert causes pairwise swaps from the bottom to the top. insert an element with value 20 Suppose that we want to insert an element with value 20 22 17 12 113135 1 6 4

47. 47 insert (20) 22 1712 113135 1 6 4 20

48. 48 insert (20) 22 1712 1120135 1 6 4 3

49. 49 insert (20) 22 17 20 11 12 135 1 6 4 3

50. 50 insert (20) 22 1720 1112135 1 6 4 3

51. 51 Implementing a Heap We will store the data from the nodes in a partially-filled array. An array of data 2127 23 42 35

52. 52 Implementing a Heap Data from the root goes in the first location of the array. An array of data 2127 23 42 35 42

53. 53 Implementing a Heap Data from the next row goes in the next two array locations. An array of data 2127 23 42 35 423523

54. 54 Implementing a Heap Data from the next row goes in the next two array locations. An array of data 2127 23 42 35 423523 2721

55. 55 Implementing a Heap Data from the next row goes in the next two array locations. An array of data 2127 23 42 35 423523 2721

56. 56 Points about the Implementation The links between the tree's nodes are not actually stored as pointers, or in any other way. manipulate the data. The only way we "know" that "the array is a tree" is from the way we manipulate the data. An array of data 2127 23 42 35 423523 2721

57. 57 Array representation of a heap Left child of node i is 2*i + 1, right child is 2*i + 2 lastIndex Unless the value is larger than lastIndex -therefore no such child Parent of node i is (i – 1)/2 unless i == 0 12 1418 6 8 3 312618148 0 1 2 3 4 5 6 7 8 9 10 11 12 lastIndex = location 5

58. 58 algorithm for the array implementation new value where lastIndex now references -- Increase lastIndex and put the new value where lastIndex now references -- that is add to the end of the array Reheap Reheap the newly added node up-heap bubbling or percolating This is called up-heap bubbling or percolating up Up-heap bubbling requires O(log n) time Adding to heap - - Implementation

59. 59 Points about the Implementation If you know the index of a node, then it is easy to figure out the indexes of that node's parent and children. [0] [0] [1] [2] [3] [4] 2127 23 42 35 423523 2721

60. 60 Points about the Implementation If we add 45 to the tree and to the array, 45’s parent is at what index? i = ? (i-1)/2 = 2 [0] [0] [1] [2] [3] [4] [5] 2127 23 42 35 423523 2721 45

61. 61 Points about the Implementation If we add 45 to the tree and the array, 45’s parent is at what index? (i-1)/2 which is 2. We do a comparison of 45 with its parent and swap if it is greater than the parent - it is greater than it [0] [0] [1] [2] [3] [4] [5] 2127 23 42 35 423523 2721 45

62. 62 Points about the Implementation If we add 45 to the tree and to the tree and the array, 45’s parent is at what index? (i-1)/2 which is 2. We do a comparison and swap. It is still out of place. [0] [0] [1] [2] [3] [4] [5] 2127 45 42 35 423545 2721 23

63. 63 Points about the Implementation 45 is still out of place, so we do another comparison and swap with 45’s parent which is 42. So what is the terminating condition)(s)? [0] [0] [1] [2] [3] [4] [5] 2127 42 45 35 453542 2721 23

64. 64 A heap is a complete binary tree, where the entry at each node is greater than or equal to the entries in its children. OR the entry at each node is less than or equal to the entries in its children. Summary

65. 65 Summary Adding/Deleting To add an entry to a heap, place the new entry at the next available spot, and perform a reheapification upward. To remove the biggest entry, move the value in the last node onto the root, and perform a reheapification downward.

66. 66 Reheapifying a Tree Fine, we know how to delete one element and restore the heap, but there are several other operations involving heaps

67. 67 If we find a systematic way of heapifying a tree, we could perform any operation because we can execute the procedure heapify

68. 68 Reheapifying a Tree heapify a tree using a quite simple idea. we can heapify a tree using a quite simple idea. Given a binary tree T (not necessarily a heap) with root R and left subtree T left and T right, if these two subtrees are heaps, if these two subtrees are heaps, we can make T a heap using the process described in the previous slide (of applying systematic swaps) We start the process of heapifying a tree from the lower levels

69. 69 Algorithm for Reheapifying Tree The algorithm aims at heapifying a tree by always comparing three nodes at a time. To start from the lower levels we choose to look at the internal nodes in reverse level order reverse level order External nodes have no children so they are (individually) heaps

70. 70 HEAPIFY void heapify (Heap h) { NodeType n; NodeType n; for (n = the internal nodes of h in reverse level- order) { for (n = the internal nodes of h in reverse level- order) { Reheapify the heap h starting at node n Reheapify the heap h starting at node n }} void heapify (Heap h) { NodeType n; NodeType n; for (n = the internal nodes of h in reverse level- order) { for (n = the internal nodes of h in reverse level- order) { Reheapify the heap h starting at node n Reheapify the heap h starting at node n }}

71. 71 6 174 121135 3 11 22 Let change this tree into a heap - A MAX HEAP

72. 72 An example 6 174 121135 3 11 22 Internal Nodes in Reverse Level-Order are: 1, 12, 17, 4, 6, start with the leftmost leaf : compare 3 To 1

73. 73 6 174 123135 1 11 22 3 and 1 have swapped 3 and 1 have swapped Internal Nodes in Reverse Level-Order: 3, 12, 17, 4, 6

74. 74 An example 6 174 12 3135 1 11 22 Compare 12 with its children and swap 12 and 22 : Compare 12 with its children and swap 12 and 22 : Internal Nodes in Reverse Level-Order: 3, 22, 17, 4, 6

75. 75 An example 6 174 223135 1 11 12 Compare 17 with its children and no changes necessary: 3, 22, 17, 4, 6 --- NEXT COMPARE 4 AND 22

76. 76 6 1722 43135 1 11 12 Compare 22 with 4 and swap : 4 IS NOW OUT OF PLACE

77. 77 An example 6 1722 123135 1 11 4 Compare 12 and 4 and swap, NEXT compare 22 to 6 : 3, 12, 17, 22, 6

78. 78 22 176 123135 1 11 4 Heapify 6 downwards until it finds its proper location : 3, 12, 17, 6

79. 79 22 1712 63135 1 11 4 Reheap down 6 : Compare 6 and 11

80. 80 An example 22 1712 113135 1 6 4 6 is in place and the tree is reheaped:

81. 81 An example 22 1712 113135 1 6 4

82. 82heapsort Heapsort is another algorithm for sorting lists The idea behind heap sort is that to sort a list we can insert the elements of a list into a heap using heapify Then remove all the elements using remove.

83. 83 The idea behind heap sort is that to sort a list we can insert the elements of a list into a heap using heapify Then remove all the elements using remove. largest element is removed first, It is guaranteed that the largest element is removed first, So by systematically removing the elements we get the list ordered ( in reverse order)