1. ADSA: Heaps/11 1 241-423 Advanced Data Structures and Algorithms Objectives – –implement heaps (a kind of array-based complete binary tree), heap sort, priority queues Semester 2, 2012-2013 11. Heaps

2. ADSA: Heaps/11 2 Contents 1. Tree Terminology 2. Binary Trees 3. The Comparator Interface 4. Generalized Array Sorting 5. Heaps 6. Sorting with a Max Heap 7. API for the Heaps Class 8. Priority Queue Collection

3. ADSA: Heaps/11 3 1. Tree Terminology continued

4. ADSA: Heaps/11 4 A path between a parent node X 0 and a subtree node N is a sequence of nodes P = X 0, X 1,..., (X k is N) – –k is the length of the path – –each node X i is the parent of X i+1 for 0  i  k-1 continued

5. ADSA: Heaps/11 5 The level of a node is the length of the path from root to that node. The height of a tree is the maximum level in the tree. continued

6. ADSA: Heaps/11 6

7. 7 2. Binary Trees In a binary tree, each node has at most two children. continued

8. ADSA: Heaps/11 8 Each node of a binary tree defines a left and a right subtree. Each subtree is a tree. Each node of a binary tree defines a left and a right subtree. Each subtree is a tree. Left child of T Right child of T

9. ADSA: Heaps/11 9 Height of a Binary Tree Node o o Node N starts a subtree T N, with T L and T R the left and right subtrees. Then: -1if T N is empty 1+max( height(T L ), height(T R ))if T N not empty height(N) = height(T N ) = { continued

10. ADSA: Heaps/11 10 degenerate binary tree (a list)

11. ADSA: Heaps/11 11 A Complete Binary Tree A complete binary tree of height h has all its nodes filled through level h-1, and the nodes at depth h run left to right with no gaps. continued

12. ADSA: Heaps/11 12

13. ADSA: Heaps/11 13 An array arr[] can be viewed as a complete binary tree if: – –the root is stored in arr[0] – –the level 1 children are in arr[1], arr[2] – –the level 2 children are in arr[3], arr[4], arr[5], arr[6] – –etc. An Array-based Complete Binary Tree continued

14. ADSA: Heaps/11 14 Integer[] arr = {5, 1, 3, 9, 6, 2, 4, 7, 0, 8}; continued

15. ADSA: Heaps/11 15 o o For arr[i] in an n-element array ‑ based complete binary tree: Left child of arr[i] isarr[2*i + 1]; or undefined if (2*i + 1)  n Right child of arr[i] isarr[2*i + 2]; or undefined if (2*i + 2)  n Parent of arr[i] isarr[(i-1)/2]; or undefined if i = 0

16. ADSA: Heaps/11 16 3. The Comparator Interface continued

17. ADSA: Heaps/11 17 3.1. Comparator vs. Comparable Comparator.compare(T x, T y) – –comparison is done between two objects, x and y – –the method is implemented outside the T class Comparable.compareTo(T y) – –comparison is done between 'this' object and object y – –the method is implemented by the T class

18. ADSA: Heaps/11 18 3.2. Benefits of Comparator v v The comparison code is in a separate place from the other code for a class. v v 1. This means that the same objects can be compared in different ways by defining different Comparators – –see the two comparators for circles

19. ADSA: Heaps/11 19 2. A Comparator can be passed to a sort method as an argument to control the ordering of objects. this means that the same sort method can sort in different ways depending on its Comparator argument. e.g. sorting an array into ascending or descending order e.g. sorting data into max heaps or min heaps

20. ADSA: Heaps/11 20 3.3. Comparing Circles public class Circle { private double xCenter, yCenter; private double radius; public Circle(double x, double y, double r) { xCenter = x; yCenter = y; radius = r; } public double getX() { return xCenter; } :

21. ADSA: Heaps/11 21 public double getY() { return yCenter; } public double getRadius() { return radius; } public double getArea() { return Math.PI * radius * radius; } : // more methods, but no comparison function } // end of Circle class

22. ADSA: Heaps/11 22 Comparing Circle Radii public class RadiiComp implements Comparator { public int compare(Circle c1, Circle c2) { double radC1 = c1.getRadius(); double radC2 = c2.getRadius(); // returns < 0 if radC1 < radC2, // 0 if radC1 == radC2, // > 0 if radC1 > radC2 return (int)(radC1 – radC2); } // end of compare() // equals() is inherited from Object superclass } // end of RadiiComp class

23. ADSA: Heaps/11 23 Comparing Circle Positions public class PosComp implements Comparator { public int compare(Circle c1, Circle c2) { double c1Dist = (c1.getX() * c1.getX()) + (c1.getY() * c1.getY()); double c2Dist = (c2.getX() * c2.getX()) + (c2.getY() * c2.getY()); // returns < 0 if c1Dist < c2Dist, // 0 if c1Dist == c2Dist, // > 0 if c1Dist > c2Dist return (int)(c1Dist - c2Dist); } // end of compare() // equals() is inherited from Object superclass } // end of PosComp class

24. ADSA: Heaps/11 24 Comparing Circles in Two Ways Circle c1 = new Circle(0, 0, 5); Circle c2 = new Circle(3, 2, 7); RadiiComp rComp = new RadiiComp(); if (rComp.compare(c1, c2) < 0) System.out.println("Circle 1 is smaller than circle 2"); PosComp pComp = new PosComp(); if (pComp.compare(c1, c2) < 0) System.out.println("Circle 1 is nearer to origin than circle 2"); Circle 1 is smaller than circle 2 Circle 1 is nearer to origin than circle 2

25. ADSA: Heaps/11 25 3.4. Comparators for Sorting The Less and Greater Comparator classes make sorting and searching functions more flexible – –see selectionSort() in the next few slides Less and Greater will also be used to create min and max heaps.

26. ADSA: Heaps/11 26 The Less Comparator import java.util.Comparator; // the < Comparator public class Less implements Comparator { public int compare(T x, T y) { return ((Comparable )x).compareTo(y); } uses T's compareTo() to compare x and y x Less y == x.compareTo y == x < y

27. ADSA: Heaps/11 27 Using Less Comparator lessInt = new Less (); Integer a = 3, b = 5; if (lessInt.compare(a, b) < 0) System.out.println(a + " < " + b); 3 < 5

28. ADSA: Heaps/11 28 The Greater Comparator // the > Comparator public class Greater implements Comparator { public int compare(T x, T y) { return - ((Comparable )x).compareTo(y); } uses T's compareTo() to compare x and y x Greater y == -(x.compareTo y) == -(x < y) == x > y

29. ADSA: Heaps/11 29 Using Greater Comparator greaterInt = new Greater (); Integer a = 9, b = 5; if (greaterInt.compare(a, b) < 0) System.out.println(a + " > " + b); 9 > 5

30. ADSA: Heaps/11 30 4. Generalized Array Sorting A single selectionSort() method can sort array in different ways by being passed different Comparator arguments. The Less or Greater Comparators let the method sort an array of objects into ascending or descending order.

31. ADSA: Heaps/11 31 Selection Sort with Comparator // new version adds a Comparator parameter public static void selectionSort(T[] arr, Comparator comp) { int smallIndex; int n = arr.length; for (int pass = 0; pass < n-1; pass++) { // scan the sublist starting at index smallIndex = pass; : Compare with selectionSort() in Part 2 which uses Comparable.

32. ADSA: Heaps/11 32 // j traverses the sublist arr[pass+1] to arr[n-1] for (int j = pass+1; j < n; j++) // if smaller element found, assign posn to smallIndex if (comp.compare(arr[j], arr[smallIndex]) < 0) smallIndex = j; // swap smallest element into arr[pass] T temp = arr[pass]; arr[pass] = arr[smallIndex]; arr[smallIndex] = temp; } } // end of selectionSort()

33. ADSA: Heaps/11 33 Example String[] arr = {"red", "green", "blue", "yellow", "teal", "orange"}; Less lessComp = new Less (); Greater gtComp = new Greater (); Arrays.selectionSort(arr, lessComp); System.out.println("Less sort: " + Arrays.toString(arr)); Arrays.selectionSort(arr, gtComp); System.out.println("Greater sort: " + Arrays.toString(arr)); Less sort: [blue, green, orange, red, teal, yellow] Greater sort: [yellow, teal, red, orange, green, blue] // ascending // descending

34. ADSA: Heaps/11 34 5. Heaps A maximum heap is an array ‑ based complete binary tree in which the value of a parent is ≥ the value of both its children. continued 0 555052251011 5 2022 12345678 lvl 0 1 23 there’s no ordering within a level (between siblings)

35. ADSA: Heaps/11 35 0 40153010 123 lvl 0 1 2

36. ADSA: Heaps/11 36 A minimum heap uses the relation ≤. continued 0 51050112052 55 2522 12345678 lvl 0 1 23

37. ADSA: Heaps/11 37 0 10153040 123 lvl 0 1 2

38. ADSA: Heaps/11 38 v v A max heap will be ordered using the Greater comparator. – –I’ll focus on max heaps in these notes v v A min heap will be ordered using the Less comparator.

39. ADSA: Heaps/11 39 Heap Uses Heapsort – –one of the best sorting methods in-place; no quadratic worst-case Selection algorithms – –finding the min, max, median, k-th element in sublinear time Graph algorithms – –Prim's minimal spanning tree; – –Dijkstra's shortest path

40. ADSA: Heaps/11 40 Max Heap Operations o o Inserting an element: pushHeap() o o Deleting an element: popHeap() o o most of the work is done by calling adjustHeap() o o Array --> heap conversion: makeHeap() o o most of the work is done by calling adjustHeap() o o Heap sorting: heapSort() o o utilizes makeHeap() then popHeap()

41. ADSA: Heaps/11 41 5.1. Inserting into a Max Heap Assume that the array is a maximum heap. – –a new item will enter the array at index last with the heap expanding by one element continued pushHeap()

42. ADSA: Heaps/11 42 Insert an item by moving the nodes on the path of parents down one level until the item is correctly placed as a parent in the heap. path of parents insert 50 continued

43. ADSA: Heaps/11 43 continued

44. ADSA: Heaps/11 44 At each step, compare item with parent – –if item is larger, move parent down one level arr[currPos] = parent; currPos = the parent index; Stop when parent is larger than item – –assign item to the currPos position arr[currPos] = item;

45. ADSA: Heaps/11 45 pushHeap() public static void pushHeap(T[] arr, int last, T item, Comparator comp) { // insert item into the heap arr[] // assume that arr[0] to arr[last-1] // are in heap order // currPos is the index that moves up the // path of parents int currPos = last; int parentPos = (currPos-1)/2; // see slide 16 : Depending on the comparator, pushHeap() can work with max or min heaps.

46. ADSA: Heaps/11 46 // move up parents path to the root while (currPos != 0) { // compare target and parent value if (comp.compare(item,arr[parentPos]) < 0) { arr[currPos] = arr[parentPos]; // move data from parent --> current currPos = parentPos; parentPos = (currPos-1)/2; // get next parent } else // heap condition is ok break; } arr[currPos] = item; // put item in right location } // end of pushHeap()

47. ADSA: Heaps/11 47 5.2. Deleting from a Heap Deletion is normally restricted to the root only – –remove the maximum element (in a max heap) To erase the root of an n ‑ element heap: – –exchange the root with the last element (the one at index n ‑ 1); delete the moved root – –filter (sift) the new root down to its correct position in the heap popHeap() call adjustHeap()

48. ADSA: Heaps/11 48 Deletion Example Delete 63 – –exchange with 18; remove 63 – –filter down 18 to correct position continued for a Max Heap

49. ADSA: Heaps/11 49 continued (63) removed

50. ADSA: Heaps/11 50 Move 18 down: – –smaller than 30 and 40; swap with 40 18 38 continued

51. ADSA: Heaps/11 51 Move 18 down: – –smaller than 38; swap with 38 18 38 18 Stop since 18 is now a leaf node.

52. ADSA: Heaps/11 52 Filter (Sift) Down a Max Heap Move root value down the tree: – –compare value with its two children – –if value < a child then heap order is wrong – –select largest child and swap with value – –repeat algorithm but with new child – –continue until value ≥ both children or at a leaf adjustHeap()

53. ADSA: Heaps/11 53 // filter a value down the heap public static void adjustHeap(T[] arr, int first, int last, Comparator comp) { int currentPos = first; // start at first T value = arr[first]; // filter value down the heap // compute the left child index int childPos = 2*currentPos + 1; // see slide 16 : adjustHeap() Depending on the comparator, adjustHeap() can work with max or min heaps.

54. ADSA: Heaps/11 54 // scan down path of children while (childPos < last) { // compare the two children; select bigger if ((childPos+1 < last) && comp.compare(arr[childPos+1], arr[childPos]) < 0) childPos = childPos + 1; // since cp+1 < cp // compare selected child with value if (comp.compare(arr[childPos],value) < 0) { // swap child and value arr[currentPos] = arr[childPos]; arr[childPos] = value; : index of right child is childPos+1

55. ADSA: Heaps/11 55 // update indices to continue the scan currentPos = childPos; childPos = 2*currentPos + 1; // left child index } else // value in right position break; } } // end of adjustHeap()

56. ADSA: Heaps/11 56 Deletion method: popHeap() Exchange the root with the last value in the heap (arr[last-1]) Call adjustHeap() to filter value down heap – –heap now has index range [0, last-1) Return the original root value see slide 47 for more details

57. ADSA: Heaps/11 57 // delete the maximum (or minimum) element in the heap // and return its value public static T popHeap(T[] arr, int last, Comparator comp) { // value that is removed from the heap T temp = arr[0]; // exchange last element in heap with the root value arr[0] = arr[last-1]; arr[last-1] = temp; // filter the value down over the range (0, last-1) adjustHeap(arr, 0, last-1, comp); return temp; }

58. ADSA: Heaps/11 58 5.3. Using Heaps import ds.util.Heaps; import ds.util.Greater; import ds.util.Less; public class UsingHeaps { public static void main(String[] args) { // integer array, and arrA and arrB heaps Integer[] intArr = {15, 29, 52, 17, 21, 39, 8}; Integer[] heapArrA = new Integer[intArr.length], Integer[] heapArrB = new Integer[intArr.length]; // comparators for maximum and minimum heaps Greater gtComp = new Greater (); Less lessComp = new Less (); :

59. ADSA: Heaps/11 59 // load intArr into heapArrA to form a maximum heap // and into heapArrB to form a minimum heap for (i = 0; i < intArr.length; i++) { Heaps.pushHeap(heapArrA, i, intArr[i], grComp); //max Heaps.pushHeap(heapArrB, i, intArr[i], lessComp);//min } // print heapArrA System.out.println("Display maximum heap:"); System.out.println( Heaps.displayHeap(heapArrA, heapArrA.length, 2)); // draw heapArrB Heaps.drawHeap(heapArrB, heapArrB.length, 2); :

60. ADSA: Heaps/11 60 // pop minimum value Integer minObj = Heaps.popHeap(heapArrB, heapArrB.length, less); System.out.println("\nMinimum value is " + minObj); // draw heapArrB before and after popHeap() // the index range is 0 to heapArrB.length-1 Heaps.drawHeap(heapArrB, heapArrB.length-1, 2); } // end of main() } // end of UsingHeaps class

61. ADSA: Heaps/11 61 remove 8 minimum heap, heapArrB max heap, heapArrA Execution continued displayHeap() drawHeap()

62. ADSA: Heaps/11 62 minimum heap, heapArrB

63. ADSA: Heaps/11 63 5.4. Complexity of Heap Operations A heap stores elements in an array-based complete tree. pushHeap() reorders elements in the tree by moving up the path of parents. adjustHeap() reorders elements in the tree by moving down the path of the children. – –their cost depends on path length continued

64. ADSA: Heaps/11 64 Assuming the heap has n elements, the maximum length for a path between a leaf node and the root is log 2 n since the tree is balanced The runtime efficiency of the algorithms is O(log 2 n)

65. ADSA: Heaps/11 65 5.5. Heapifying O(n)! Transforming an array into a heap is called "heapifying the array". Turn an n ‑ element array into a heap by filtering down each parent in the tree – –begin with the last parent at index (n-2)/2 – –end with the root node at index 0 continued makeHeap()

66. ADSA: Heaps/11 66 Integer[] arr = {9, 12, 17, 30, 50, 20, 60, 65, 4, 19}; The grey nodes are the parents. Adjust in order: 50, 30, 17, 12, 9 continued Max Heap Creation

67. ADSA: Heaps/11 67 continued

68. ADSA: Heaps/11 68 continued

69. ADSA: Heaps/11 69

70. ADSA: Heaps/11 70 // arrange array elements into a heap public static void makeHeap(T[] arr, Comparator comp) { int lastPos = arr.length; // heap size int heapPos = (lastPos - 2)/2; // see slide 16 // the index of the last parent // filter down every parent in order // from last parent up to root while (heapPos >= 0) { adjustHeap(arr, heapPos, lastPos, comp); heapPos--; } } // end of makeHeap() Depending on the comparator, makeHeap() can create max or min heaps.

71. ADSA: Heaps/11 71 6. Sorting with a Max Heap If the array is a maximum heap, it has an efficient sorting algorithm: – –For each iteration i, the largest element is arr[0]. – –Exchange arr[0] with arr[i] and then reorder the array so that elements in the index range [0, i) are a heap. – –This is done by popHeap(), which is O(log 2 n) heapSort()

72. ADSA: Heaps/11 72 Max Heap Sort continued

73. ADSA: Heaps/11 73 the max heap sort is into ascending order

74. ADSA: Heaps/11 74 Heapsort Heap sort is a modified version of selection sort for an array that is a heap. – –for each i = n, n-1,..., 2, call popHeap() which pops arr[0] from the heap and assign it at index i-1. A maximum heap is sorted into ascending order A minimum heap is sorted into descending order.

75. ADSA: Heaps/11 75 public static void heapSort(T[] arr, Comparator comp) { Heaps.makeHeap(arr, comp); // "heapify" arr[] int n = arr.length; // iteration through arr[n-1]... arr[1] for (int i = n; i > 1; i--) { // popHeap() moves largest (smallest) to arr[n-1] Heaps.popHeap(arr, i, comp); } } // end of heapSort() Depending on the comparator, heapSort() can work with max or min heaps.

76. ADSA: Heaps/11 76 Example Integer[] arr = {7,1,9,0,8,2,4,3,6,5}; // make a max heap Arrays.heapSort(arr, new Greater () ); System.out.println("Sort ascending: " + Arrays.toString(arr)); // make a min heap Arrays.heapSort(arr, new Less () ); System.out.println("Sort decending: " + Arrays.toString(arr)); Sort ascending: [0,1,2,3,4,5,6,7,8,9] Sort descending: [9,8,7,6,5,4,3,2,1,0] // max heap // min heap

77. ADSA: Heaps/11 77 The worst case running time of makeHeap() is closer to O(n), not O(n log 2 n). During the second phase of the heap sort, popHeap() executes n-1 times. Each operation has efficiency O(log 2 n). The worst-case complexity of the heap sort is O(n) + O(n log 2 n) = O(n log 2 n).

78. ADSA: Heaps/11 78 7. API for the Heaps Class continued

79. ADSA: Heaps/11 79

80. ADSA: Heaps/11 80 8. Priority Queue Collection o o In a priority queue, all the elements have priority values. o o A deletion always removes the element with the highest priority.

81. ADSA: Heaps/11 81 o o Two types of priority queues: – –maximum priority queue remove the largest value first what I’ll be using – –minimum priority queue remove the smallest value first

82. ADSA: Heaps/11 82 PQueue Interface o o The generic PQueue resembles a queue with the same method names. continued

83. ADSA: Heaps/11 83

84. ADSA: Heaps/11 84 HeapPQueue Class Example // create a max priority queue of Strings HeapPQueue pq = new HeapPQueue (); pq.push("green"); pq.push("red"); pq.push("blue"); // output the size, and element with highest priority System.out.println(pq.size() + ", " + pq.peek()); // use pop() to empty the pqueue and list elements // in decreasing priority order while ( !pq.isEmpty() ) System.out.print( pq.pop() + " "); 3, red red green blue The max priority for Strings is z --> a order

85. ADSA: Heaps/11 85 Implementing PQueue We can use a heap (the HeapPQueue class) to implement the PQueue interface. The user can specify either a Greater (max heap) or Less (min heap) comparator this dictates whether deletion removes the maximum or the minimum element from the collection max heap --> maximum priority queue min heap --> minimum priority queue

86. ADSA: Heaps/11 86

87. ADSA: Heaps/11 87 The HeapPQueue Class public class HeapPQueue implements PQueue { private T[] heapElt; // the heap of queue elems private int numElts; // number of elements in queue private Comparator comp; public HeapPQueue() // create an empty maximum priority queue { comp = new Greater (); // so ordered big  small numElts = 0; heapElt = (T[]) new Object[10]; } // more methods... } // end of HeapPQueue class

88. ADSA: Heaps/11 88 public T peek() // return the highest priority item; O(1) { // check for an empty heap if (numElts == 0) throw new NoSuchElementException( "HeapPQueue peek(): empty queue"); return heapElt[0]; // return heap root } // end of peek()

89. ADSA: Heaps/11 89 // erase the highest priority item and return it public T pop() // O(log n) { // check for an empty priority queue if (numElts == 0) throw new NoSuchElementException( "HeapPQueue pop(): empty queue"); // pop heap and save return value in top T top = Heaps.popHeap(heapElt, numElts, comp); numElts--; // heap has one less element return top; } // end of pop()

90. ADSA: Heaps/11 90 public void push(T item) // insert item into the priority queue; O(log n) { // if full then double the capacity if (numElts == heapElt.length) enlargeCapacity(); // insert item into the heap Heaps.pushHeap(heapElt, numElts, item, comp); numElts++; } // end of push()

91. ADSA: Heaps/11 91 Heapify Analysis Heapify performs better than O(n log n) because most of the node adjusting is done over heights less than log n. Heapify performs better than O(n log n) because most of the node adjusting is done over heights less than log n. level height 0 1 2 3 3 = log n 2 1 0 Not examinable

92. ADSA: Heaps/11 92

93. ADSA: Heaps/11 93 Cost of Heap Building less than O(log n) most of the time

94. ADSA: Heaps/11 94 The summation converges to 2. Why? The summation converges to 2. Why?

95. ADSA: Heaps/11 95 Proof of Convergence The sum of a geometric series: The sum of a geometric series: Take the derivatives of both sides: Take the derivatives of both sides:

96. ADSA: Heaps/11 96