2014-T2 Lecture 29 School of Engineering and Computer Science, Victoria University of Wellington  Marcus Frean, Lindsay Groves, Peter Andreae and Thomas Kuehne, VUW COMP 103 Marcus Frean Priority Queues and Heaps

2 RECAP-TODAY RECAP  Linked Structures linked list idea  nice way to implement Stacks, or Queues  Tree Structures (in particular Binary Search Trees (BST)) BSTs idea  nice way to implement a Set, Bag, or Map in-order traversals  TreeSort TODAY  Priority Queue = variation on Queue – “best” first  Heap idea  nice way to implement Priority Queue HeapSort  Reading: Chapter 17 (section )

3 Queues and Priority Queues  Queues: Oldest out first  Priority Queues: Best out first  Emergency room, 111 calls, job scheduling in factory, etc...  Operating system process scheduling  Graph/network algorithms: route planning, find shortest/cheapest path BobJackJimIan Sometimes a lower number means higher priority Ian 3 Bob 5 Jack 1 Jim 2 Ian 5 Jack

4 Implementing Priority Queues  Unsorted list ( array or linked list ):  Fast to enqueue (“offer”): O(1)  Slow to dequeue (“poll”): O(n) need to search for highest priority item  Sorted list ( array or linked list ):  Fast to dequeue (“poll”): O(1) ‏  Slow to enqueue (“offer”): O(n) have to search for insertion point  Binary Search Tree  Fast to enqueue & dequeue (O(log n))  But, not cheap to keep balanced [if balanced]

5 Implementing Priority Queues: Ideas?  A BST is always fully sorted  supports ascending order through in-order traversal  Do we need this for Priority Queues?  what if we enqueue 1000 elements but only need the first three?  can we exploit that we are always just interested in a maximum element?  is there a “lazy” sorting strategy?

6 Implementing Priority Queues: POT idea  Abandon idea of total order  effort may never pay off  (almost) no need to worry about lower structure as long as we keep the priority item at the top  Only aim for partial order  Partially Ordered Tree  Fast to enqueue (“offer”): O(log n)  Fast to dequeue (“poll”): O(log n) ‏  Easy to keep balanced  Fast to construct from unordered list!

7 Partially Ordered Trees Binary Search Tree  Binary tree  All in left subtree < parent, All in right subtree ≥ parent Partially Ordered Tree  Binary tree  children ≤ parent,  Order of children not important Cat 19 Cat 19 Eel 26 Eel 26 Gnu 13 Gnu 13 Ant 9 Ant 9 Bee 35 Bee 35 Eel 26 Eel 26 Cat 19 Cat 19 Dog 14 Dog 14 Fox 3 Fox 3 Ant 9 Ant 9 Hen 23 Hen 23 Gnu 13 Gnu 13 Jay 2 Jay 2 Jay 2 Jay 2 Keep highest priority element at the root

8 Partially Ordered Tree: Adding  Easier to add and remove because the order is not complete  Add (draft)  start at top and navigate downwards to the right level  “bubble up” to correct position by swapping Bee 35 Bee 35 Eel 26 Eel 26 Cat 19 Cat 19 Dog 14 Dog 14 Fox 3 Fox 3 Ant 9 Ant 9 Hen 23 Hen 23 Gnu 13 Gnu 13 Jay 2 Jay 2 Fly 1 Fly 1

9 Partially Ordered Tree: Adding  Easier to add and remove because the order is not complete  Add  insert next to bottom-rightmost  “bubble up” to correct position by swapping Bee 35 Bee 35 Eel 26 Eel 26 Cat 19 Cat 19 Dog 14 Dog 14 Fox 3 Fox 3 Ant 9 Ant 9 Hen 23 Hen 23 Gnu 13 Gnu 13 Jay 2 Jay 2 Kea 24 Kea 24

10 Partially Ordered Tree: Removing  Easier to add and remove because the order is not complete  Remove (draft)  “pull up” largest child and recurse  But: tree may become unbalanced! Bee 35 Bee 35 Eel 26 Eel 26 Cat 19 Cat 19 Dog 14 Dog 14 Fox 3 Fox 3 Ant 9 Ant 9 Hen 23 Hen 23 Gnu 13 Gnu 13 Jay 2 Jay 2

11 Partially Ordered Tree: Removing II  Easier to add and remove because the order is not complete  Remove (better)  replace root by bottom-rightmost node  “sink down” to correct position by swapping  keeps tree balanced – and complete! Eel 26 Eel 26 Kea 24 Kea 24 Dog 14 Dog 14 Cat 19 Cat 19 Ant 9 Ant 9 Hen 25 Hen 25 Gnu 13 Gnu 13 Jay 2 Jay 2 Fox 3 Fox 3 Bee 35 Bee 35

12 Partially Ordered Tree: Removing II  Easier to add and remove because the order is not complete  Remove (better)  replace root by bottom-rightmost node  “sink down” to correct position by swapping  keeps tree balanced – and complete! Hen 23 Hen 23 Kea 24 Kea 24 Dog 14 Dog 14 Cat 19 Cat 19 Ant 9 Ant 9 Fox 3 Fox 3 Gnu 13 Gnu 13 Jay 2 Jay 2 Eel 26 Eel 26

13 Partially Ordered Tree: Removing II  Easier to add and remove because the order is not complete  Support required for  locating bottom-rightmost node [add & remove (II)]  child  parent [“bubble up” for add]  parent  child [“sink down” for remove] Hen 23 Hen 23 Kea 24 Kea 24 Dog 14 Dog 14 Cat 19 Cat 19 Ant 9 Ant 9 Fox 3 Fox 3 Gnu 13 Gnu 13 Jay 2 Jay 2 Eel 26 Eel 26 invariant repairers!

14 Heap  A complete, partially ordered, binary tree complete = every level full, except bottom, where nodes are to the left  Implemented in an array using breadth-first order Bee 35 Eel 26 Kea 19 Dog 14 Fox 7 Ant 9 Hen 23 Gnu 13 Jay 2 Cat 4 Bee 35 Eel 26 Kea 19 Dog 14 Fox 7 Ant 9 Hen 23 Gnu 13 Jay 2 Cat

15 Heap  Bottom right node is last element used  We can compute the index of parent and children of a node:  the children of nodei are at(2i+1) and (2i+2)  the parent of nodei is at (i-1)/2  Gasp! There are no gaps! Bee 35 Eel 26 Kea 19 Dog 14 Fox 7 Ant 9 Hen 23 Gnu 13 Jay 2 Cat Bee 35 Eel 26 Kea 19 Dog 14 Fox 7 Ant 9 Hen 23 Gnu 13 Jay 2 Cat 4

16 Heap: add Insert at bottom of tree and bubble up:  Put new item at end: 10  Compare with parent:p = (10-1)/2 = 4 ⇒ Fox/7  If larger than parent  swap  Compare with parent: p = (4-1)/2 = 1 ⇒ Kea/19  If larger than parent  swap  Compare with parent: p = (1-1)/2 = 0 ⇒ Bee/35 Bee 35 Eel 26 Kea 19 Dog 14 Fox 7 Ant 9 Hen 23 Gnu 13 Jay 2 Cat Pig 21 11

17 Heap: remove  Remove item at 0:  Move last item to 0  Find largest childc 1 = 2  0+1 = 1, c 2 = 2  0+2 = 2  if largest child is larger  swap  Find largest childc 1 = 2  2+1 = 5, c 2 = 2  2+2 = 6  if largest child is larger  swap  Find largest childc 1 = 2  5+1 = 11 : No such child Bee 35 Eel 26 Pig 21 Dog 14 Kea 19 Ant 9 Hen 23 Gnu 13 Jay 2 Cat Fox 7 11

18 HeapQueue: Analysis  Cost of offer: = cost of “bubble up” =O(log(n)) log(n) comparisons, 2 log(n) assignments  Cost of poll: = cost of “sink down” = O(log(n)) 2 log(n) comparisons, 2 log(n) assignments  Conclusion: HeapQueue is always fast!