Priority Queues & Heaps CS 261 – Data Structures Priority Queues & Heaps

Priority Queues Not really a FIFO queue – misnomer!! Associates a “priority” with each object: First element has the highest priority (typically, lowest value) A data structure that supports the priority queue interface: void add(newValue); TYPE getMin(); void removeMin(); Examples of priority queues: To do list with priorities Active processes in an OS

Priority Queues: Heaps Heap: has 2 completely different meanings Classic data structure used to implement priority queues Memory space used for dynamic allocation We will study the data structure (not dynamic memory allocation) Heap data structure: a complete binary tree in which every node’s value is less than or equal to the values of its children Review: a complete binary tree is a tree in which Every node has at most two children (binary) The tree is entirely filled except for the bottom level which is filled from left to right (complete) Longest path is ceiling(log n) for n nodes

Maintaining the Heap: Addition 2 Add element: 4 3 5 9 10 7 8 New element in next open spot. 14 12 11 16 4 Place new element in next available position, then fix it by “percolating up”

Heap: Example 2 3 5 9 10 7 8 14 12 11 16 Root = Smallest element Next open spot Last filled position (not necessarily the last object added)

Maintaining the Heap: Addition (cont.) 2 3 5 9 10 4 8 2 14 12 11 16 7 3 4 After first iteration (swapped with 7) 9 10 5 8 14 12 11 16 7 Percolating up: while new value is less than parent, swap value with parent After second iteration (swapped with 5) New value not less than parent  Done

Maintaining the Heap: Removal Since each node’s value is less than or equal to the values of its children, the root is always the smallest element Thus, the methods getMin and removeMin access and remove the root node, respectively Heap removal (removeMin): Replace root with the element in the last filled position Fix heap by “percolating down”

Maintaining the Heap: Removal removeMin : Move value in last element into root 2. Percolate down Root = Smallest element 2 3 5 9 10 7 8 14 12 11 16 Last filled position

Maintaining the Heap: Removal (cont.) 16 3 5 9 10 7 8 3 14 12 11 16 5 Root object removed (16 copied to root and last node removed) 9 10 7 8 14 12 11 After first iteration (swapped with 3) Percolating down: while greater than smallest child swap with smallest child

Maintaining the Heap: Removal (cont.) 3 9 5 16 10 7 8 8 3 14 12 11 9 5 After second iteration (moved 9 up) 12 10 7 8 14 16 11 After third iteration (moved 12 up) Reached leaf node  Stop percolating Percolating down: while greater than smallest child swap with smallest child

Maintaining the Heap: Removal (cont.) Root = New smallest element 3 9 5 12 10 7 8 14 16 11 New last filled position

Heap Representation Recall that a complete binary tree can be efficiently represented by an array or a vector: Children of node at index i are stored at indices Parent of node at index i is at index 2i + 1 and 2i + 2 floor((i - 1) / 2) 2 1 3 5 9 4 10 7 6 8 14 12 11 16

Heap Implementation: add (cont.) void addHeap(struct DynArr *heap, TYPE val) { int pos = sizeDynArr(heap); /* Get index of next open spot. */ int up = (pos – 1) / 2; /* Get parent index (up the tree). */ TYPE parent; if (pos >= heap->cap) /* Make room for new element (but */ _setCapDynArr(heap, 2 * heap->cap); /* don’t add it yet). */ if (pos > 0) parent = heap->data[up]; ... } Example: add 4 to the heap Prior to addition, size = 11 Parent position (up) Next open spot (pos) 2 1 3 2 5 3 9 4 10 5 7 6 8 7 14 8 12 9 11 10 16 11 4

Creating New Abstraction When we create a heap, we could either Pass the container directly as a dynamic array, or Create a new structure for our heap struct Heap { struct DynArr arr; /* Or could re-implement DynArr. */ }; One leverages existing structure (less work), but presents conflicting ADT interfaces implemented by DynArr (e.g., Stack, Bag, Heap, etc.) Other is better encapsulation, but more work

Heap Implementation: add (cont.) void addHeap(struct DynArr *heap, TYPE val) { ... /* While not at root and new value is less than parent. */ while (pos > 0 && LT(val, parent)) { heap->data[pos] = parent; /* New value is smaller  move parent down. */ pos = up; up = (pos - 1) / 2; /* Move position and compute parent idx. */ if (pos > 0) parent = heap->data[up]; /* Get parent (if not at root). */ } After first iteration: “swapped” new value (4) with parent (7) New parent value: 5 up pos 2 1 3 2 5 3 9 4 10 5 4 6 8 7 14 8 12 9 11 10 16 11 7

Heap Implementation: add (cont.) void addHeap(struct DynArr *heap, TYPE val) { ... /* While not at root and new value is less than parent. */ while (pos > 0 && LT(val, parent)) { heap->data[pos] = parent; /* New value is smaller  move parent down. */ pos = up; up = (pos - 1) / 2; /* Move position and compute parent idx. */ if (pos > 0) parent = heap->data[up]; /* Get parent (if not at root). */ } After second iteration: “swapped” new value (4) with parent (5) New parent value: 2 up pos 2 1 3 2 4 3 9 4 10 5 6 8 7 14 8 12 9 11 10 16 11 7

Heap Implementation: add (cont.) void addHeap(struct DynArr *heap, TYPE val) { ... /* While not at root and new value is less than parent. */ while (pos > 0 && LT(val, parent)) { heap->data[pos] = parent; /* New value is smaller  move parent down. */ pos = up; up = (pos - 1) / 2; /* Move position and compute parent idx. */ if (pos > 0) parent = heap->data[up]; /* Get parent (if not at root). */ } /* End while: reached root or parent is smaller than new value. */ heap->data[pos] = val; /* Now add new value. */ } /* Done. */ Second part of while test fails: stop iteration and add new value up pos 2 1 3 2 4 3 9 4 10 5 6 8 7 14 8 12 9 11 10 16 11 7

Heap Implementation: add (cont.) 2 3 5 9 10 7 8 up 14 12 11 16 4 pos 2 1 3 2 5 3 9 4 10 5 7 6 8 7 14 8 12 9 11 10 16 11 4

Heap Implementation: add (cont.) 2 3 5 up 9 10 4 8 pos 14 12 11 16 7 2 1 3 2 5 3 9 4 10 5 4 6 8 7 14 8 12 9 11 10 16 11 7

Heap Implementation: add (cont.) 2 3 4 up 9 10 5 pos 8 14 12 11 16 7 2 1 3 2 4 3 9 4 10 5 6 8 7 14 8 12 9 11 10 16 11 7

Heap Implementation: add (cont.) 2 3 4 9 10 5 pos 8 14 12 11 16 7 2 1 3 2 4 3 9 4 10 5 6 8 7 14 8 12 9 11 10 16 11 7

Heap Implementation: removeMin void removeMinHeap(struct DynArr *heap) { int last = sizeDynArr(heap) - 1; if (last != 0) /* Copy the last element to */ heap->data[0] = heap->data[last]; /* the first position. */ removeAtDynArr(heap, last); /* Remove last element. */ _adjustHeap(heap, last, 0); /* Rebuild heap property. */ } First element (data[0]) Last element (last) 2 1 3 2 4 3 9 4 10 5 6 8 7 14 8 12 9 11 10 16 11 7 7 1 3 2 4 3 9 4 10 5 6 8 7 14 8 12 9 11 10 16 11

Heap Implementation: removeMin (cont.) 2 3 4 9 10 5 8 14 12 11 16 7 last 2 1 3 2 4 3 9 4 10 5 6 8 7 14 8 12 9 11 10 16 11 7

Heap Implementation: removeMin (cont.) 7 3 4 9 10 5 8 New root 14 12 11 16 max 7 1 3 2 4 3 9 4 10 5 6 8 7 14 8 12 9 11 10 16 11

Heap Implementation: removeMin (cont.) 7 val > min  Copy min to parent spot (idx) Move idx to child 3 4 val = 7 9 10 5 8 Smallest child (min = 3) 14 12 11 16 max 7 1 3 2 4 3 9 4 10 5 6 8 7 14 8 12 9 11 10 16 11

Heap Implementation: removeMin (cont.) 3 val > min  Copy min to parent spot (idx) Move idx to child 3 4 val = 7 9 10 5 8 idx 14 12 11 16 max 3 1 3 2 4 3 9 4 10 5 6 8 7 14 8 12 9 11 10 16 11

Heap Implementation: removeMin (cont.) 3 val < min  Copy val into idx spot Done 7 4 val = 7 9 10 5 8 idx 14 12 11 16 max Smallest child (min = 9) 3 1 7 2 4 3 9 4 10 5 6 8 7 14 8 12 9 11 10 16 11

Heap Implementation: removeMin (cont.) 3 val < min  Copy val into idx spot Done 7 4 val = 7 9 10 5 8 idx 14 12 11 16 max 3 1 7 2 4 3 9 4 10 5 6 8 7 14 8 12 9 11 10 16 11

Useful Routines void swap(struct DynArr *arr, int i, int j) { /* Swap elements at indices i and j. */ TYPE tmp = arr->data[i]; arr->data[i] = arr->data[j]; arr->data[j] = tmp; } int minIdx(struct DynArr *arr, int i, int j) { /* Return index of smallest element value. */ if (LT(arr->data[i], arr->data[j]) return i; return j;

Recursive _adjustHeap void _adjustHeap(struct DynArr *heap, int max, int pos) { int leftIdx = pos * 2 + 1; int rghtIdx = pos * 2 + 2; if (rghtIdx < max) { /* Have two children? */ /* Get index of smallest child (_minIdx). */ /* Compare smallest child to pos. */ /* If necessary, swap and call _adjustHeap(max, minIdx). */ } else if (leftIdx < max) { /* Have only one child. */ /* Compare child to parent. */ /* If necessary, swap and call _adjustHeap(max, leftIdx). */ /* Else no children, we are at bottom  done. */

Priority Queues: Performance Evaluation So, which is the best implementation of a priority queue? SortedVector SortedList Heap add O(n) Binary search Slide data up Linear search O(log n) Percolate up getMin O(1) get(0) Returns head.obj Get root node removeMin Slide data down O(1): Reverse Order removeFront() Percolate down

Priority Queues: Performance Evaluation Remember that a priority queue’s main purpose is rapidly accessing and removing the smallest element! Consider a case where you will insert (and ultimately remove) n elements: ReverseSortedVector and SortedList: Insertions: n * n = n2 Removals: n * 1 = n Total time: n2 + n = O(n2) Heap: Insertions: n * log n Removals: n * log n Total time: n * log n + n * log n = 2n log n = O(n log n)

Priority Queue Application: Simulation Original, and one of most important, applications Discrete event driven simulation: Actions represented by “events” – things that have (or will) happen at a given time Priority queue maintains list of pending events  Highest priority is next event Event pulled from list, executed  often spawns more events, which are inserted into priority queue Loop until everything happens, or until fixed time is reached

Priority Queue Applications: Example Discrete event-driven simulation – i.e., a simulation of a process where events occur Example: Ice cream store People arrive People order People leave Simulation algorithm: Determine time of each event using random number generator with some distribution Put all events in priority queue based on when it happens Simulation framework pulls minimum (next to happen) and executes event

Priority Queue Applications Many types of events … but they all have a time when they occur Can we store various types of events in a priority queue? Heterogeneous collection Keep a pointer to action, or a flag to tell what action to perform Use the less-than macro, LT(A, B), to compare based on time

Your Turn Lesson 22: Heaps and Priority Queues Complete functions: _adjustHeap and addHeap Next lecture: Building heaps and heap sort