1. Cpt S 223 – Advanced Data Structures Priority Queues
(Heaps)
Nirmalya Roy
School of Electrical Engineering and Computer Science
Washington State University

2. Motivation
Queues are a standard mechanism for ordering tasks on a first-come, first-served basis
However, some tasks may be more important or timely than others (higher priority)
Priority queues
Store tasks using a partial ordering based on priority
Ensure highest priority task at head of queue
Heaps are the underlying data structure of priority queues

3. Priority Queues Main operations
insert (i.e., enqueue)
Dynamic insert
Specification of a priority level (0 (high), 1, 2, …, low)
deleteMin (i.e., dequeue)
Finds the current minimum element (element with highest priority) in the queue, deletes it from the queue, and returns it
Performance goal is for operations to be “fast”

4. Priority Queue Implementations
Unordered list
O(1) for insert
O(N) for deleteMin
Ordered list
O(N) for insert
O(1) for deleteMin
Balanced BST
O(log2 N) for insert and deleteMin
Can it be done faster?

5. A priority queue data structure
Binary Heap
A priority queue data structure

6. Binary Heap A binary heap is a binary tree with two properties
Structure property
Heap-order property

7. Structure Property of Binary Heap
A binary heap is a complete binary tree
Each level is completely filled
Bottom level may be partially filled from left to right
Height of a complete binary tree with N elements is log2 N

8. Binary Heap Example N = 10 Every level (except last) completely filled
Array representation of the binary heap above:

9. Not a Binary Heap A B C D E F H G
Why is the above cannot be a binary heap?

10. Heap-Order Property of Binary Heap
Heap-order property (for a “min binary heap”)
For every node X, key(parent(X))  key(X)
Except root node, which has no parent
Thus, minimum key always at the root
Alternatively, for a “max binary heap”, always keep the maximum key at the root
insert and deleteMin must always maintain heap-order property

11. Heap-Order Property Example
An example of a min binary heap 13 14 16 19 21 68 26 65 31 32
Duplicates are allowed
No order implied for elements at the same tree level or among siblings

12. Not a Min Binary Heap The following is NOT a min binary heap 17 14 1619 21 68 26 65 11 31 32
Why is the above NOT a min binary heap?

13. Heap-Order Property Example
An example of a max binary heap 30 28 27 20 11 2 15 13 1 9
Duplicates are allowed
No order implied for elements at the same tree level or among siblings

14. Not a Max Binary Heap The following is NOT a max binary heap 30 32 2720 11 2 15 13 1 12
Why is the above NOT a max binary heap?

15. Implementing Complete Binary Trees as Arrays
Given an element at position i in the array
i’s left child is at position 2i
i’s right child is at position 2i + 1
i’s parent is at position i / 2
Heap data structures
Draw the heap as trees with the implication that an actual implementation will use simple arrays
An estimate of the maximum heap size is required in advance

16. An array (of Comparable objects) and an integer for heap size
Fix heap after deleteMin

17. Heap Insert
Insert new element into the heap at the next available slot (“hole”)
Needed to maintain a complete binary tree
Then, “percolate” (or “trickle”) the element up the heap while min heap-order property is not satisfied
This strategy is know as percolate up

18. Heap Insert: Example 14 vs. 31 Insert 14: 14 vs. 21 14 vs. 13
Heap-order okay; structure okay
Percolating up

19. Heap Insert: Implementation
O(log N) time
Insert 14

20. Heap DeleteMin Minimum element is always at the root
A hole is created at the root when minimum element is removed
Heap decreases by one in number of elements (or size)
Last element X should be moved somewhere in the heap
If it can be placed in the hole, then we are done
Push smaller of the hole’s children into the hole, thus pushing the hole down one level
Repeat this step until X can be placed in the hole
Percolate down until the heap-order property is not restored

21. Heap DeleteMin: Example
Remove value at the root
Move 31 (last element) to the root
Is 31 > min(14, 16)?
If yes, swap 31 with min(14, 16)
If no, leave 31 in hole

22. Heap DeleteMin: Example (cont’d)
Percolating down…
Is 31 > min(19, 21)?
If yes, swap 31 with min(19, 21)
If no, leave 31 in hole
Is 31 > min(65, 26)?
If yes, swap 31 with min(65, 26)
If no, leave 31 in hole

23. Heap DeleteMin: Example (cont’d)
Percolating down…
Heap-order property okay;
Structure okay;
Done.

24. Heap DeleteMin: Implementation
O(log N) time

25. Heap DeleteMin: Implementation (cont’d)
Percolating down…
Left child
Right child
Pick child to swap with

26. Other Heap Operations decreaseKey(p, v) increaseKey(p, v) remove(p)
Lowers value of item p to v
Need to percolate up
E.g., change job priority
increaseKey(p, v)
Increases value of item p to v
Need to percolate down
remove(p)
First, decreaseKey(p, -)
Then, deleteMin
E.g., terminate job
Run-times for all 3 functions?
Assume: We already know the
location of node containing p

27. Building a Heap
What if all N elements are all available upfront to build the heap?
To build a heap with N elements
Default method takes O(N log N) time.
New method will take O(N) time – i.e., optimal

28. Building a Heap (cont’d)
Construct a heap from an initial set of N items
Solution 1
Perform N successive inserts
O(1) constant time on average and O(log N) worst-case
O(N) average case, but O(N log2 N) worst-case
Solution 2
Randomly populate initial heap with structure property
Perform a percolate-down from each internal node (i.e., H[size/2] to H[1])
To take care of heap-order property
Insert 33 or
Insert 14

29. Building a Heap: Example
Input: 150, 80, 40, 30, 10, 70, 110, 100, 20, 90, 60, 50, 120, 140, 130
So, let us look at each internal
node, and fix if necessary
Randomly initialize heap
Structure property satisfied
Heap-order property NOT satisfied
Leaves are all valid heaps (implicit)

30. Building a Heap: Example (cont’d)

31. Building a Heap: Example (cont’d)
Dotted lines show the path of percolating down

32. Building a Heap: Example (cont’d)
Dotted lines show the path of percolating down

33. Building a Heap: Example (cont’d)
Dotted lines show the path of percolating down

34. Building a Heap: Analysis
Running time = ?
O(sum of the heights of all internal nodes)
Theorem 6.1
For the perfect binary tree of height h containing 2h+1 – 1 nodes, the sum of heights of the nodes is 2h+1 – 1 – (h + 1)
Proof. Tree consists of 1 node at height h, 2 nodes at height (h-1), 22 nodes at height (h-2) and in general 2i nodes at height (h-i)
The sum of the heights of all the nodes
Since h = log N, then the sum of heights is O(N)
Will be slightly better in practice
Implication: Each insertion costs O(1) amortized time

35. Binary Heap Operations Worst-Case Analysis
Height of heap is log2 N
insert: O(log2 N)
2.607 comparisons on average, i.e., O(1)
deleteMin: O(log2 N)
decreaseKey: O(log2 N)
increaseKey: O(log2 N)
remove: O(log2 N)
buildHeap: O(N)

36. Building a Heap: Implementation
Start with the lowest, rightmost internal node

37. Applications of Heaps Operating system scheduling Graph algorithms
Process jobs by priority
Graph algorithms
Find shortest path
Event simulation
Instead of checking for events at each time click, look up next event to happen

38. An Application: The Selection Problem
Given a list of n elements, find the kth smallest element
Solution 1: Read the elements into an array and sort them, returning the appropriate element
Running time is O(n2)
Solution 2: Using simple sorting algorithm
Sort the list (O(n log n))
Pick the kth element
A better approach
Use a MinHeap

39. Selection Using a MinHeap
Input: n elements
Algorithm:
buildHeap(n elements) ==> O(n)
Perform k deleteMin() operations ==> O(k lg n)
Report the kth deleteMin() output
Total run-time = O(n + k lg n)
If k = O(n / lg n) then the run-time becomes O(n)

40. Other Types of Heaps d-Heaps Leftist heaps Skew heaps
Generalization of binary heaps (i.e., 2-heaps)
All nodes have d children
Insert in O(logd N) time; deleteMin in O(d logd N) time
Leftist heaps
Like a binary heap, it has both a structural and heap order property
Binary tree, but not perfectly balanced
Supports merging of two heaps in O(m + n) time (i.e., sub-linear)
Skew heaps
Binary tress with heap order but without any structural constraints
O(log n) amortized run-time

41. Mergeable Heaps Heap merge operation
Useful for many applications
Merge two (or more) heaps into one
Identify new minimum element
Maintain heap-order property
Merge in O(log N) time
Still support insert and deleteMin in O(log N) time
Insert = merge existing heap with one-element heap
d-Heaps require O(N) time to merge

42. Leftist Heaps Null path length npl(X) of node X Leftist heap property
Length of the shortest path from X to a node without two children
Leftist heap property
For every node X in heap, npl(leftChild(X)) ≥ npl(rightChild(X))
Leftist heaps have deep left subtrees and shallow right subtrees
Thus if operations reside in right subtree, they will be faster

43. Leftist Heaps
npl(X) shown in nodes
Leftist heap
Not a leftist heap

44. Leftist Heaps Merge heaps H1 and H2 DeleteMin
Assume root(H1) > root(H2)
Recursively merge H1 with right subheap of H2
If result is not leftist, then swap the left and right subheaps
Running time O(log N)
DeleteMin
Delete root and merge children

45. Leftist Heaps: Example
Merge H2 (larger root) with right sub-heap of H1 (smaller root)
Root 6 of H2 > Root 3 of H1
Right subheap

46. Leftist Heaps: Example
Merge H2 (larger root) with right sub-heap of H1 (smaller root)

47. Leftist Heaps: Example
Attach previous heap as H1’s right child.
Leftist heap?

48. Leftist Heaps: Example
Swap root’s children to make leftist heap.

49. Skew Heaps Self-adjusting version of leftist heap
The relationship of Skew heaps are analogous to leftist heaps as splay trees are to AVL trees
Skew merge same as leftist merge, except we always swap left and right subheaps
No need to maintain or test NPL of nodes
Worst case is O(N)
Amortized cost of M operations is O(M log N)

50. Binomial Heaps

51. Binomial Heap A forest of heap-ordered binomial trees
Structure property
Heap-order property
The structural property of each heap-ordered tree in binomial heap is different from binary heap
The heap-order property applies to each binomial tree

52. Definition: A “Binomial Tree” Bk
A binomial tree of height k is called Bk
It has 2k nodes
The number of nodes at depth d is the binomial co-efficient
A binomial tree need not to be a binary tree
is the form of the coefficients in binomial theorem
Depth B3
# of nodes
d = 0
d = 1
d = 2
d = 3

53. Binomial Heap with 31 Nodes
We know:
Binomial heap should be a forest of binomial trees
Each binomial tree has power of 2 elements
How many binomial trees do we need?
n = 31 = B4 B3 B2 B1 B0

54. Binomial Heap with 31 Nodes (cont’d)
Bk == Bk-1 + Bk-1 B4 B2 B2 B3 B3
A binomial tree Bk consists of a root with children B0, B1, B2 ….Bk-1
How many trees are there in this forest if the # of nodes are N?

55. Binomial Heap Property
Lemma: There exists a binomial heap for every positive value of n
Proof:
Any arbitrary n can be represented in powers of two (binary representation)
Have one binomial tree for each power of two with co-efficient of 1
e.g., if n = 10
Binary Representation ==> (1010)2 ==> forest contains {B3, B1}

56. Binomial Heaps: Heap-Order Property
Each binomial tree should contain the minimum element at the root of every subtree
Just like binary heap, except that the tree here is a binomial tree structure (and not a complete binary tree)
The order across binomial trees is irrelevant

57. Definition of Binomial Heaps
A binomial heap of n nodes is:
A forest of binomial trees as dictated by the binary representation of n (this is the structure property)
Each binomial tree is a min-heap or a max-heap (this is the heap-order property)

58. Binomial Heaps: Example

59. Key Properties
Could there be multiple trees of the same height in a binomial heap? No
How many trees could there be, in the worst-case, in a binomial heap of n nodes? log n
Given n, can we tell (for sure) how many trees will there be of a given height k? Check if bit is 1 in the kth LSB that means Bk exists if and only if the kth least significant bit is 1 in the binary representation of n

60. Binomial Heap: Operations
insert(x)
merge(H1, H2)
x = deleteMin()

61. insert(x) in Binomial Heap
Goal
To insert a new element x into a binomial heap H
Observation
Element x can be viewed as a single element binomial heap
insert(x) == merge(H, {x})
So, if we decide how to do merge we will automatically figure out how to implement both insert() and deleteMin().

62. merge(H1, H2) Let n1 be the number of nodes in H1
Therefore, the new heap is going to have n1 + n2 nodes
Logic:
Merge trees of same height, starting from lowest height trees
If only one tree of a given height, then just copy that
May need carryover (just like adding two binary numbers)

63. merge(H1, H2): Example
carryover + B0 B1 B2 13 14 26 16 ? B2

64. merge(H1, H2): Example Result
Merge takes O(log n) time (i.e. comparisons).

65. Merging Two Binomial Trees of the Same Height+ =
Simply compare the roots.
Merge is defined only for binomial trees with the same height

66. Merging Two Binomial Trees of the Same Height (cont’d)

67. Merging More Than Two Trees
Merging three or more binomial trees of the same height could generate carry-overs 12 23 7 21 24 + 51 24 + 10 8
= ? 65 65 11
Merge any two and leave the third as carry-over.

68. findMin()
Goal: Given a binomial heap, H, find the minimum and delete it
Observation: The root of each binomial tree in H contains its minimum element
Approach: Therefore, return the minimum of all the roots (minimums)
Complexity: O(log n) comparisons

69. findMin() and deleteMin() Example
Find the binomial tree with smallest root say Bk and original priority queue let be H
Remove Bk from the forest of trees in H, forming the new binomial queue H’
Remove the root of Bk creating binomial trees B0, B1, B2 ….Bk-1 which collectively
forms H’’
Merge H’ and H’’ B0 B2 B3
B0’
B1’
B2’
For deleteMin(): After delete, how to adjust the heap?
New heap: Merge {B0, B2} and {B0’, B1’, B2’}.

70. Binomial Queue Run-time Summary
insert
O(log n) worst-case
O(1) amortized time if insertion is done in a sequence
deleteMin, findMin
merge

71. An Implementation of a Binomial Heap
Array of binomial trees, children of each node are kept in a linked list
Trees use first-child, right-sibling representation
Analogous to a bit-based representation of a binary number n B7 B6 B5 B4 B3 B2 B1 B0
Maintain a linked list of tree pointers (for the forest)
What is the value of n?

72. Priority Queues in STL
Binary Heap is implemented by the class template named priority_queue found in standard header file queue
Default is max-heap
Methods
Push, pop, top, empty, clear
Duplicates allowed, only one largest is removed if several
#include <queue>
#include <iostream>
using namespace std;
int main(void) {
priority_queue<int> Q;
for (int i = 0; i < 100; i++)
Q.push(i);
while (!Q.empty()) {
cout << Q.top() << endl;
Q.pop(); }
Calls deleteMax()
For min-heap, declare priority queue as:
priority_queue<int, vector<int>, greater<int> > Q;

73. Run-time Per Operation
Cpt S 223
Run-time Per Operation
Types of Heap
Insert
DeleteMin
Merge (=H1+H2)
Binary Heap
O(log n) worst-case
O(1) amortized for buildHeap (constant time on average)
O(log n)
O(n)
Leftist Heap
Skew Heap
Binomial Heap
O(log n) worst case
O(1) amortized for sequence of n inserts (constant time on average)
Washington State University

74. Summary
Priority queues maintain the minimum or maximum element of a set
Standard binary heap implementation is elegant for its simplicity and speed
It requires no links and only a constant amount of space
Leftist heap is a wonderful example of power of recursion
Binomial queue is a simple idea to achieve good time bound
Support O(log N) operations in worst-case
insert, deleteMin, merge
Many applications of priority queues ranging from operating systems scheduling to simulation