1. CSE 326 Heaps and the Priority Queue ADT
David Kaplan
Dept of Computer Science & Engineering
Autumn 2001

2. Back to Queues Some applications Problems? ordering CPU jobs
simulating events
picking the next search site
Problems?
short jobs should go first
earliest (simulated time) events should go first
most promising sites should be searched first
Alright, now we all remember what a tree is. Let’s harken back to queues now.
Here are some possible applications of a queue which aren’t quite right.
All of these want to favor certain entries in the Queue. How do we handle that?
Heaps
CSE 326 Autumn

3. Priority Queue ADT Priority Queue operations
create
destroy
insert
deleteMin
is_empty
Priority Queue property: for two elements in the queue, x and y, if x has a lower priority value than y, x will be deleted before y
F(7) E(5)
D(100) A(4)
B(6)
insert
deleteMin
G(9)
C(3)
We need a new ADT for this. One that returns the best (which we’ll generally define as lowest priority value) node next.
I’m often going to represent priority queues as collections of priorities; remember that they are collections of data with priorities. MOREOVER, some priority queues don’t even need priorities, just the ability to compare priorities. (all of the ones we’ll talk about fall in this category).
Heaps
CSE 326 Autumn

4. Applications of the Priority Q
Hold jobs for a printer in order of length
Store packets on network routers in order of urgency
Simulate events
Select symbols for compression
Sort numbers
Anything greedy
There are, of course, gobs of applications for priority queues. Here are just a few.
Heaps
CSE 326 Autumn

5. Naïve Priority Q Data Structures
Unsorted list:
insert:
deleteMin:
Sorted list:
OK, let’s look at some ways we might implement priority Qs.
Unsorted lists give us fast inserts, but we have to search the whole list to delete.
Sorted lists give us fast delete (it’s just the first element!), but we have to either search the whole list (for linked list) or move the whole list (for array) on an insert.
Heaps
CSE 326 Autumn

6. Binary Search Tree Priority Q Data Structure (aka BST PQD :-)8
insert:
deleteMin: 5 11 2 6 10 12
A binary search tree is a binary tree in which all nodes in the left subtree of a node have lower values than the node. All nodes in the right subtree of a node have higher value than the node.
We’ll talk more about these later. For now, let’s get a feel for how well they work as PQ data structures.
How long might an insert take here? (log n)
How about deleteMin?
It’s always the leftmost node in the tree, right? So, it might take log n time to find the node.
What about worst case?
The book says that these lopsided deletes don’t cause a problem. Ask me about that via if you’re curious why not. 4 7 9 14 13
Heaps
CSE 326 Autumn

7. Binary Heap Priority Q Data Structure
Heap-order property
parent’s key is less than children’s keys
result: minimum is always at the top
Structure property
complete tree: fringe nodes packed to the left
result: depth is always O(log n); next open location always known 13 14 15 9 11 8 10 6 7 5 4 2
Alright, there are two problems with binary search trees as pqs.
First, they’re overkill. Why keep everything ordered? We just need to know the least at any given time.
Second, they’re not guaranteed to be complete, so we have worst case O(n) times.
How do we find the minimum?
Heaps
CSE 326 Autumn

8. Nifty Storage Trick Calculations 13 14 15 9 11 8 10 6 7 5 4 2
children:
parent:
root:
next free: 2 3 4 5 6 7 8 9 10 11 12
It so happens that we can store a complete tree in an array and still be able to find children and parents easily. We’ll take advantage of this.
Child:
left = 2*node
right = 2*node + 1
parent = floor(node/2)
nextfree = length + 1
root = 1
What do we 1 2 3 4 5 6 7 8 9 10 11 12 13 13 2 4 5 7 6 10 8 11 9 15 14 13
Note: Walking the array in index order gives us level-order traversal!!!
Heaps
CSE 326 Autumn

9. BinaryHeap::DeleteMin2
pqueue.deleteMin() 13 14 15 9 11 8 10 6 7 5 4 2 ? 4 5 7 6 10 8
OK, to delete, we start by plucking out that value in O(1) time.
Now, we have a hole at the top, and a node that isn’t proper for a complete tree at the bottom.
So, we fix it by putting the value in the hole, and moving the hole until we restore the heap-order property. 11 9 15 14 13
Heaps
CSE 326 Autumn

10. Percolate Down ? 13 4 5 4 5 7 6 10 8 7 6 10 8 11 9 15 14 13 11 9 15 14 4 4 13 5 6 5 7 6 10 8 7 13 10 8 11 9 15 14
Done! 11 9 15 14

11. DeleteMin Code runtime: Object deleteMin() { assert(!isEmpty());
returnVal = Heap[1];
size--;
newPos =
percolateDown(1,
Heap[size+1]);
Heap[newPos] =
Heap[size + 1];
return returnVal; }
int
percolateDown(int hole, Object val) {
while (2*hole <= size) {
left = 2*hole;
right = left + 1;
if (right <= size &&
Heap[right] < Heap[left])
target = right;
else
target = left;
if (Heap[target] < val) {
Heap[hole] = Heap[target];
hole = target; }
break;
return hole;
Note that there are three things going on here: find the smaller child
if the smaller child is still smaller than the moving value, move the smaller child up
otherwise, we’ve found the right spot, and stop.
runtime:
Heaps
CSE 326 Autumn

12. BinaryHeap::Insert pqueue.insert(3) 20 14 12 9 11 8 10 6 7 5 4 2 2 4 5
Inserting works similarly. We tack a hole on at the end of the tree, and move that hole _up_ until it’s in the right place for the new value. 11 9 12 14 20 ?
Heaps
CSE 326 Autumn

13. Percolate Up 2 2 4 5 4 5 3 7 6 10 8 7 6 ? 8 3 11 9 12 14 20 ? 11 9 12 14 20 10 2 2 3 4 ? 4 3 7 6 5 8 7 6 5 8 11 9 12 14 20 10 11 9 12 14 20 10

14. Insert Code runtime: void insert(Object o) { assert(!isFull());
size++;
newPos =
percolateUp(size,o);
Heap[newPos] = o; }
int percolateUp(int hole,
Object val) {
while (hole > 1 &&
val < Heap[hole/2])
Heap[hole] = Heap[hole/2];
hole /= 2; }
return hole;
Notice that this code is a _lot_ easier.
runtime:
Heaps
CSE 326 Autumn

15. Performance of Binary Heap
worst case
Binary heap avg case
AVL tree worst case
AVL tree avg case
Insert
O(log n)
O(1)
percolates ~1.6 levels
DeleteMin
In practice: binary heaps much simpler to code, lower constant factor overhead
Heaps
CSE 326 Autumn

16. Changing Priorities
In many applications the priority of an object in a priority queue may change over time
if a job has been sitting in the printer queue for a long time increase its priority
unix “renice”
Sysadmin may raise priority of a critical task
Must have some (separate) way to find the position in the queue of the object to change (e.g. a hash table)
No log(N) find (as with BSTs) – why not?
Heaps
CSE 326 Autumn

17. Other Priority Queue Operations
decreaseKey
given a pointer to an object in the queue, reduce its priority value
increaseKey
given a pointer to an object in the queue, increase its priority value
remove
given a pointer to an object in the queue, remove it
buildHeap
given a set of items, build a heap
Why do I insist on a pointer to each item in decreaseKey, increaseKey, and remove?
Because it’s hard to find an item in a heap; it’s easy to delete the minimum, but how would you find some arbitrary element?
This is where the Dictionary ADT and its various implementations which we will study soon come up.
What about build heap? Is that really a necessary operation?
It turns out that the necessity for this is based _purely_ on the fact that we can do it faster than the naïve implementation!
Heaps
CSE 326 Autumn

18. DecreaseKey, IncreaseKey, Remove
void decreaseKey(int obj) {
assert(size >= obj);
temp = Heap[obj];
newPos = percolateUp(obj, temp);
Heap[newPos] = temp; }
void remove(int obj) {
assert(size >= obj);
percolateUp(obj,
NEG_INF_VAL);
deleteMin(); }
Note: changeKey functions assume that key value has already been changed!
void increaseKey(int obj) {
assert(size >= obj);
temp = Heap[obj];
newPos = percolateDown(obj, temp);
Heap[newPos] = temp; }
The code for these is pretty simple. Notice that these are called just after the key is changed, they do not include the new value (just assume it’s there).
This is different from the book’s version. And, it’s not necessarily better.
Heaps
CSE 326 Autumn

19. BuildHeap (Floyd’s Method)12 5 11 3 10 6 9 4 8 1 7 2
pretend it’s a heap and fix the heap-order property! 12 5 11
Pull other slide back up and ask how long it takes.
O(n log n) worst case, but O(n) average case.
Can we get a worst case O(n) bound?
Let’s try pretending it’s a heap already and just fixing the heap-order property.
The red nodes are the ones that are out of order.
Question: which nodes MIGHT be out of order in any heap? 3 10 6 9
Thank you, Floyd! 4 8 1 7 2
Heaps
CSE 326 Autumn

20. Build(this)Heap 12 12 5 11 5 11 3 10 2 9 3 1 2 9 4 8 1 7 6 4 8 10 7 6 12 12 5 2 1 2 3 1 6 9 3 5 6 9 4 8 10 7 11 4 8 10 7 11

21. Finish Build(ing)(this)Heap1 3 2 4 5 6 9
How long does this take? Well, everything above the fringe might move 1 step.
Everything height 2 or greater might move 2 steps. …
In other words n * Sum I=0 to n of I/2i
But, this is (as n->inf) just 2. See chapter I to see why.
So, the runtime is O(n). 12 8 10 7 11
Runtime?
Heaps
CSE 326 Autumn

22. Complexity of Build Heap
Note: size of a perfect binary tree doubles (+1) with each additional layer
At most n/4 percolate down 1 level at most n/8 percolate down 2 levels at most n/16 percolate down 3 levels…
Comp
O(n)
Heaps
CSE 326 Autumn

23. Thinking about Heaps Observations Realities
finding a child/parent index is a multiply/divide by two
operations jump widely through the heap
each operation looks at only two new nodes
inserts are at least as common as deleteMins
Realities
division and multiplication by powers of two are fast
looking at one new piece of data sucks in a cache line
with huge data sets, disk accesses dominate
Now, let’s look at some facts about heaps and see if we can’t come up with a new priority queue implementation.
Heaps
CSE 326 Autumn

24. Solution: d-Heaps
Each node has d children
Still representable by array
Good choices for d:
optimize performance based on # of inserts/removes
d = 2k for efficiency (array index calcs)
fit one set of children in a cache line
fit one set of children on a memory page/disk block 1 3 7 2 4 8 5 12 11 10 6 9
D-heaps address all these problems. 12 1 3 7 2 4 8 5 12 11 10 6 9
What do d-heaps remind us of???
Heaps
CSE 326 Autumn

25. Merging Heaps Given two heaps, merge them into one heap
first attempt: insert each element of the smaller heap into the larger.
runtime:
second attempt: concatenate heaps’ arrays and run buildHeap.
What if you had networked printers or CPUs and one of them went down.
Now you want to merge the priority queues!
How can we do that?
Here’s two naïve ways.
1st attempt: If the two trees are the same size: O(n log n)
2nd attempt: O(n) (that’s how long buildHeap takes)
How about O(log n) time?
Heaps
CSE 326 Autumn

26. Solution: Leftist Heaps
Idea
Localize all maintenance work in one small part of the heap
Leftist heap:
almost all nodes are on the left
all the merging work is on the right
Leftist heaps are a solution to that.
They put all the work on the right side of the tree
and most of the nodes on the left (or at least not on the right)
Heaps
CSE 326 Autumn

27. Null Path Length
The null path length (NPL) of a node is the number of nodes between it and a null in the tree
npl(null) = -1
npl(leaf) = 0
npl(single-child node) = 0 2 1 1
Before I make this more concrete, let’s get a totally unrelated random definition that we probably will never use.
Note that I’m assuming a binary tree here.
Null path length is the number of nodes between a given node and a null.
Npl of null is -1
npl of a leaf or a node with just one child is 0
Notice that I’m putting the null path length, NOT the values in the nodes.
another way of looking at it:
NPL is the height of complete subtree rooted at this node 1
Heaps
CSE 326 Autumn

28. Leftist Heap Properties
Heap-order property
parent’s priority value  childrens’ priority values
 minimum element is at the root
Leftist property
 nodes, NPL(left subtree)  NPL(right subtree)
 tree is at least as “heavy” on the left as the right
So, here are the formal properties of a leftist heap.
Heap-order, of course.
And, the leftist property; this means that the npl of the right child is never greater than that of the left child.
Are leftist trees necessarily complete or balanced? NO. In fact, a _great_ leftist tree is a string of nodes all on the left!
Are leftist trees complete? Balanced? Socialist?
Heaps
CSE 326 Autumn

29. Leftist Tree Examples NOT leftist leftist leftist 2 2 1 1 1 1 1 1 11 1 1 1 1 1 1
Here are some training examples to help you learn.
The red nodes break the leftist property for the first tree.
The other two trees are leftist.
every subtree of a leftist
tree is leftist, comrade!
Heaps
CSE 326 Autumn

30. Right Path in a Leftist Tree is Short
Theorem:
If right path-length is at least r, the tree has at least 2r - 1 nodes
Proof by induction:
Basis: r = 1. Tree has at least one node: = 1
Inductive step: Assume true for r’< r.
Right subtree has a right path of at least r - 1 nodes, so it has at least 2(r-1) - 1 nodes.
Left subtree must also have a right path of at least r - 1 (otherwise, there is a null path of r - 3, less than the right subtree).
Again, the left has 2(r-1) - 1 nodes. All told then, there are at least:
2(r-1) (r-1) = 2r - 1
 Leftist tree with at least n nodes has right path of at most log n nodes
Why do we have this leftist property?
Because it guarantees that the right path is really short compared to the number of nodes in the tree without being as stodgy as the heap structure property.
Here’s a proof which boils down to the fact that if the left path is r long, the null path length must be at least r (otherwise somewhere along there, there’s a node with a larger right npl than left).
Since the tree is complete at least to depth r, there are at least 2r - 1 nodes in the tree.
Heaps
CSE 326 Autumn

31. Merging Two Leftist Heaps
Merge(T1,T2) returns one leftist heap containing all elements of the two (distinct) leftist heaps T1 and T2
merge a T1 a
merge
a < b L1 R1 L1 R1
How do we merge two leftist heaps?
We put the smaller root as the new root,
hang its left subtree on the left.
Recursively merge its right subtree and the other tree.
Are we done?
No. Because we haven’t ensured that the leftist property is intact. b T2 b L2 R2 L2 R2
Heaps
CSE 326 Autumn

32. Merge Continued a a npl(R’) > npl(L1) L1 R’ R’ L1 runtime:
So, we check if the leftist property is intact. If it isn’t we switch the right and left children.
How long does this take?
How much work do we do at each step?
CONSTANT
And, we recursively traverse the whole right path of both trees.
How long are those paths?
At most log n.
So how much work?
O(log n)
Pretty cool, huh?
R’ = Merge(R1, T2)
runtime:
Heaps
CSE 326 Autumn

33. Operations on Leftist Heaps
merge two trees of total size n: O(log n)
insert into heap size n: O(log n)
pretend node is a size 1 leftist heap
insert by merging original heap with one node heap
deleteMin with heap size n: O(log n)
remove and return root
merge left and right subtrees
merge
Now that we have that wicked merge op. Let’s just use it to implement all the other ops.
Is one node of size one leftist heap?
merge
Heaps
CSE 326 Autumn

34. Merge Example merge 3 5 merge 7 10 12 5 5 14 merge 3 10 12 10 12 7 8 8? 1 3 5
merge 7 10 12 1 ? 5 5 1 14
merge 3 10 12 10 12 7 8 8 8
Alright, let’s try an example.
I’m unwinding the recursion here.
I pulled a fast one merging the last two nodes. I could have reduced that to merging a node with null (which is clearly the original node) but I didn’t for convenience.
Don’t _you_ do that on tests! 14 8 12

35. Sewing Up the Example ? ? 2 3 3 3 7 ? 7 1 7 5 1 5 5 14 14 14 10 8 10 10 8 8
No, we need to put the example back together, fixing the leftist property as we go.
Are we done here?
NO!
The root’s right child has a higher npl than its left child! 12 12 12
Done?
Heaps
CSE 326 Autumn

36. Finally 2 2 3 3 7 5 1 5 1 7 14 10 8 10 8 14
So, we swap the root’s children.
Is the final tree a leftist heap? 12 12
Heaps
CSE 326 Autumn

37. Iterative Leftist Merge
Downward Pass
Merge right paths
merge 1 5 2 3 10 12 1 7 5 1 3
It turns out that we can do an iterative version of this.
First, we traverse down the right side, picking up the lesser node of each tree as we go. 14 10 8 7 8 12 14
Heaps
CSE 326 Autumn

38. Iterative Leftist Merge (part deux)2 2 2 3 3 3 7 5 1 7 5 1 7 5 1 14 10 8 14 10 8 14 10 8 12 12 12
Then, we go back up and fix the leftist property from bottom to top.
What do we need to do this iteratively?
A STACK of the nodes on the right path! 2
Upward Pass
Fix-up Leftist Heap Property 3 5 1 7 10 8 14
What do we need to do leftist merge iteratively? 12

39. (One More) Amortized Time
To write off an expenditure for (office
equipment, for example) by prorating
over a certain period.
time
A nonspatial continuum in which
events occur in apparently
irreversible succession from the past
through the present to the future.
am·or·tized time
Running time limit resulting from writing off expensive
runs of an algorithm over multiple cheap runs of the
algorithm, usually resulting in a lower overall running time
than indicated by the worst possible case.
If M operations take total O(M log N) time, amortized time per operation is O(log N)
Heaps
CSE 326 Autumn

40. Skew Heaps Problems with leftist heaps Solution: skew heaps
extra storage for NPL
two pass merge (with stack!)
extra complexity/logic to maintain and check NPL
Solution: skew heaps
blind adjusting version of leftist heaps
amortized time for merge, insert, and deleteMin is O(log n)
worst case time for all three is O(n)
merge always switches children when fixing right path
iterative method has only one pass
There are some problems with leftist heaps.
We’ll use skew heaps to fix those problems.
Motivation? Remember that we “often” get heavy right sides when we merge.
So, why not just assume the right side is heavy and move it over automatically?
Heaps
CSE 326 Autumn

41. Merging Two Skew Heaps merge T1 a a merge L1 R1 L1 R1 a < b T2 b b
This merge should look very similar but simpler than the previous algorithm. L2 R2 L2 R2
Heaps
CSE 326 Autumn

42. Skew Heap Example 3 3 merge merge 7 5 7 5 5 merge 14 10 14 12 10 12 108 3 8 3
This happens to end up with a leftist tree and start out with leftist trees. Is that guaranteed? 7 8 5 7 14 8 10 14 12

43. Skew Heap Code void merge(heap1, heap2) { case {
heap1 == NULL: return heap2;
heap2 == NULL: return heap1;
heap1.findMin() < heap2.findMin():
temp = heap1.right;
heap1.right = heap1.left;
heap1.left = merge(heap2, temp);
return heap1;
otherwise:
return merge(heap2, heap1); }
Here’s the code. I won’t go into it except to say that it is _easy_.
The iterative version is a touch more intricate but also fairly easy and requires NO STACK!
Heaps
CSE 326 Autumn

44. Heaps o’ Heaps Binary Heaps d-Heaps Leftist Heaps Skew Heaps
Pros
Cons
Binary Heaps
Simple, fast
BuildHeap
No easy merge
d-Heaps
Large data sets
More complex
Leftist Heaps
Merge
Extra storage
Skew Heaps
Simple code
Good amortized speed
Binomial Queues ?
? (check out Weiss 6.8)
OK, I said you were being cheated unless you got a comparison of data structures. Who can tell me some good things about each of these?
Binary: simple, memory efficient, fast, can run buildHeap. cannot merge quickly.
d: Faster than binary heaps. Possibly much faster in some applications. Slightly more complicated. Otherwise, all binary advantages.
Leftist: supports merge quickly. Supports all other operations asymptotically quickly. Practically, extra storage, link traversals and recursion make it bigger and slower.
Skew: slightly less storage, somewhat faster and simpler than leftist. NOT guaranteed fast on EVERY operation (but guaranteed amortized speed.
Binomial queues? Never heard of ‘em. Read up on them in your book!
Heaps
CSE 326 Autumn