1. CSE 326: Data Structures Priority Queues & Binary Heaps
Ben Lerner
Summer 2007

2. Administrivia Due tonight: Project #1 Released today: Project #2a
Due Wednesday: HW #1
Released Wednesday: HW #2
Office hours today and tomorrow

3. A new problem
Application: Find the smallest (i.e. highest priority) item quickly
Operating system needs to schedule jobs by importance
Hospitals needs to treat patients by severity of problem

4. Priority Queue ADT Security line at the airport ??? Printer queues ???
operations: insert, deleteMin
6 2
15 23
insert
deleteMin

5. Priority Queue ADT PQueue data : collection of data with priority
PQueue operations
insert
deleteMin
(also: create, destroy, is_empty)
PQueue property: for two elements in the queue, x and y, if x has a lower priority value than y, x will be deleted before y
We need a new ADT for this. One that returns the best (which we’ll generally define as lowest priority value) node next.
I’m often going to represent priority queues as collections of priorities; remember that they are collections of data with priorities. MOREOVER, some priority queues don’t even need priorities, just the ability to compare priorities. (all of the ones we’ll talk about fall in this category).

6. Applications of the Priority Q
Select print jobs in order of decreasing length
Forward packets on network routers in order of urgency
Select most frequent symbols for compression
Sort numbers, picking minimum first
Anything greedy
There are, of course, gobs of applications for priority queues. Here are just a few.

7. Implementations of Priority Queue ADT
insert
deleteMin
Unsorted list (Array)
O(1)
O(N)
Unsorted list (Linked-List)
Sorted list (Array)
Sorted list (Linked-List)
Binary Search Tree (BST)
Don’t worry
Binary heap
O(log N)
O(1)/O(N)worst-array full,
should say WHY, might
reject on full instead.
O(N) – to find value
O(1)
O(N) – to find value
O(log N) to find loc w. Bin search, but O(N) to move vals
O(1) to find val, but O(N) to move vals, (or O(1) if in reverse order)
O(N) to find loc, O(1) to do the insert
O(1)
O(N)
O(N)
We are going to do an insert = O(log N) worst case, O(1) on average (on average 1.67 levels)
Deletemin O(log N) – worst and average case.
Plus – good memory usage
O(log N) close to O(1) levels on average
O(log N)
Binary Heap

8. Tree Review Tree T A root(T): leaves(T): children(B): parent(H):
siblings(E):
ancestors(F):
descendents(G):
subtree(C): A
DEFJ..NI B C D E F G H I
Trees have many uses, and we’ll take about them later. To start, let’s just review the basics.
Let’s review the words:
root: A
leaf: DEFJKLMNI
child:A - C or H - K leaves have no children
parent: C - A or L - H the root has no parent
sibling: D - E or F or J - K,L,M, or N
grandparent: G to A
grandchild: C to H or I
ancestor: the node itself or any ancestor’s parent
descendent: the node itself or any child’s descendent
subtree: a node and all its descendents
Its parent or parent’s ancestor
Its child or child’s descendent J K L M N
Itself plus all descendents

9. More Tree Terminology # edges Tree T A depth(T): height(G): degree(B):
branching factor(T):
# edges on path from root to self
# edges on longest path from node to leaf B C
# children of a node D E F G
MAX # children of a node for a BST this is?? H I
Depth: the number of edges along the path from the root to the node
height: the number of edges along the longest path from the node to a leaf
degree: the number of children of the node
branching factor: the maximum degree of any node (or sometimes the average)
preorder traversal: running through all the nodes in the tree, starting with the parent, then all the children
postorder traversal: run through all the nodes starting with the children and then the parents J K L M N

10. Some More Tree Terminology
D(1) = 0, D(2-3) = 1 D(4-7) = 2, D(8-15) = 3
Some More Tree Terminology
Tree T
T is binary if …
T is n-ary if …
T is complete if … A
each node has at most two children B C D E F G
each node has at most n children
Binary: each node has at most two children
n-ary: each child has at most n children
complete: Each row of the tree is filled in left to right before the next one is started
HOW DEEP CAN A COMPLETE TREE BE?
D(N)  D(N/2) + 1 H I J
Each row is filled in left to right before the next one is started
How deep is a complete tree with n nodes?

11. Brief interlude: Some Definitions:
skip
A Perfect binary tree – A binary tree with all leaf nodes at the same depth. All internal nodes have 2 children.
height h
2h+1 – 1 nodes
2h – 1 non-leaves
2h leaves 11 5 21
This is NOT a heap! (but it has the heap structure property) 2 9 16 25 1 3 7 10 13 19 22 30

12. Binary Heap Properties
Structure Property
Ordering Property
Describe these for a binary search tree.
For an AVL tree.

13. Heap Structure Property
A binary heap is a complete binary tree.
Complete binary tree – binary tree that is completely filled, with the possible exception of the bottom level, which is filled left to right.
Examples:
Since they have this regular structure property,
we can take advantage of that to store them in a compact manner.
Since they have this regular structure property, we can take advantage of that to store them in a compact manner.
If I asked to you store a plain old binary tree in an array, how would you do it?

14. Representing Complete Binary Trees in an Array1 2 3 4 5 6 9 8 10 11 12 A
From node i:
left child:
right child:
parent: B C
2 * i 7 D E F G
(2 * i)+1 H I J K L
└ i / 2┘
Left child of node I = 2 8 I
Right child I = (2*1) +1
Parent of node I is at i/2 (floor)
implicit (array) implementation: A B C D E F G H I J K L 1 2 3 4 5 6 7 8 9 10 11 12 13

15. Why better than tree with pointers?
no pointers (space).
*2, /2, + are faster operations than dereferencing a pointer.
(faster operations) but also, better locality.
can get to parent easily
Can we use the implicit representation for binary search trees?
Uses less space (only values, no pointers)
*2 and /2,+ are usually faster operations than dereferencing a pointer
An array MAY get better locality in general than a linked structure.
Can get to the parent easily (and without an extra pointer)

16. Heap Order Property
Heap order property: For every non-root node X, the value in the parent of X is less than (or equal to) the value in X.
This is the order for a MIN heap – could do the same for a max heap. 10 10 20 80 20 80
Tree is PARTIALLY ORDERED.
For each node: its value is less than value of all of its descendants.
This is the definition for a MIN heap – could do the same for a max heap. 40 60 85 99 30 15 50
700
not a heap
This is a PARTIAL order (diff than BST)
For each node, its value is less than all of its descendants (no distinction between left and right)

17. These are all valid binary heaps (minimum)
Heap order property
A heap provides limited ordering information
Each path is sorted, but the subtrees are not sorted relative to each other
A binary heap is NOT a binary search tree -1 2 1 1 4 6 2 6 7 5 8 4 7
These are all valid binary heaps (minimum)

18. FindMin and DeleteMin Insert(val): FindMin: Easy! DeleteMin:
Return root value A[1]
Run time = ?
DeleteMin:
Delete (and return) value at root node
Insert(val):
Insert value into heap 2 4 3 7 5 8 9 11 9 6 10

19. DeleteMin
Delete (and return) value at root node 4 3 7 5 8 9 11 9 6 10

20. Maintain the Structure Property
We now have a “Hole” at the root
Need to fill the hole with another value
When we get done, the tree will have one less node and must still be complete 4 3 7 5 8 9 11 9 6 10 4 3 7 5 8 9 11 9 6 10

21. Maintain the Heap Property
The last value has lost its node
we need to find a new place for it
We can do a simple insertion sort operation to find the correct place for it in the tree 10 4 3 7 5 8 9 11 9 6

22. DeleteMin: Percolate Down10 ? 10 3 ? 3 4 3 4 4 8 7 5 8 9 7 5 8 9 7 5 10 9 11 9 6 11 9 6 11 9 6
Keep comparing with children A[2i] and A[2i + 1]
Copy smaller child up and go down one level
Done if both children are  item or reached a leaf node
What is the run time?

23. DeleteMin: Run Time Analysis
Run time is O(depth of heap)
A heap is a complete binary tree
Depth of a complete binary tree of N nodes?
height = log2(N)  - 1
Run time of DeleteMin is O(log N)

24. Insert Add a value to the tree
Structure and heap order properties must still be correct when we are done 2 3 4 8 7 5 10 9 11 9 6

25. Maintain the Structure Property
The only valid place for a new node in a complete tree is at the end of the array
We need to decide on the correct value for the new node, and adjust the heap accordingly 2 3 4 8 7 5 10 9 11 9 6

26. Maintain the Heap Property
The new value goes where?
We can do a simple insertion sort operation to find the correct place for it in the tree 3 4 8 7 5 10 9 11 9 6 2

27. Insert: Percolate Up 2 3 3 ? 3 4 8 4 8 8 7 5 10 9 7 10 9 7 4 10 9 ? ? 11 9 6 11 9 6 5 11 9 6 5 2 2
Start at last node and keep comparing with parent A[i/2]
If parent larger, copy parent down and go up one level
Done if parent  item or reached top node A[1]
Run time?

28. Insert: Done 2 3 8 7 4 10 9 11 9 6 5
Run time?

29. Insert: 16, 32, 4, 69, 105, 43, 2 1 2 3 4 5 6 7 8
Deletemin - > returns 2
Deletemin -> returns 4

30. More Priority Queue Operations
decreaseKey
given a pointer to an object in the queue, reduce its priority value
Solution: change priority and ____________________________
increaseKey
given a pointer to an object in the queue, increase its priority value
Solution: change priority and _____________________________
percolateUp
Why do I insist on a pointer to each item in decreaseKey, increaseKey, and remove?
Because it’s hard to find an item in a heap; it’s easy to delete the minimum, but how would you find some arbitrary element?
This is where the Dictionary ADT and its various implementations which we will study soon come up.
decreaseKey: percolateUp
increaseKey: percolateDown
percolateDown
Why do we need a pointer? Why not simply data value?
It’s hard to find in a pQ

31. Could imagine for a process
More Heap Operations
decreaseKey(objPtr, amount): raise the priority of a object, percolate up
increaseKey(objPtr, amount): lower the priority of a object, percolate down
remove(objPtr): remove a object, move to top, them delete. 1) decreaseKey(objPtr, )
2) deleteMin()
Worst case Running time for all of these:
FindMax?
ExpandHeap – when heap fills, copy into new space.
O(log N)
Need another structure that points to the processes.
findMax = O(N) – not easy to find, how long in BST?
Expand heap – need to allocate an array at start, if too small, double and copy. O(N)
O(N)

32. Build Heap
BuildHeap {
for i = N/2 to 1 by –1 PercDown(i,A[i]) }
N=11 1 11 11 2 3 5 10 5 10 4 5 6 7 9 4 8 9 9 3 8 9 2 7 6 3 2 7 6 4 8 11 9 10

33. Build Heap 11 11 5 10 5 8 2 3 8 9 2 3 10 9 9 7 6 4 9 7 6 4

34. Build Heap 11 2 2 8 3 8 5 3 10 9 5 4 10 9 9 7 6 4 9 7 6 11

35. Analysis of Build Heap Assume N = 2K –1 Level 1: k -1 steps for 1 item
Level 2: k - 2 steps of 2 items
Level 3: k - 3 steps for 4 items
Level i : k - i steps for 2i-1 items

36. One More Operation Merge two heaps. Ideas?
Anyone have a way to merge two heaps? We’ll find out tomorrow that we can actually do this in O(log n) time!
Can do in Θ(log n) worst case time.
Next lecture!