1. CSC212 Data Structure - Section AB
Lecture 17
Heaps and Priority Queues
Instructor: Zhigang Zhu
Department of Computer Science
City College of New York
This lecture was modified from the 1997 presentation with the textbook, with new conventions provided in the second edition (2001) of the textbook - Z. Zhu

2. Heaps
Chapter 11 has several programming projects, including a project that uses heaps.
This presentation shows you what a heap is, and demonstrates two of the important heap algorithms.
This lecture introduces heaps, which are used in the Priority Queue project of Chapter 11. The lecture includes the algorithms for adding to a heap (including reheapification upward), removing the top of a heap (including reheapification downward), and implementing a heap in a partially-filled array.
Prior to this lecture, the students need a good understanding of complete binary trees. It would also help if they have seen binary search trees and the priority queue class.
Data Structures
and Other Objects
Using C++

3. Topics Heap Definition Heap Applications
priority queues (chapter 8), sorting (chapter 13)
Two Heap Operations – add, remove
reheapification upward and downward
why is a heap good for implementing a priority queue?
Heap Implementation
using binary_tree_node class
using fixed size or dynamic arrays

4. Heaps Definition A heap is a certain kind of complete binary tree.
A heap is a data structure with several applications, including a way to implement Priority Queues, as shown in Chapter 11. The definition of a heap is a special kind of complete binary tree.
You probably recall that a complete binary tree requires that its nodes are added in a particular order...

5. Heaps A heap is a certain kind of complete binary tree. Root
The first node of a complete binary tree is always the root...
When a complete
binary tree is built,
its first node must be
the root.

6. Heaps Complete binary tree. Left child of the root The second node is
...the second node is always the left child of the root...
The second node is
always the left child
of the root.

7. Heaps Complete binary tree. Right child of the root The third node is
...then the right child of the root...
The third node is
always the right child
of the root.

8. level from left-to-right.
Heaps
Complete binary tree.
...and so on. The nodes always fill each level from left-to-right...
The next nodes
always fill the next
level from left-to-right.

9. level from left-to-right.
Heaps
Complete binary tree.
...from left-to-right...
The next nodes
always fill the next
level from left-to-right.

10. level from left-to-right.
Heaps
Complete binary tree.
...from left-to-right...
The next nodes
always fill the next
level from left-to-right.

11. level from left-to-right.
Heaps
Complete binary tree.
...from left-to-right...
The next nodes
always fill the next
level from left-to-right.

12. Heaps Complete binary tree.
...and when a level is filled you start the next level at the left.

13. Heaps A heap is a certain kind of complete binary tree. 45 35 23 27 2122 4 19
So, a heap is a complete binary tree. Each node in a heap contains a key, and these keys must be organized in a particular manner. Notice that this is not a binary search tree, but the keys do follow some semblance of order.
Can you see what rule is being enforced here?
Each node in a heap
contains a key that
can be compared to
other nodes' keys.

14. Heaps A heap is a certain kind of complete binary tree. 45 35 23 27 2122 4 19
The heap property requires that each node's key is >= to the keys of its children.
This is a handy property because the biggest node is always at the top. Because of this, a heap can easily implement a priority queue (where we need quick access to the highest priority item).
The "heap property"
requires that each
node's key is >= the
keys of its children

15. What it is not: It is not a BST
In a binary search tree, the entries of the nodes can be compared with a strict weak ordering. Two rules are followed for every node n:
The entry in node n is NEVER less than an entry in its left subtree
The entry in the node n is less than every entry in its right subtree.
BST is not necessarily a complete tree

16. What it is: Heap Definition
A heap is a binary tree where the entries of the nodes can be compared with the less than operator of a strict weak ordering. In addition, two rules are followed:
The entry contained by the node is NEVER less than the entries of the node’s children
The tree is a COMPLETE tree.
Q: where is the largest entry?  for what....
Complete binary tree
Every level except the deepest level must contains as many nodes as possible, and at the deepest level, all the nodes are as far left as possible

17. Application : Priority Queues
A priority queue is a container class that allows entries to be retrieved according to some specific priority levels
The highest priority entry is removed first
If there are several entries with equally high priorities, then the priority queue’s implementation determines which will come out first (e.g. FIFO)
Heap is suitable for a priority queue

18. The Priority Queue ADT with Heaps
The entry with the highest priority is always at the root node
Focus on two priority queue operations
adding a new entry
remove the entry with the highest priority
In both cases, we must ensure the tree structure remains to be a heap
we are going to work on a conceptual heap without worrying about the precise implementation
later I am going to show you how to implement...

19. Adding a Node to a Heap 45
Put the new node in the next available spot.
Push the new node upward, swapping with its parent until the new node reaches an acceptable location. 35 23 27 21 22 4 19 42
We can add new elements to a heap whenever we like. Because the heap is a complete binary search tree, we must add the new element at the next available location, filling in the levels from left-to-right.
In this example, I have just added the new element with a key of 42.
Of course, we now have a problem: The heap property is no longer valid. The 42 is bigger than its parent 27.
To fix the problem, we will push the new node upwards until it reaches an acceptable location.

20. Adding a Node to a Heap 45
Put the new node in the next available spot.
Push the new node upward, swapping with its parent until the new node reaches an acceptable location. 35 23 42 21 22 4 19 27
Here we have pushed the 42 upward one level, swapping it with its smaller parent 27.
We can't stop here though, because the parent 35 is still smaller than the new node 42.

21. Adding a Node to a Heap 45
Put the new node in the next available spot.
Push the new node upward, swapping with its parent until the new node reaches an acceptable location. 42 23 35 21 22 4 19 27
Can we stop now? Yes, because the 42 is less than or equal to its parent.

22. Adding a Node to a Heap 45
The parent has a key that is >= new node, or
The node reaches the root.
The process of pushing the new node upward is called reheapification upward. 42 23 35 21 22 4 19 27
In general, there are two conditions that can stop the pushing upward:
1. We reach a spot where the parent is >= the new node, or
2. We reach the root.
This process is called reheapification upward (I didn't just make up that name, really).
Note: Note: we need to easily go from child to parent as well as parent to child.

23. Removing the Top of a Heap45
Move the last node onto the root. 42 23 35 21 22 4 19 27
We can also remove the top node from a heap. The first step of the removal is to move the last node of the tree onto the root. In this example we move the 27 onto the root.

24. Removing the Top of a Heap27
Move the last node onto the root. 42 23 35 21 22 4 19
Now the 27 is on top of the heap, and the original root (45) is no longer around. But the heap property is once again violated.

25. Removing the Top of a Heap27
Move the last node onto the root.
Push the out-of-place node downward, swapping with its larger child until the new node reaches an acceptable location. 42 23 35 21 22 4 19
We'll fix the problem by pushing the out-of-place node downward. Perhaps you can guess what the downward pushing is called....reheapification downward.

26. Removing the Top of a Heap42
Move the last node onto the root.
Push the out-of-place node downward, swapping with its larger child until the new node reaches an acceptable location. 27 23 35 21 22 4 19
When we push a node downward it is important to swap it with its largest child. (Otherwise we are creating extra problems by placing the smaller child on top of the larger child.) This is what the tree looks like after one swap.
Should I continue with the reheapification downward?

27. Removing the Top of a Heap42
Move the last node onto the root.
Push the out-of-place node downward, swapping with its larger child until the new node reaches an acceptable location. 35 23 27 21 22 4 19
Yes, I swap again, and now the 27 is in an acceptable location.

28. Removing the Top of a Heap42
The children all have keys <= the out-of-place node, or
The node reaches the leaf.
The process of pushing the new node downward is called reheapification downward. 35 23 27 21 22 4 19
Reheapification downward can stop under two circumstances:
1. The children all have keys that are <= the out-of-place node.
2. The out-of-place node reaches a leaf.

29. Priority Queues Revisited
A priority queue is a container class that allows entries to be retrieved according to some specific priority levels
The highest priority entry is removed first
If there are several entries with equally high priorities, then the priority queue’s implementation determines which will come out first (e.g. FIFO)
Heap is suitable for a priority queue

30. Adding a Node: same priority45
Put the new node in the next available spot.
Push the new node upward, swapping with its parent until the new node reaches an acceptable location. 35 23 27 21 22 4 19
45*
We can add new elements to a heap whenever we like. Because the heap is a complete binary search tree, we must add the new element at the next available location, filling in the levels from left-to-right.
In this example, I have just added the new element with a key of 42.
Of course, we now have a problem: The heap property is no longer valid. The 42 is bigger than its parent 27.
To fix the problem, we will push the new node upwards until it reaches an acceptable location.

31. Adding a Node : same priority45
Put the new node in the next available spot.
Push the new node upward, swapping with its parent until the new node reaches an acceptable location. 35 23
45* 21 22 4 19 27
Here we have pushed the 42 upward one level, swapping it with its smaller parent 27.
We can't stop here though, because the parent 35 is still smaller than the new node 42.

32. Adding a Node : same priority45
Put the new node in the next available spot.
Push the new node upward, swapping with its parent until the new node reaches an acceptable location.
45* 23 35 21 22 4 19 27
Can we stop now? Yes, because the 42 is less than or equal to its parent.

33. Adding a Node : same priority45
The parent has a key that is >= new node, or
The node reaches the root.
The process of pushing the new node upward is called reheapification upward.
45* 23 35 21 22 4 19 27
In general, there are two conditions that can stop the pushing upward:
1. We reach a spot where the parent is >= the new node, or
2. We reach the root.
This process is called reheapification upward (I didn't just make up that name, really).
Note: Implementation determines which 45 will be in the root, and will come out first when popping.

34. Removing the Top of a Heap
45*
The children all have keys <= the out-of-place node, or
The node reaches the leaf.
The process of pushing the new node downward is called reheapification downward. 35 23 27 21 22 4 19
In general, there are two conditions that can stop the pushing upward:
1. We reach a spot where the parent is >= the new node, or
2. We reach the root.
This process is called reheapification upward (I didn't just make up that name, really).
Note: Implementation determines which 45 will be in the root, and will come out first when popping.

35. Heap Implementation Use binary_tree_node class Use arrays (page 459)
node implementation is for a general binary tree
but we may need to have doubly linked node
Use arrays (page 459)
A heap is a complete binary tree
which can be implemented more easily with an array than with the node class
and do two-way links

36. Formulas for location children and parents in an array representation
Root at location [0]
Parent of the node in [i] is at [(i-1)/2]
Children of the node in [i] (if exist) is at [2i+1] and [2i+2]
Test:
complete tree of 10, 000 nodes
parent of 4999 is at (4999-1)/2 = 2499
children of 4999 is at 9999 (V) and 10,000 (X)

37. Implementing a Heap 42
We will store the data from the nodes in a partially-filled array. 35 23 27 21
This slide shows the typical way that a heap is implemented. For the most part, there is nothing new here, because you already know how to implement a complete binary tree using a partially-filled array. That is what we are doing with the heap.
An array of data

38. Implementing a Heap 42
Data from the root goes in the first location of the array. 35 23 27 21
Following the usual technique for implementing a complete binary tree, the data from the root is stored in the first entry of the array. 42
An array of data

39. Implementing a Heap 42
Data from the next row goes in the next two array locations. 35 23 27 21
The next two nodes go in the next two locations of the array. 42 35 23
An array of data

40. Implementing a Heap 42
Data from the next row goes in the next two array locations. 35 23 27 21
and so on. 42 35 23 27 21
An array of data

41. Implementing a Heap 42
Data from the next row goes in the next two array locations. 35 23 27 21
As with any partially-filled array, we are only concerned with the front part of the array. If the tree has five nodes, then we are only concerned with the entries in the first five components of the array. 42 35 23 27 21
An array of data
We don't care what's in
this part of the array.

42. Important Points about the Implementation42
The links between the tree's nodes are not actually stored as pointers, or in any other way.
The only way we "know" that "the array is a tree" is from the way we manipulate the data. 35 23 27 21
With this implementation of a heap, there are no pointers. The only way that we know that the array is a heap is the manner in which we manipulate it. 42 35 23 27 21
An array of data

43. Important Points about the Implementation42
If you know the index of a node, then it is easy to figure out the indexes of that node's parent and children. Formulas are given in the book. 35 23 27 21
The manipulations are the same manipulations that you've used for a complete binary tree, making it easy to compute the index where various nodes are stored. 42 35 23 27 21
[0] [1] [2] [3] [4]

44. Formulas for location children and parents in an array representation
Root at location [0]
Parent of the node in [i] is at [(i-1)/2]
Children of the node in [i] (if exist) is at [2i+1] and [2i+2]
Test:
complete tree of 10, 000 nodes
parent of 4999 is at (4999-1)/2 = 2499
children of 4999 is at 9999 (V) and 10,000 (X)

45. class heap {
public:
....
void push(const Item& entry); // add
Item& pop(); // remove the highest
private:
Item data[CAPACITY];
size_type used; }
Wrap Up...
Can you implement the add and remove with the knowledge of these formulas?
Add (in-class quiz)
put the new entry in the last location
Push the new node upward, swapping with its parent until the new node reaches an acceptable location
Remove (in-class quiz)
move the last node to the root
Push the out-of-place node downward, swapping with its larger child until the new node reaches an acceptable location
template <class Item>
class heap {
public:
void push(const Item& entry); // add
Item& pop(); // remove the highest priority entry
private:
Item data[CAPACITY];
size_type used; }
void heap<Item>::push(const Item& entry)
size_type i;
data[used++] = entry;
i = used –1;
while ( ( i != 0 && (data[i] > data[( i-1)/2]) {
swap(data[i], data[( i-1)/2]);
i = ( i-1)/2;
Item& heap<Item>::pop()
Item entry;

46. template <class Item>
class heap {
public:
heap () { used = 0;}
void push(const Item& entry); // add
Item& pop(); // remove the highest
size_t parent (size_t k) const { return (k-1)/2;}
size_t l_child (size_t k) const { return 2*k+1;}
size_t r_child (size_t k) const { return 2*k+2;}
private:
Item data[CAPACITY];
size_type used; }
Wrap Up...
Can you implement the add and remove with the knowledge of these formulas?
Add (in-class quiz)
put the new entry in the last location
Push the new node upward, swapping with its parent until the new node reaches an acceptable location
Remove (in-class quiz)
move the last node to the root
Push the out-of-place node downward, swapping with its larger child until the new node reaches an acceptable location
template <class Item>
class heap {
public:
void push(const Item& entry); // add
Item& pop(); // remove the highest priority entry
private:
Item data[CAPACITY];
size_type used; }
void heap<Item>::push(const Item& entry)
size_type i;
data[used++] = entry;
i = used –1;
while ( ( i != 0 && (data[i] > data[( i-1)/2]) {
swap(data[i], data[( i-1)/2]);
i = ( i-1)/2;
Item& heap<Item>::pop()
Item entry;

47. Assignment 7
Due Dec 11 (Friday)
Implement a Priority Queue using Heap

48. Summary
A heap is a complete binary tree, where the entry at each node is greater than or equal to the entries in its children.
To add an entry to a heap, place the new entry at the next available spot, and perform a reheapification upward.
To remove the biggest entry, move the last node onto the root, and perform a reheapification downward.
A quick summary . . .

49. THE END Presentation copyright 1997 Addison Wesley Longman,
For use with Data Structures and Other Objects Using C++
by Michael Main and Walter Savitch.
Some artwork in the presentation is used with permission from Presentation Task Force
(copyright New Vision Technologies Inc) and Corel Gallery Clipart Catalog (copyright
Corel Corporation, 3G Graphics Inc, Archive Arts, Cartesia Software, Image Club
Graphics Inc, One Mile Up Inc, TechPool Studios, Totem Graphics Inc).
Students and instructors who use Data Structures and Other Objects Using C++ are welcome
to use this presentation however they see fit, so long as this copyright notice remains
intact.
Feel free to send your ideas to:
Michael Main
THE END