1. Heaps, Heap Sort and Priority Queues
CMPT 438 Algorithms
Heaps, Heap Sort
and Priority Queues

2. A Job Scheduling Application
The key is the priority of the jobs in the queue
The job with the highest priority needs to be executed next
Operations
Insert, remove maximum
Data structures
Priority queues
Arrays
Ordered
Unordered

3. PQ Implementations & Cost
Worst-case asymptotic costs for a PQ with N items
Insert
Remove max
unordered array ? ?
ordered array ? ?
Can we implement both operations efficiently?

4. Background on Trees
Def: Binary tree = structure composed of a finite set of nodes that :
Is composed of three disjoint sets of nodes: a root node, a left subtree and a right subtree 4
root
Right subtree
Left subtree 1 3 2 16 9 10 14 8

5. Special Types of Trees
Full binary tree 2 14 8 1 16 7 4 3 9 10 12
Def: Full binary tree = a binary tree in which each node is either a leaf or has degree (number of children) exactly 2.
Def: Complete binary tree = a binary tree in which all leaves have the same depth and all internal nodes have degree 2.
Complete binary tree 2 1 16 4 3 9 10

6. The Heap Data Structure
Def: A heap is a nearly complete binary tree with the following two properties:
Structural property: all levels are full, except possibly the last one, which is filled from left to right
Order (heap) property: for any node x
Parent(x) ≥ x
Largest element in the heap? 8 7 4
It doesn’t matter that 4 in level 1 is smaller than 5 in level 2
Largest element in a heap 5 2
Heap

7. Definitions
Height of a node = the number of edges on a longest simple path from the node down to a leaf
Depth of a node = the length of a path from the root to the node
Height of tree = lgn, for a heap of n elements
Height of root = 3 4 1 3
Height of (2)= 1 2 16 9 10
Depth of (10)= 2
Height of a heap 14 8

8. Array Representation of Heaps
A heap can be stored as an array A.
Root of tree
Parent(i) {}
Left(i) {}
right(i) {}

9. Array Representation of Heaps
A heap can be stored as an array A.
Root of tree is A[1]
Parent(i) { return i/2; }
Left(i) { return 2*i; }
right(i) { return 2*i + 1; }
The elements in the subarray A[(n/2+1) .. n] are leaves
The root is the maximum element of the heap

10. Heap Types
Max-heaps (largest element at root), have the max-heap property:
for all nodes i, excluding the root:
A[PARENT(i)] ≥ A[i]
Min-heaps (smallest element at root), have the min-heap property:
A[PARENT(i)] ≤ A[i]

11. Operations on Heaps Maintain the max-heap property
MAX-HEAPIFY
Create a max-heap from an unordered array
BUILD-MAX-HEAP
Sort an array in place
HEAPSORT
Priority queue operations

12. Max-Heapify()

13. Max-Heapify()

15. Max-Heapify()

16. Max-Heapify()

17. Max-Heapify()
Running time?

18. Max-Heapify()

19. Building a Heap 8 2 4 14 7 1 16 10 9 3 5 6 2 14 8 1 16 7 4 3 9 10 4 1 3 2 16 9 10 14 8 7

20. Example: A 4 1 3 2 16 9 10 14 8 7
i = 5
i = 4
i = 3 2 14 8 1 16 7 4 3 9 10 5 6 2 14 8 1 16 7 4 3 9 10 5 6 14 2 8 1 16 7 4 3 9 10 5 6
i = 2
i = 1 14 2 8 1 16 7 4 10 9 3 5 6 14 2 8 16 7 1 4 10 9 3 5 6 8 2 4 14 7 1 16 10 9 3 5 6

21. Building a Heap
Convert an array A[1 … n] into a max-heap (n = length[A])
The elements in the subarray A[(n/2+1) .. n] are leaves
Apply MAX-HEAPIFY on elements between 1 and n/2 2 14 8 1 16 7 4 3 9 10 5 6 A: 4 1 3 2 16 9 10 14 8 7

22. Building a Heap
Convert an array A[1 … n] into a max-heap (n = length[A])
The elements in the subarray A[(n/2+1) .. n] are leaves
Apply MAX-HEAPIFY on elements between 1 and n/2
Alg: BUILD-MAX-HEAP(A)
n = length[A]
for i ← n/2 downto 1
do MAX-HEAPIFY(A, i) 2 14 8 1 16 7 4 3 9 10 5 6 A: 4 1 3 2 16 9 10 14 8 7

23. Running Time of BUILD MAX HEAP
HEAPIFY takes O(h)  the cost of HEAPIFY on a node i is proportional to the height of the node i in the tree
Height
Level
No. of nodes
h0 = 3 (lgn)
i = 0 20
h1 = 2
i = 1 21
h2 = 1
i = 2 22
h3 = 0
i = 3 (lgn) 23
hi = h – i height of the heap rooted at level i
ni = 2i number of nodes at level i

24. Running Time of BUILD MAX HEAP
Running time of BUILD-MAX-HEAP: T(n) = O(n)
Cost of HEAPIFY at level i  number of nodes at that level
Replace the values of ni and hi computed before
Multiply by 2h both at the nominator and denominator and
write 2i as
Change variables: k = h - i
The sum above is smaller than the sum of all elements to 
and h = lgn
The sum above is smaller than 2

25. Heapsort
Given BuildHeap(), an in-place sorting algorithm can be easily constructed:

26. Heapsort
Given BuildHeap(), an in-place sorting algorithm can be easily constructed:

27. Heapsort
Given BuildHeap(), an in-place sorting algorithm can be easily constructed:

28. An Example for HEAPSORT

29. An Example for HEAPSORT