1. HEAPS

2. Priority Queues Insert head head Linked-list Æ Æ 9 2 5 8 10 15 2 5 8

3. Priority Queues Supports the following operations. Insert element x.
Return min/max element.
Return and delete minimum/maximum element.
Increase/Decrease key of element x to k.
Max/Min-HEAPIFY
BUILD-Max/Min-HEAP
HEAPSORT
Applications.
Dijkstra's shortest path algorithm.
Prim's MST algorithm.
Event-driven simulation.
Huffman encoding.
Heapsort.
. . .
Event driven simulation: generate random inter-arrival times between customers, generate random processing times per customer, Advance clock to next event, skipping intervening ticks

4. Time Complexity
Insert:
Delete:
n deletions and insertions:
O(n)
O(1)
O(n2)

5. Heap
A max (min) heap is a complete binary tree such that the data stored in each node is greater (smaller) than the data stored in its children, if any.

6. Heap Types
Max-heaps (largest element at root), have the max-heap property:
for all nodes i, excluding the root:
A[PARENT(i)] ≥ A[i]
Min-heaps (smallest element at root), have the min-heap property:
A[PARENT(i)] ≤ A[i]

7. Binary Heap: Definition
Almost complete binary tree.
filled on all levels, except last, where filled from left to right
Min-heap ordered.
every child greater than (or equal to) parent 06 14 45 78 18 47 53 83 91 81 77 84 99 64

8. Binary Heap: Properties
Min element is in root.
Heap with N elements has height = log2 N. 06
N = 14 Height = 3 14 45 78 18 47 53 83 91 81 77 84 99 64

9. 20 8 15 4 10 7 2 8 7 6 2

10. Larger than its parent 20 8 15 4 10 7 9
Not a heap

11. Not a complete tree – Not a heap
Missing left child 25 12 10 5 9 7 11 1 6 3 8
Not a complete tree – Not a heap

12. Where to add the next node?
Complete Binary Tree 1 3 2 5 4 7 6 8 9 10 11 12
Where to add the next node?
How to find it?

13. After the first 1 from left: 0 – left; 1 – right3 2 5 4 7 6 8 9 10 11 12 13 14 15
After the first 1 from left: 0 – left; 1 – right

14. 1 3 2 5 4 7 6 8 9 10 11 12 1 3 1 3 1 1 6 3 7 3 13 6 14 7
Next
Next

15. Binary Heaps: Array Implementation
Implementing binary heaps.
Use an array: no need for explicit parent or child pointers.
Parent(i) = i/2
Left(i) = 2i
Right(i) = 2i + 1 06
parent and children formula require only simple bit twiddling operations 1 14 45 2 3 78 18 47 53 4 5 6 7 83 91 81 77 84 99 64 8 9 10 11 12 13 14

16. Storing a Complete Binary Tree in an Array1 3 2 5 4 7 6 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
for any node at index i
parent = i/ left child = i * 2 right child = i*2+1

17. Binary Heap: Insertion
Insert element x into heap.
Insert into next available slot.
Bubble up until it's heap ordered.
Peter principle: nodes rise to level of incompetence 06 14 45 78 18 47 53 42
next free slot 83 91 81 77 84 99 64

18. Binary Heap: Insertion
Insert element x into heap.
Insert into next available slot.
Bubble up until it's heap ordered.
Peter principle: nodes rise to level of incompetence
swap with parent 06 14 45 78 18 47 53 42 42 83 91 81 77 84 99 64

19. Binary Heap: Insertion
Insert element x into heap.
Insert into next available slot.
Bubble up until it's heap ordered.
Peter principle: nodes rise to level of incompetence
swap with parent 06 14 45 78 18 47 42 42 83 91 81 77 84 99 64 53

20. Binary Heap: Insertion
Insert element x into heap.
Insert into next available slot.
Bubble up until it's heap ordered.
Peter principle: nodes rise to level of incompetence
stop: heap ordered 06 14 42 78 18 47 45 83 91 81 77 84 99 64 53
O(log N) operations.

21. Binary Heap: Decrease Key
Decrease key of element x to k.
Bubble up until it's heap ordered. 06 14 42 78 18 47 45 83 91 81 77 84 99 64 53
O(log N) operations.

22. Binary Heap: Delete Min
Delete minimum element from heap.
Exchange root with rightmost leaf.
Bubble root down until it's heap ordered.
power struggle principle: better subordinate is promoted 06 14 42 78 18 47 45 83 91 81 77 84 99 64 53

23. Binary Heap: Delete Min
Delete minimum element from heap.
Exchange root with rightmost leaf.
Bubble root down until it's heap ordered.
power struggle principle: better subordinate is promoted 53 14 42 78 18 47 45 83 91 81 77 84 99 64 06

24. Binary Heap: Delete Min
Delete minimum element from heap.
Exchange root with rightmost leaf.
Bubble root down until it's heap ordered.
power struggle principle: better subordinate is promoted
exchange with left child 53 14 42 78 18 47 45 83 91 81 77 84 99 64

25. Binary Heap: Delete Min
Delete minimum element from heap.
Exchange root with rightmost leaf.
Bubble root down until it's heap ordered.
power struggle principle: better subordinate is promoted
exchange with right child 14 53 42 78 18 47 45 83 91 81 77 84 99 64

26. Binary Heap: Delete Min
Delete minimum element from heap.
Exchange root with rightmost leaf.
Bubble root down until it's heap ordered.
power struggle principle: better subordinate is promoted
stop: heap ordered 14 18 42 78 53 47 45 83 91 81 77 84 99 64
O(log N) operations.

27. Binary Heap: Delete /Extract Max25 24 20 18 15 7 16 8 9 10 11 12 22 3 3 25 3 24 22 3 3 16 25
delete
Last Node#

28. Binary Heap: Delete /Extract Max

29. Max-Heapify Example ? 17 98 86 ? 17 86 65 ? 41 ? left child is greater
right child is greater 13 65 17 44 32 29 9 10 17 44 23 21 17
children of 2: 2*2, 2*2+1 = 4, 5
children of root: 2*1, 2*1+1 = 2, 3
index 1 2 3 4 5 6 7 8 10 11 12
value 98 86 41 13 65 32 29 9 44 23 21 17 86 65 17 44 17 17

30. Procedure MaxHeapify MaxHeapify(A, i) 1. l = left(i); 2. r = right(i);
3. if (l  heap-size[A] && A[l] > A[i])
largest = l;
else largest = I;
6. if (r  heap-size[A] && A[r] > A[largest])
largest = r;
8. if (largest != i)
Swap(A[i], A[largest])
MaxHeapify(A, largest)
Assumption:
Left(i) and Right(i) are max-heaps.

31. MaxHeapify – Example MaxHeapify(A, 2) 26 14 24 24 14 20 18 18 14 24 1417 19 13 12 14 18 14 11

32. Running Time for MaxHeapify
MaxHeapify(A, i)
1. l = left(i);
2. r = right(i);
3. if (l  heap-size[A] && A[l] > A[i])
largest = l;
else largest = I;
6. if (r  heap-size[A] && A[r] > A[largest])
largest = r;
8. if (largest != i)
Swap(A[i], A[largest])
MaxHeapify(A, largest)
Time to fix node i and its children = (1)
PLUS
Time to fix the subtree rooted at one of i’s children = T(size of subree at largest)

33. Running Time for MaxHeapify(A, n)
T(n) = T(largest) + (1)
largest  2n/3 (worst case occurs when the last row of tree is exactly half full)
T(n)  T(2n/3) + (1)  T(n) = O(lg n)
Alternately, MaxHeapify takes O(h) where h is the height of the node where MaxHeapify is applied.

34. Building a heap BuildMaxHeap(A) 1. heap-size[A] = length[A]
Use MaxHeapify to convert an array A into a max-heap.
How?
Call MaxHeapify on each element in a bottom-up manner.
BuildMaxHeap(A)
1. heap-size[A] = length[A]
2. for i = length[A]/2 down to 1
do MaxHeapify(A, i)

35. BuildMaxHeap – Example
Input Array: 24 21 23 22 36 29 30 34 28 27
Initial Heap:
(not max-heap) 24 21 23 22 36 29 30 34 28 27

36. BuildMaxHeap – Example
MaxHeapify(10/2 = 5)
MaxHeapify(4) 36 24 24
MaxHeapify(3)
MaxHeapify(2) 34 24 36 21 21 23 30 23
MaxHeapify(1) 28 34 22 24 22 21 21 36 36 36 27 29 23 30 34 22 28 24 27 21

37. Running Time of BuildMaxHeap
BuildMaxHeap(A)
1. heap-size[A] = length[A]
2. for i = length[A]/2 down to 1
do MaxHeapify(A, i)
(logn)
(n logn)
Cost of a MaxHeapify call  No. of calls to MaxHeapify
O(lg n)  O(n) = O(nlg n)

38. Heapsort Goal: Sort an array using heap representations Idea:
Build a max-heap from the array
Swap the root (the maximum element) with the last element in the array
“Discard” this last node by decreasing the heap size
Call MAX-HEAPIFY on the new root
Repeat this process until only one node remains

39. Example: A=[7, 4, 3, 1, 2] MAX-HEAPIFY(A, 1, 2) MAX-HEAPIFY(A, 1, 3)

40. Heapsort(A) HeapSort(A) 1. Build-Max-Heap(A)
2. for i = length[A] downto 2
swap( A[1], A[i] );
heap-size[A] = heap-size[A] – 1;
MaxHeapify(A, 1);

41. Algorithm Analysis HeapSort(A) 1. Build-Max-Heap(A) (n)
In-place
Not Stable
Build-Max-Heap takes O(n) and each of the n-1 calls to Max-Heapify takes time O(lg n).
Therefore, T(n) = O(n lg n)
HeapSort(A)
1. Build-Max-Heap(A)
2. for i = length[A] downto 2
swap( A[1], A[i] );
heap-size[A] = heap-size[A] – 1;
MaxHeapify(A, 1);
(n)
(n -1)
(logn)
(n logn)

42. Binary Heap: Delete /Extract Max25 24 20 18 15 7 16 8 9 10 11 12 22 3 3 25 3 24 22 3 3 16 25
delete
Last Node#

43. Binary Heap: Delete /Extract Max

44. Analysis Binary Heap: Delete /Extract Max
(1)
(logn)
( logn)

45. Binary Heap: Return/Find Max

46. Analysis Binary Heap: Return/Find Max
(1)
(1)

47. Binary Heap: Increase Key
The node i has its key increased to 15.

48. Binary Heap: Increase Key

49. Analysis Binary Heap: Increase Key
The running time of HEAP-INCREASE-KEY on an n-element heap is O(lg n), since the path traced from the node to the root has length O(lg n).

50. Binary Heap: Insertion
Insert element x into heap.
Insert into next available slot.
Bubble up until it's heap ordered.
Peter principle: nodes rise to level of incompetence 98 84 75 78 68 67 53 89
next free slot 33 21 51 47 34 49 40

51. Binary Heap: Insertion
Insert element x into heap.
Insert into next available slot.
Bubble up until it's heap ordered.
Peter principle: nodes rise to level of incompetence
swap with parent 98 84 75 78 68 67 53 89 42 33 21 51 47 34 49 40

52. Binary Heap: Insertion
Insert element x into heap.
Insert into next available slot.
Bubble up until it's heap ordered.
Peter principle: nodes rise to level of incompetence
swap with parent 98 84 75 78 68 67 89 53 42 33 21 51 47 34 49 40

53. Binary Heap: Insertion
Insert element x into heap.
Insert into next available slot.
Bubble up until it's heap ordered.
Peter principle: nodes rise to level of incompetence
stop: heap ordered 98 84 89 78 68 67 75 42 53 33 21 51 47 34 49 40
O(log N) operations.

54. Binary Heap: Insertion

55. Analysis Binary Heap: Insertion
The running time of Insert on an n-element heap is O(lg n), since the path traced from the node to the root has length O(lg n).

56. Priority Queues Operation Build -heap N insert log N find-min 1
delete-min
log N
union N
decrease-key
log N
delete
log N
is-empty 1