Computer Science and Software Engineering University of Wisconsin - Platteville 12. Heap Yan Shi CS/SE 2630 Lecture Notes Partially adopted from C++ Plus Data Structure textbook slides

What is a Heap?  Shape: complete binary tree  Order: for each node in the heap, the value stored in that node is greater than or equal to the value in each of its children. —Root always has the largest element  By default, when we say “heap”, we mean “maximum heap” —minimum heap: for each node in the heap, the value stored in that node is less than or equal to the value in each of its children

Are these Heaps? C A T treePtr

Are these Heaps?

Store a Heap in Array tree [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes For any tree node nodes[index] left child: nodes[index*2 + 1] right child: nodes[index*2 + 2] parent: nodes[(index – 1)/2]

Construct a Heap  How to guarantee it is a complete binary tree? —treat the data as an array/sequence, add nodes to the tree in a top-down, left-to-right manner  How to guarantee the order? —bubble the values as high as it can go

Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes

Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] nodes 10 Add nodes[0] to the tree: it has no parent, so we have a heap.

Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes Add nodes[1] to the tree: 20 > 10, so we need to bubble 20 up. Exchange 20 and 10 in both the tree and the array.

Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes Add nodes[1] to the tree: now we have a heap.

Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes Add nodes[2] to the tree: 12 < 20. It cannot go higher. We have a heap. 12

Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes Add nodes[3] to the tree: 40 < 10. We need to bubble 40 up. Exchange 40 and 10 in both tree and array

Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes Add nodes[3] to the tree: 40 < 20. We need to bubble 40 up. Exchange 40 and 20 in both tree and array

Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes Add nodes[3] to the tree: 40 cannot go higher. We have a heap

Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes Add nodes[4] to the tree: 20 = 20. It cannot go higher. We have a heap

Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes Add nodes[5] to the tree: 15 < 12. We need to bubble 15 up. Exchange 15 and 12 in both tree and array

Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes Add nodes[5] to the tree: 15 < 12. We need to bubble 15 up. Exchange 15 and 12 in both tree and array

Example: Done! [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes Add nodes[5] to the tree: 15 cannot go higher. We have a heap. All nodes are added. Done!

Construct a Heap: Bubble Up Algorithm  Given nodes[] and size  For each node at index i: —Leaf = i; Done = false —while Leaf is not 0 AND Done == false  Parent = (Leaf-1)/2  if nodes[Leaf] > nodes[Parent] Swap nodes[Leaf] and nodes[Parent] Leaf = Parent;  otherwise Done = true  What is the time complexity? O(nlog 2 n)

ReheapUp Algorithm  Based on the previous algorithm.  What if we want to add a new item to an existing heap? —add the item as the last element in the array  it is the only item that violates the order —restore the order in a bottom-up manner —we can do it recursively! void ReheapUp( int root, int bottom ) { int parent ; if ( bottom > root ){ parent = ( bottom - 1 ) / 2; if ( nodes [ parent ] < nodes [ bottom ] ){ Swap ( nodes [ parent ], nodes [ bottom ] ); ReheapUp ( root, parent ); }

Sorting Using Heap  Heap is perfect for sorting: the root is always the largest!  Basic algorithm: —use heap in which top node contains largest value —repeat until heap is empty:  take root (largest value) off heap, put it in its place  reconstruct the heap  How to reconstruct the heap?

Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes Take root off heap

Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes 20 Put the last one in the array (12) as the new root: Only the root violates the order. Its left and right subtrees are still heaps! We also have space to store the old root What should we put here?

Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes < 20 and 12 < 15 Swap 12 with its child: which one? The larger one so that we guarantee the new parent >= both children

Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes < 20: Swap 12 with

Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes < 20: Swap 12 with 20 It is already a leaf node. We stop

ReheapDown Algorithm  To facilitate heapSort.  Root is the only node that violates the order: —reorder the heap by pushing down the root as low as possible  This procedure is also known as Heapify.  How to use Heapify to remove a node? ReheapDown ( root, bottom ) maxChild = index of the larger child of nodes[root] if nodes[root] < nodes[maxChild] Swap( nodes[root], nodes[maxChild] ReheapDown( maxChild, bottom )

Example: Continued [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes 20 Swap the root (20) and bottom (12) Remove 20 from the heap

Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes ReheapDown 15 10

Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes Swap the root (20) and bottom (10). It is a heap now. Remove 20 from the heap 15 10

Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes ReheapDown 15

Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes Swap the root (15) and bottom (10) Remove 15 from the heap 10

Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes Reheap

Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes Swap the root (12) and bottom (10) Remove 12 from the heap

Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes 10 We have only one node in the heap. Simply remove 10 from the heap and we are done!

Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes Note that now the array is sorted in ascending order! This is called Heap Sort!

Heap Sort  Algorithm: 1.Construct a heap H of n nodes 2.for i: length(H)-1 to 1 1. Swap H[0] and H[i] 2. Reduce H’s size by 1 3. ReheapDown( 0, i - 1 ) What is the time complexity?  Construct a heap using ReheapUp takes O(nlog 2 n)  ReheapDown takes O(log 2 n) which repeats n-1 times  O(nlog 2 n)

A Better Algorithm to Construct a Heap  Inspired by the Heapify process, we have another algorithm to build a heap:  How to guarantee it is a complete binary tree? —treat the data as an array/sequence, and convert it into a complete binary tree in a top-down, left-to-right manner  How to guarantee the order? —Leaf nodes are already heaps (single node) —Starting from the first non-leaf node till the root, make sure each node is larger than its children  if not, do ReheapDown

Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes Leaf nodes are already heaps. So we start from Level 1: ReheapDown( 2, 5);

Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes ReheapDown( 1, 5);

Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes ReheapDown( 0, 5);

Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] tree.nodes Now we have a heap!

Construct a Heap by Heapify  Algorithm: 1.Elements are stored in array A 2.for i: from (size(A)/2 – 1) to 0 1. ReheapDown( i, size(A)-1)  What is the time complexity? O(n) !!!

Why O(n) to build a heap?  Assume it is a full tree.  Total work to build a heap is:  Since, we can get  So we have T(n) = 2 h+1  Since h = log 2 n, we have T(n) = n+1  O(n)

Summary  Insert a new item to a heap: —The last element violates the order —Use ReheapUp algorithm  Heap Sort: —Keep taking off the root which is the largest —replace the root with the last element in the heap —The new root violates the order —Use ReheapDown algorithm —O(nlog 2 n)  Build a heap: —by ReheapUp: O(nlog 2 n) —by ReheapDown: O(n)