Heaps and the Heapsort Heaps and priority queues Heap structure property Heap ordering property Removal of top priority node Inserting a new node into the heap The heap sort

Heaps and priority queues A heap is a data structure used to implement an efficient priority queue. The idea is to make it efficient to extract the element with the highest priority ­ the next item in the queue to be processed. We could use a sorted linked list, with O(1) operations to remove the highest priority node and O(N) to insert a node. Using a tree structure will involve both operations being O(log2N) which is faster.

Heap structure and position numbering 1 A heap can be visualised as a binary tree in which every layer is filled from the left. For every layer to be full, the tree would have to have a size exactly equal to 2n­1, e.g. a value for size in the series 1, 3, 7, 15, 31, 63, 127, 255 etc. So to be practical enough to allow for any particular size, a heap has every layer filled except for the bottom layer which is filled from the left.

Binary Heap Properties Structure Property Ordering Property Describe these for a binary search tree. For an AVL tree.

Tree Review A DEFJ..NI Its parent or parent’s ancestor Tree T A root(T): leaves(T): children(B): parent(H): siblings(E): ancestors(F): descendents(G): subtree(C): A DEFJ..NI B C D E F G H I Trees have many uses, and we’ll take about them later. To start, let’s just review the basics. Let’s review the words: root: A leaf: DEFJKLMNI child:A - C or H - K leaves have no children parent: C - A or L - H the root has no parent sibling: D - E or F or J - K,L,M, or N grandparent: G to A grandchild: C to H or I ancestor: the node itself or any ancestor’s parent descendent: the node itself or any child’s descendent subtree: a node and all its descendents Its parent or parent’s ancestor Its child or child’s descendent J K L M N Itself plus all descendents

More Tree Terminology # edges on path from root to self depth(B): height(G): degree(B): branching factor(T): # edges on path from root to self # edges on longest path from node to leaf B C # children of a node D E F G MAX # children of a node for a BST this is?? H I Depth: the number of edges along the path from the root to the node height: the number of edges along the longest path from the node to a leaf degree: the number of children of the node branching factor: the maximum degree of any node (or sometimes the average) preorder traversal: running through all the nodes in the tree, starting with the parent, then all the children postorder traversal: run through all the nodes starting with the children and then the parents J K L M N

Some Definitions: skip A Perfect binary tree – A binary tree with all leaf nodes at the same depth. All internal nodes have 2 children. height h 2h+1 – 1 nodes 2h – 1 non-leaves 2h leaves 11 5 21 This is NOT a heap! (but it has the heap structure property) 2 9 16 25 1 3 7 10 13 19 22 30

Heap Structure Property A binary heap is a complete binary tree. Complete binary tree – binary tree that is completely filled, with the possible exception of the bottom level, which is filled left to right. Examples: Since they have this regular structure property, we can take advantage of that to store them in a compact manner. Since they have this regular structure property, we can take advantage of that to store them in a compact manner. If I asked to you store a plain old binary tree in an array, how would you do it?

Representing Complete Binary Trees in an Array 1 2 3 4 5 6 9 8 10 11 12 A From node i: left child: right child: parent: B C 2 * i 7 D E F G (2 * i)+1 H I J K L └ i / 2┘ Left child of node I = 2 * I Right child I = (2*i) +1 Parent of node I is at i/2 (floor) implicit (array) implementation: A B C D E F G H I J K L 1 2 3 4 5 6 7 8 9 10 11 12 13

Heap Order Property Heap order property: For every non-root node X, the value in the parent of X is less than (or equal to) the value in X. This is the order for a MIN heap – could do the same for a max heap. 10 10 20 80 20 80 Tree is PARTIALLY ORDERED. For each node: its value is less than value of all of its descendants. This is the definition for a MIN heap – could do the same for a max heap. 40 60 85 99 30 15 50 700 not a heap This is a PARTIAL order (diff than BST) For each node, its value is less than all of its descendants (no distinction between left and right)

Heap Operations findMin: insert(val): percolate up. How? findMin: insert(val): percolate up. deleteMin: percolate down. Is the tree unique? Swap 85 and 99. Swap 700 and 85? 10 20 80 Is the tree unique? Swap 85 and 99. Swap 700 and 85? 40 60 85 99 50 700 65

Heap – Insert(val) Basic Idea: Put val at “next” leaf position Percolate up by repeatedly exchanging node until no longer needed How long does this take? How long does this take? – max # of exchanges = O(log N) On “average” only need to move up 1.67 levels so get O(1) How long does step 1 take? QUESTION: Max number of exchanges is ? O(log N) – must percolate up to root. On average, analysis shows that you tend to only need to move up 1.67 levels, so get O(1) on average. Optimization in the book – bubble up an EMPTY space, and then do a swap (reduces the # of swaps).

Insert: percolate up 10 Now insert 90. (no swaps, even though 99 is larger!) Now insert 7. 20 80 40 60 85 99 50 700 65 15 Optimization, bubble up an empty space to reduce # of swaps 10 15 80 Now insert 90. (no swaps, even though 99 is larger!) Now insert 7. 40 20 85 99 50 700 65 60

Heap – DeleteMin Basic Idea: Remove root (that is always the min!) Put “last” leaf node at root Find smallest child of node Swap node with its smallest child if needed. Repeat steps 3 & 4 until no swaps needed. If I < = both children, then you are done. How long does step 1 take? QUESTION: Max number of exchanges is ? O(log N) – must percolate all the way to the bottom level. In fact this is often the case, you just took the value from the bottom, there is a good chance it has to go down to the lowest level. O(log N) Optimization in the book – bubble up an EMPTY space down, and then do a swap (reduces the # of swaps). Max # of exchanges? = O(log N), there is a good chance goes to bottom

DeleteMin: percolate down Max # of exchanges? = O(log N), There is a good chance goes to bottom (started at bottom) vs. insert - Could also use the percolate empty bubble down 10 20 15 40 60 85 99 50 700 65 15 Deletemein 20 65 40 60 85 99 50 700

Working on Heaps What are the two properties of a heap? Structure Property Order Property How do we work on heaps? Fix the structure Fix the order

BuildHeap: Floyd’s Method 10 11 12 0 1 2 3 5 11 3 10 6 9 4 8 1 7 2 12 Add elements arbitrarily to form a complete tree. Pretend it’s a heap and fix the heap-order property! 12 Red nodes need to percolate down 5 11 Let’s try pretending it’s a heap already and just fixing the heap-order property. The red nodes are the ones that are out of order. Question: which nodes MIGHT be out of order in any heap? 3 10 6 9 4 8 1 7 2

BuildHeap: Floyd’s Method 10 11 12 0 1 2 3 12 5 11 Red nodes need to percolate down 3 10 6 9 Let’s try pretending it’s a heap already and just fixing the heap-order property. The red nodes are the ones that are out of order. Question: which nodes MIGHT be out of order in any heap? 4 8 1 7 2

BuildHeap: Floyd’s Method 12 5 11 3 10 2 9 4 8 1 7 6

BuildHeap: Floyd’s Method 12 12 5 11 5 11 3 10 2 9 3 1 2 9 4 8 1 7 6 4 8 10 7 6

BuildHeap: Floyd’s Method 12 12 5 11 5 11 3 10 2 9 3 1 2 9 4 8 1 7 6 4 8 10 7 6 12 5 2 3 1 6 9 4 8 10 7 11

BuildHeap: Floyd’s Method 12 12 5 11 5 11 3 10 2 9 3 1 2 9 4 8 1 7 6 4 8 10 7 6 12 12 5 2 1 2 3 1 6 9 3 5 6 9 4 8 10 7 11 4 8 10 7 11

Finally… - Runtime bounded by sum of heights of nodes, which is linear. O(n) - How many nodes at height 1, and height 2, up to root, - See text, Thm. 6.1 p. 194 for detailed proof. 1 3 2 4 5 6 9 How long does this take? Well, everything above the fringe might move 1 step. Everything height 2 or greater might move 2 steps. Most nodes move only a small number of steps  the runtime is O(n). (see text for proof) Full sum = (I=0 to height) SUM (h-I) * 2^i 12 8 10 7 11 runtime:

Facts about Heaps Observations: Realities: Finding a child/parent index is a multiply/divide by two Operations jump widely through the heap Each percolate step looks at only two new nodes Inserts are at least as common as deleteMins Realities: Division/multiplication by powers of two are equally fast Looking at only two new pieces of data: bad for cache! With huge data sets, disk accesses dominate At library – writing a research paper Now, let’s look at some facts about heaps and see if we can’t come up with a new priority queue implementation.

Buildheap pseudocode private void buildHeap() { for ( int i = currentSize/2; i > 0; i-- ) percolateDown( i ); } Adding the items one at a time is O(n log n) in the worst case