A Kind of Binary Tree Usually Stored in an Array Heap Data Structure A Kind of Binary Tree Usually Stored in an Array Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Introduction We saw the Heap sort Now it is time to examine the Heap data structure Unfortunately, there are at least two things named heap Segment of memory in which the memory manager satisfies requests from the new operator or malloc A binary tree like data structure, usually in an array We are interested in the latter today Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Generalities Like many data structures we have a two piece item Key and data AKA key value pair Key has unrestricted form Like trees and hash tables Unlike vectors Needs no pointers even though organized like a tree Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Heaps Heaps are organized more loosely than trees We have nodes that have 0, 1 or 2 descendants The ordering of a heap is different than that of a tree Any path from root to leaf must contain keys that are strictly in ascending (or descending) order How is this different than a tree? Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Trees and Heaps Trees maintain a stronger ordering All items to left are smaller, all to right larger Or the reverse Heaps only require paths from root to leaves to be ascending (or descending) Another requirement: Every level must be full except the leaf level All the leaves must be as far left as possible Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill A Heap 4 5 8 7 13 7 9 9 8 11 8 12 16 20 Clearly not a normal sorted binary tree All paths from root to leaves is in ascending order Only the rightmost leaves may be absent Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Some Definitions A binary tree is completely full if it has height N and contains 2N-1 nodes In other words all interior nodes have both descendants A binary tree is complete if all the leaves are at the bottom two levels and all nodes above the leaves have two descendants A complete tree may also be filled from left Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Complete Picture 12 12 17 17 7 7 9 19 2 9 15 19 Complete Completely full 12 17 7 2 9 15 Complete, left filled Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Order A tree is strongly ordered Makes a good search structure Numerous shapes and balances A heap is weakly ordered Searching a heap is not easy and requires backtracking Always complete and left filled This makes it fit in an array nicely Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Trees in Vectors We want to be able to store a tree or a heap in an array or vector Using no pointers and no subscripts We do this with a clever numbering of the nodes The root is in the first element The left descendant of the root in the second The right descendant of the root in the third Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Trees in Vectors Again In general the descendants of a node are based on its position The descendants of node N are at 2N and 2N+1 Unfortunately this only works if your subscript starts at one For C if the subscript is N then the left is 2N+1 and the right 2N+2 Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Vectorizing 1 Level 0 1 9 7 1 7 Level 1 is 2 2 9 12 9 15 19 3 12 Level 2 is last 4 The descendants of subscript N are Left: 2N+1 Right: 2N+2 Arrays are always of size 2N-1 In this case N=3 Notice that levels are contiguous 4 9 5 15 6 19 Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Complete: Not so good 2 2 17 7 1 7 2 17 12 9 19 3 12 4 9 How are empty slots denoted? 5 6 19 Copyright © 2011-2017 - Curt Hill

Complete Left Filled: Great 2 2 17 7 1 7 2 17 12 9 25 3 12 4 9 The descendants of subscript N are Left: 2N+1 Right: 2N+2 Any array length can now be used Notice again weak ordering, but all paths are ascending from root to leaves 5 25 Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Balance Heaps are always reasonably balanced Not perfectly, but always AVL Thus many things will have order of log2N Insertion Deletion This keeps the analysis easy Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Now What? Heaps are most commonly used in priority queues and heap sort What do these have in common? Need to find the largest or smallest item in the heap Then remove it, while maintaining the properties of the heap Lets look at how this works Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill General Process Remove the root Replace it with something There is only one item that is easy to move The rightmost leaf on the bottom row Swap it with the smaller of the two subnodes Keep this up until it finds its place Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Starting Heap 4 1 5 2 8 4 3 7 4 7 5 8 5 9 7 13 6 13 7 9 7 9 9 8 11 8 12 16 20 8 8 9 11 10 8 11 12 12 16 13 20 Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Remove Root 1 5 2 8 3 7 4 7 5 8 5 9 7 13 6 13 7 9 7 9 9 8 11 8 12 16 20 8 8 9 11 10 8 11 12 12 16 13 20 Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Move Last 1 1 5 2 8 3 7 4 7 5 8 5 9 7 13 6 13 7 9 7 9 9 8 11 8 12 16 20 8 8 9 11 10 8 11 12 12 16 13 20 Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Move Last 2 20 1 5 2 8 20 3 7 4 7 5 8 5 9 7 13 6 13 7 9 7 9 9 8 11 8 12 16 8 8 9 11 10 8 11 12 12 16 13 Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Find Smallest 20 1 5 2 8 20 3 7 4 7 5 8 5 9 7 13 6 13 7 9 7 9 9 8 11 8 12 16 8 8 9 11 10 8 11 12 12 16 13 Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Swap 5 1 20 2 8 5 3 7 4 7 20 8 5 9 7 13 6 13 7 9 7 9 9 8 11 8 12 16 8 8 9 11 10 8 11 12 12 16 13 Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Find Smallest 5 1 20 2 8 5 3 7 4 7 20 8 5 9 7 13 6 13 7 9 7 9 9 8 11 8 12 16 8 8 9 11 10 8 11 12 12 16 13 Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Swap 5 1 7 2 8 5 3 20 4 7 7 8 5 9 20 13 6 13 7 9 7 9 9 8 11 8 12 16 8 8 9 11 10 8 11 12 12 16 Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Find Smallest 5 1 7 2 8 5 3 20 4 7 7 8 5 9 20 13 6 13 7 9 7 9 9 8 11 8 12 16 8 8 9 11 10 8 11 12 12 16 Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Swap 5 1 7 2 8 5 3 8 4 7 7 8 5 9 8 13 6 13 7 9 7 9 9 20 11 8 12 16 8 20 9 11 10 8 11 12 12 16 Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Done 5 1 7 2 8 5 3 8 4 7 7 8 5 9 8 13 6 13 7 9 7 9 9 20 11 8 12 16 8 20 9 11 Now a heap ready for next removal 10 8 11 12 12 16 Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Commentary The one to remove can be moved in constant time Putting heap back into shape requires Log2N moves Removing everything is NLog2N Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Insertion Something like the reverse of the above process There is only one place to put it Just to right of leftmost bottom item Then sift it up to correct position Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Starting 5 1 7 2 8 5 3 8 4 7 7 8 5 9 8 13 6 13 7 9 7 9 9 20 11 8 12 8 20 9 11 Want to insert 17 10 8 11 12 Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Insert 17 5 1 7 2 8 5 3 8 4 7 7 8 5 9 8 13 6 13 7 9 7 9 9 20 11 8 12 17 8 20 9 11 10 8 Done quickly 11 12 12 17 Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Insert 6 5 1 7 2 8 5 3 8 4 7 7 8 5 9 8 13 6 13 7 9 7 9 9 20 11 8 12 17 6 8 20 9 11 10 8 11 12 12 17 13 6 Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Swap 6 – 13 5 1 7 2 8 5 3 8 4 7 7 8 5 9 8 13 6 13 7 9 7 9 9 20 11 8 12 17 6 8 20 9 11 10 8 11 12 12 17 13 6 Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Swap 6 – 8 5 1 7 2 8 5 3 8 4 7 7 8 5 9 8 6 6 6 7 9 7 9 9 20 11 8 12 17 13 8 20 9 11 10 8 11 12 12 17 13 13 Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Done 5 1 7 2 6 5 3 8 4 7 7 6 5 9 8 8 6 8 7 9 7 9 9 20 11 8 12 17 13 8 20 9 11 10 8 11 12 12 17 13 13 Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Commentary When sifting down it easy to find the next two sub-nodes 2N+1 and 2N+2 When sifting up how do we determine this? Odd subscripts are left sub-tree nodes and evens are rights So if N is odd subtract 1 and divide by 2, otherwise subtract 2 and divide by 2 Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Applications Priority Queues Heap sort Selections of more than one in some ordered way What is the difference between a Priority Queue and a Sort? Copyright © 2011-2017 - Curt Hill

Priority Queue and Sort If all the items enter the queue before any are removed then there is no difference Normally a sort is discrete: All enter, sort occurs, all leave A priority queue is continuous: During the operation items may enter and leave in any order Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Selections A single selection is usually the first pass of a Selection sort If more are desired, such as the first M items of N, then a heap may be the better structure Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Heap Sort Analysis Two steps make heap sort Build the heap in the array Like insertion sort, pick off items one at a time and sift into heap Should be NLog2N Pick out each root and sift the items Should also be NLog2N Cannot be worse than 2NLog2N Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Can we do better? Actually yes We can actually build the heap in O(N) Lets consider that nice trick Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Making a Heap Work from bottom up and make each sub-tree a valid heap Only the top half of the array needs to be considered Leaves are automatically correct Look at each non-leaf node and compare with its two descendants If either is wrong swap and recursively look at the new descendant Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Heap sort If we make the correction a function then the sort is very simple Make a heap using a sift function Make the largest item the top of the heap The sort becomes exchanging the top of the heap with last element and resifting it Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Priority Queue Times In previous class we observed the list implementation of a priority queue Removal is constant time but insertion is linear (N) With heap implementation both are Log2N which is better Heaps are harder to code than lists, so the decision is based upon how much traffic is expected Copyright © 2011-2017 - Curt Hill

Copyright © 2011-2017 - Curt Hill Finally Heaps are weakly ordered trees within an array Insertion and removal are Log2N See also the STL Priority Queue Copyright © 2011-2017 - Curt Hill