DATA STRUCTURES AND ALGORITHMS Lecture Notes 8 Prepared by İnanç TAHRALI

2 ROAD MAP PRIORITY QUEUES (HEAPS) Model Simple Implementations Binary Heap Applications of Priority Queues d-Heaps Leftist Heaps

3 Model Priority Queue is a data structure that allows at least two operations insert equivalent to enqueue operation in queue ADT deleteMin finds, returns and removes the minimum element in priority queue equivalent to dequeue operation in queue ADT

4 Model

5 Many applications Printer queues Multiuser O/S process scheduling Many graph algorithms

6 How to Implement a Priority Queue Linked list Unsorted insertion takes O(1) time deletion takes O(N) time Sorted insertion takes O(N) time deletion takes O(1) time

7 How to Implement a Priority Queue Binary Search Tree Insertion and deletion takes O(logN) time Uses pointers Complicated Constant running time is large Supports more functions that are not necessary

8 How to Implement a Priority Queue Heap Does not use pointers N insertion takes O(N) time Insertion takes O(1) on amortize DeleteMin takes O(logN) Efficient merge of priority queues by using pointers

9 Heap Structures Binary Heap d-Heap Leftist Heap Binomial Heap Fibonacci Heap Binary heap is very common ! We will refer to binary heaps as “heaps” Heaps have 2 properties Structure property Heap-Order property

10 Structure Property of Heaps Binary heap is a complete binary tree completely filled except the bottom level which is filled from left to right if heap has height h then number of nodes is between 2 h and 2 h+1 -1 height of a heap with N nodes is O(logN) complete binary tree it is a regular structure so can be represented in an array !

11 Structure Property of Heaps For an element in position i 2i is the left child 2i+1 is the right child i/2 is the parent array implementation requires an estimate on max-size Array implementation of complete binary tree

12 Heap-Order Property For each node X, key in the parent is smaller than the key in X (except the root) Minimum element can always be found at the root Only left tree is a heap !

13 // class interface for priority queues template class BinaryHeap { public: explicit BinaryHeap( int capacity = 100 ); bool isEmpty( ) const; bool isFull( ) const; const Comparable & findMin( ) const; void insert( const Comparable & x ); void deleteMin( ); void deleteMin( Comparable & minItem ); void makeEmpty( ); private: int currentSize; // Number of elements in heap vector array; // The heap array void buildHeap( ); void percolateDown( int hole ); };

14 /* Construct the binary heap. template BinaryHeap ::BinaryHeap( int capacity ) : array( capacity + 1 ), currentSize( 0 ) { } /*Test if the priority queue is logically empty. template bool BinaryHeap ::isEmpty( ) const { return currentSize == 0; } /*Test if the priority queue is logically full. template bool BinaryHeap ::isFull( ) const { return currentSize == array.size( ) - 1; } /*Make the priority queue logically empty. template void BinaryHeap ::makeEmpty( ) { currentSize = 0;

15 Basic Heap Operations - insert Algorithm create a hole in the next available array position while inserting X in the hole violate heap order Percolate up the hole (by swapping with the parent) insert X in the hole

16 Example Attempt to insert 14 Creating a hole Bubbling the hole up

17 Example Remaining two steps to insert 14

18 Anaylsis of Insert Operation Analysis If the hole percolated up d levels run-time is O(d) In worst case O(logN) On average case O(1) comparisons

19 /* Insert item x into the priority queue template void BinaryHeap ::insert( const Comparable & x ) { if( isFull( ) ) throw Overflow( ); // Percolate up int hole = ++currentSize; for( ; hole > 1 && x < array[ hole / 2 ]; hole /= 2 ) array[ hole ] = array[ hole / 2 ]; array[ hole ] = x; }

20 Basic Heap Operations-deleteMin Algorithm Find the minimum (at the root) Remove the minimum  a hole is created at root While the last element X can not be inserted in the hole without violating the heap order Percolate down the hole  swap with smaller child Insert X into the hole Return minimum

21 Example Creation of the hole at the root First 13 is removed, and 31 is placed in the heap

22 Example Next two steps in deleteMin

23 Example Last two steps in deleteMin

24 Anaylsis of deleteMin Operation A path from root to an internal node is traced Run time = O(logN) in worst and average case

25 /* Remove the smallest item from the priority queue template void BinaryHeap ::deleteMin( Comparable & minItem ) { if( isEmpty( ) ) throw Underflow( ); minItem = array[ 1 ]; array[ 1 ] = array[ currentSize-- ]; percolateDown( 1 ); } /* Internal method to percolate down in the heap. template void BinaryHeap ::percolateDown( int hole ) { int child; Comparable tmp = array[ hole ]; for( ; hole * 2 <= currentSize; hole = child ) { child = hole * 2; if(child != currentSize && array[child + 1]<array[child] ) child++; if( array[ child ] < tmp )array[ hole ] = array[ child ]; elsebreak; } array[ hole ] = tmp; }

26 Other Heap Operations decreaseKey Percolate up the node increaseKey Percolate down the node remove Removes the node at position p from heap perform decreaseKey perform deleteMin buildHeap Insert N keys to an empty heap perform N insertions Worst case  O(NlogN) Average case  O(N) First build the structure and then put in heap order

27 Algorithm of buildHeap Operation /* Establish heap order property template void BinaryHeap ::buildHeap( ){ for( int i = currentSize / 2; i > 0; i-- ) percolateDown( i ); }

28 Example Left : initial heap; right: after percolateDown(7) Left : after percolateDown(6); right: after percolateDown(5)

29 Example Left : after percolateDown(4); right: after percolateDown(3) Left : after percolateDown(2); right: after percolateDown(1)

30 Sum of heights of the nodes Theorem : For the perfect binary tree of height h containing 2 h+1 -1 nodes, the sum of the heights of the nodes is 2 h+1 -1 –(h+1) Proof : Tree consist of 1 node at height h, 2 nodes at height h-1, 2 2 nodes at height h-2, in general 2 i nodes at height h-i

31 The sum of the heights of all the nodes is then Multiplying by 2 gives the equation If we subtract two equations

32 Applications of Priority Queues 1- Selection Problem input = N elements output = k th largest element Algorithm: read N elements in an array  O(n) buildHeap  O(n) perform k deleteMin  O(k logn) Total RunTime= O(n + k logn) if k = O(n/logn)  runtime = O(n) if k = Γn/2  runtime = O(n logn)

33 Applications of Priority Queues 2- Event Simulation Assume a bank with k tellers Customers arrive and wait until a teller available Customer arrival time and service time has a distribution function Problem : Use a simulation to determine number of tellers to ensure that the average customer wait is low. Events Customer arrival  use regular queue to determine next event Customer left  use priority queue to determine next event There are C customers  2C events Run time = O (c log k)

34 d-Heaps Nodes have at most d children Shallow  Insert  one comparison for each percolate Delete  d-1 comparisons Good when number of insertions are much greater than deletion

35 A 3-Heap

36 Leftist Heaps Support merge operation efficiently Merge  O(N) for binary heaps Using arrays θ(N) Efficient implementation Pointers makes others slower Leftist heap has both a structural property and an ordering property Its difference from binary heap is that, leftist heaps are not perfectly balanced

37 Leftist Heaps Definition : Null path length of a node x is the length of the shortest path from X to a node without two children. Only left tree is lefist heap

38 Theorem : A leftist tree with r nodes on the right path must have at least 2 r -1 nodes Proof by induction: –if r = 1, there must be at least one tree node. –otherwise, suppose that the theorem is true for 1,2,…,r. –consider a leftist tree with r+1 noes on the right path –then the root has a right subtree with r nodes on the right path, and a left subtree with at least r nodes on the right path (otherwise it would not be leftist) –applying in the inductive hypothesis to these subtrees yields a minimum of 2 r -1 nodes in each subtree –this plus the root gives at least 2 r+1 -1 nodes in the tree, prooving the theorem.

39 Leftist Heap Operations Fundamental operation on leftist heaps is “merging” Algorithm if H1 or H2 is NULL return the other one compare the root values (assume H1 is smaller) H1->right = Merge(H1->right, H2) swap the left and right subtrees if necessary if leftist heap property is violated update the Npl of the root of H1 return H1

40 Leftist Heap Operations Two leftist heaps H 1 and H 2

41 Leftist Heap Operations Result of merging H 2 and H 1 ’s right subheap

42 Leftist Heap Operations Result of attaching leftist heap of previous figure as H 1 ’s right child

43 Leftist Heap Operations Result swapping children of H 1 ’s root

44 Leftist Heap Operations Result of merging right paths of H 1 and H 2

45 template class LeftistNode { Comparable element; LeftistNode *left; LeftistNode *right; int npl; LeftistNode(const Comparable & theElement, LeftistNode *lt=NULL, LeftistNode *rt = NULL, int np=0 ) : element( theElement ), left( lt ), right( rt ), npl( np ) { } friend class LeftistHeap ; };

46 template class LeftistHeap { public: LeftistHeap( ); LeftistHeap( const LeftistHeap & rhs ); ~LeftistHeap( ); bool isEmpty( ) const; bool isFull( ) const; const Comparable & findMin( ) const; void insert( const Comparable & x ); void deleteMin( ); void deleteMin( Comparable & minItem ); void makeEmpty( ); void merge( LeftistHeap & rhs ); const LeftistHeap & operator=( const LeftistHeap & rhs );

47 private: LeftistNode *root; LeftistNode * merge( LeftistNode *h1, LeftistNode *h2 ) const; LeftistNode * merge1( LeftistNode *h1, LeftistNode *h2 ) const; void swapChildren( LeftistNode * t ) const; void reclaimMemory( LeftistNode * t ) const; LeftistNode * clone( LeftistNode *t ) const; };

48 /*Construct the leftist heap. template LeftistHeap ::LeftistHeap( ) { root = NULL; } /* Copy constructor. template LeftistHeap ::LeftistHeap( const LeftistHeap & rhs ) { root = NULL; *this = rhs; } /*Destruct the leftist heap. template LeftistHeap ::~LeftistHeap( ) { makeEmpty( ); }

49 /*Merge rhs into the priority queue. rhs becomes empty. rhs must be different from this. template void LeftistHeap ::merge( LeftistHeap & rhs ) { if( this == &rhs ) // Avoid aliasing problems return; root = merge( root, rhs.root ); rhs.root = NULL; } /*Internal method to merge two roots. template LeftistNode * LeftistHeap ::merge( LeftistNode * h1, LeftistNode * h2 ) const { if( h1 == NULL )return h2; if( h2 == NULL )return h1; if( h1->element element ) return merge1( h1, h2 ); else return merge1( h2, h1 ); }

50 /*Internal method to merge two roots. Assumes trees are not empty, and h1's root contains smallest item. template LeftistNode * LeftistHeap :: merge1( LeftistNode * h1, LeftistNode * h2 ) const { if( h1->left == NULL ) // Single node h1->left = h2; // Other fields in h1 already accurate else { h1->right = merge( h1->right, h2 ); if( h1->left->npl right->npl ) swapChildren( h1 ); h1->npl = h1->right->npl + 1; } return h1; }

51 /*Insert item x into the priority queue, maintaining heap order. template void LeftistHeap ::insert( const Comparable & x ) { root = merge( new LeftistNode ( x ), root ); } /* Remove the smallest item from the priority queue. template void LeftistHeap ::deleteMin( ) { if( isEmpty( ) ) throw Underflow( ); LeftistNode *oldRoot = root; root = merge( root->left, root->right ); delete oldRoot; }