Presentation is loading. Please wait.

Presentation is loading. Please wait.

Priority Queues, Heaps, and Heapsort CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1.

Published byBrice Henry Modified over 8 years ago

Similar presentations

Presentation on theme: "Priority Queues, Heaps, and Heapsort CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1."— Presentation transcript:

1 Priority Queues, Heaps, and Heapsort CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1

2 Priority Queues So far we have seen sorting methods that work in batch mode: – They are given all the items at once – They sort the items. – Done! Another case of interest is online methods, that deal with data that change. Goal: support (efficiently): – Insertion of a new element. – Deletion of the max element. – Initialization (organizing an initial set of data). The abstract data type that supports these operations is called priority queue. 2

3 Priority Queues - Applications Scheduling: – Flight take-offs and landings. – Programs getting executed on a computer. – Real-time requests for information on a database system. – Computer simulations and games, to schedule a sequence of events. Waiting lists: – Students getting admitted to college. – Patients getting admitted to a hospital. Lots more… 3

4 Priority Queues and Sorting Priority queues support: – Insertion of a new element. – Deletion of the max element. – Initialization (organizing an initial set of data). These operations support applications that batch methods, like quicksort, mergesort, do not support. However, these operations can also support sorting: Given items to sort: – Initialize a priority queue that contains those items. – Initialize result to empty list. – While the priority queue is not empty: Remove max element from queue, add it to beginning of result. We will see an implementation (heapsort) of this algorithm that takes Θ(N lg N) time. 4

5 Naïve Implementation Using Arrays Initialization: – Given N data, just store them on an array. – Time: Θ(???) Insertion of a new item: – (Assumption: the array has enough memory.) – Store the item at the end of the array. – Time: Θ(???) Deletion of max element: – Scan the array to find max item. – Delete that item. – Time: Θ(???) 5

6 Naïve Implementation Using Arrays Initialization: – Given N data, just store them on an array. – Time: Θ(N), good! Insertion of a new item: – (Assumption: the array has enough memory.) – Store the item at the end of the array. – Time: Θ(1), good! Deletion of max element: – Scan the array to find max item. – Delete that item. – Time: Θ(N), bad! 6

7 Naïve Implementation Using Lists Initialization: – Given N data, just store them on an list. – Time: Θ(N), good! Insertion of a new item: – Store the item at the beginning (or end) of the list. – Time: Θ(1), good! Deletion of max element: – Scan the list to find max item. – Delete that item. – Time: Θ(N), bad! 7

8 Using Ordered Arrays/Lists Initialization: – Given N data, sort them. – Time: Θ(???) Insertion of a new item: – (Assumption: if using an array, it must have enough memory.) – Insert the item at the right place, to keep array/list sorted. – Time: Θ(???) Deletion of max element: – Delete the last item. – Time: Θ(???) 8

9 Using Ordered Arrays/Lists Initialization: – Given N data, sort them. – Time: O(N lg N). OK! Insertion of a new item: – (Assumption: if using an array, it must have enough memory.) – Insert the item at the right place, to keep array/list sorted. – Time: O(N). Bad! Deletion of max element: – Delete the last item. – Time: Θ(1). Good! 9

10 Using Heaps (New Data Type) Initialization: – Given N data, heapify them (we will see how in a few slides). – Time: Θ(N). Good! Insertion of a new item: – Insert the item at the right place, to maintain the heap property. (details in a few slides). – Time: O(lg N). Good! Deletion of max element: – Delete the first item. – Rearrange other items, to maintain the heap property. (details in a few slides). – Time: O(lg N). Good! 10

11 Definition of Heaps We have two equivalent representations of heaps: – As binary trees. – As arrays. Thus, we have two logically equivalent definitions: A binary tree is a heap if, for every node N in that tree, the key of N is larger than or equal to the keys of the children of N, if any. An array A (with 1 as the first index) is a heap if, for every position N of A: – If A[2N] is not out of bounds, then A[N] >= A[2N]. – If A[2N + 1] is not out of bounds, then A[N] >= A[2*N + 1]. 11

12 Representing a Heap Consider this array: We can draw the array as a tree. – The children of A[N] are A[2N] and A[2N+1]. 12 position123456789101112 valueXTOGSMNAERAI

13 Representing a Heap Consider this array: We can draw the array as a tree. – The children of A[N] are A[2N] and A[2N+1]. – This example shows that the tree and array representations are equivalent. 13 position123456789101112 valueXTOGSMNAERAI X T N O G S I M A A E R

14 Representing a Heap A binary tree representing a heap should be complete. All levels are full, except possibly for the last level. At the last level: – Nodes are placed on the left. – Empty positions are placed on the right. 14 X T N O G S I M A A E R

15 Increasing a Key Also called “increasing the priority” of an item. Such an operation can lead to violation of the heap property. Easy to fix: – Exchange items as needed, between node and parent, starting at the node that changed key. 15 X T N O G S I M A A E R

16 Increasing a Key Also called “increasing the priority” of an item. Such an operation can lead to violation of the heap property. Easy to fix: – Exchange items as needed, between node and parent, starting at the node that changed key. Example: – An E changes to a V. 16 X T N O G S I M A A EVEV R

17 Increasing a Key Also called “increasing the priority” of an item. Such an operation can lead to violation of the heap property. Easy to fix: – Exchange items as needed, between node and parent, starting at the node that changed key. Example: – An E changes to a V. – Exchange V and G. Done? 17 X T N O V S I M A A G R

18 Increasing a Key Also called “increasing the priority” of an item. Such an operation can lead to violation of the heap property. Easy to fix: – Exchange items as needed, between node and parent, starting at the node that changed key. Example: – An E changes to a V. – Exchange V and G. – Exchange V and T. Done? 18 X V N O T S I M A A G R

19 Increasing a Key Also called “increasing the priority” of an item. Such an operation can lead to violation of the heap property. Easy to fix: – Exchange items as needed, between node and parent, starting at the node that changed key. Example: – An E changes to a V. – Exchange V and G. – Exchange V and T. Done. 19 X V N O T S I M A A G R

20 Increasing a Key Implementation: fixUp(Item a[], int k) { while ((k > 1) && (less(a[k/2], a[k]))) { exch(a[k], a[k/2]); k = k/2; } 20 X V N O T S I M A A G R

21 Decreasing a Key Also called “decreasing the priority” of an item. Such an operation can lead to violation of the heap property. Easy to fix: – Exchange items as needed, between node and largest child, starting at the node that changed key. 21 X T N O G S I M A A E R

22 Decreasing a Key Also called “decreasing the priority” of an item. Such an operation can lead to violation of the heap property. Easy to fix: – Exchange items as needed, between node and largest child, starting at the node that changed key. Example: – An X changes to a B. 22 XBXB T N O G S I M A A E R

23 Decreasing a Key Also called “decreasing the priority” of an item. Such an operation can lead to violation of the heap property. Easy to fix: – Exchange items as needed, between node and largest child, starting at the node that changed key. Example: – An X changes to a B. – Exchange B and T. 23 T B N O G S I M A A E R

24 Decreasing a Key Also called “decreasing the priority” of an item. Such an operation can lead to violation of the heap property. Easy to fix: – Exchange items as needed, between node and largest child, starting at the node that changed key. Example: – An X changes to a B. – Exchange B and T. – Exchange B and S. 24 T S N O G B I M A A E R

25 Decreasing a Key Also called “decreasing the priority” of an item. Such an operation can lead to violation of the heap property. Easy to fix: – Exchange items as needed, between node and largest child, starting at the node that changed key. Example: – An X changes to a B. – Exchange B and T. – Exchange B and S. – Exchange B and R. 25 T S N O G R I M A A E B

26 Decreasing a Key Implementation: fixDown(Item a[], int k, int N) { int j; while (2*k <= N) { j = 2*k; if ((j < N) && less ((a[j], a[j+1]))) j++; if (!less(a[k], a[j])) break; exch(a[k], a[j]); k = j; } 26 T S N O G R I M A A E B

27 Insertions and Deletions To insert an item to a heap: – Insert the item to the end of the heap. – Call fix up to restore the heap property. – Time = O(???) The only element we care to delete from a heap is the maximum element. This element is always the first element of the heap. To delete the maximum element: – Exchange the first and last elements of the heap. – Delete the last element (which is the maximum element). – Call fixDown to restore the heap property. – Time = O(???) 27

28 Insertions and Deletions To insert an item to a heap: – Insert the item to the end of the heap. – Call fix up to restore the heap property. – Time = O(lg N) The only element we care to delete from a heap is the maximum element. This element is always the first element of the heap. To delete the maximum element: – Exchange the first and last elements of the heap. – Delete the last element (which is the maximum element). – Call fixDown to restore the heap property. – Time = O(lg N) 28

29 Batch Initialization Batch initialization of a heap is the process of converting an unsorted array of data into a heap. We will see two methods that are pretty easy to implement: Top-down batch initialization. – O(N lg N) time. – O(N) extra space (in addition to the space that the input array already takes). Bottom-up batch initialization. – O(N) time. – O(1) extra space (in addition to the space that the input array already takes). 29

30 Top-Down Batch Initialization Heap top_down_heap_init(Item * array, int N) Heap result = newHeap(N). for counter = 0,..., N-1. heap_insert(array[counter]). return result. How much time does this take? 30

31 Top-Down Batch Initialization Heap top_down_heap_init(Item * array, int N) Heap result = newHeap(N). for counter = 0,..., N-1. heap_insert(array[counter]). return result. How much time does this take? – We need to do N insertions. – Each insertion takes O(lg N) time. – So, in total, we need O(N lg N) time. 31

32 Bottom-Up Batch Initialization struct heap_struct { int length; Item * array; }; 32 typedef struct heap_struct * Heap; Heap bottom_up_heap_init(Item * array, int N) for counter = N/2,..., 1 fixDown(array, counter, N). Heap result = malloc(sizeof(*result)). result->array = array. result->N = N. return result.

33 Visualizing Bottom-Up Initialization N = 14 Data to be organized into a heap: 33 position1234567891011121314 value5040301560102845355595908560

34 Visualizing Bottom-Up Initialization N = 14 counter = 7 fixDown(counter, N): 34 position123456*7891011121314 value5040301560102845355595908560

35 Visualizing Bottom-Up Initialization N = 14 counter = 7 fixDown(counter, N): 35 position123456*7891011121314 value5040301560106045355595908528

36 Visualizing Bottom-Up Initialization N = 14 counter = 6 fixDown(counter, N): 36 position12345*67891011121314 value5040301560106045355595908528

37 Visualizing Bottom-Up Initialization N = 14 counter = 6 fixDown(counter, N): 37 position12345*67891011121314 value5040301560906045355595108528

38 Visualizing Bottom-Up Initialization N = 14 counter = 5 fixDown(counter, N): 38 position1234*567891011121314 value5040301560906045355595108528

39 Visualizing Bottom-Up Initialization N = 14 counter = 5 fixDown(counter, N): 39 position1234*567891011121314 value5040301595906045355560108528

40 Visualizing Bottom-Up Initialization N = 14 counter = 4 fixDown(counter, N): 40 position123*4567891011121314 value5040301595906045355560108528

41 Visualizing Bottom-Up Initialization N = 14 counter = 4 fixDown(counter, N): 41 position123*4567891011121314 value5040304595906015355560108528

42 Visualizing Bottom-Up Initialization N = 14 counter = 3 fixDown(counter, N): 42 position12*34567891011121314 value5040304595906015355560108528

43 Visualizing Bottom-Up Initialization N = 14 counter = 3 fixDown(counter, N): 43 position12*34567891011121314 value5040904595306015355560108528

44 Visualizing Bottom-Up Initialization N = 14 counter = 3 fixDown(counter, N): 44 position12*34567891011121314 value5040904595856015355560103028

45 Visualizing Bottom-Up Initialization N = 14 counter = 2 fixDown(counter, N): 45 position1*234567891011121314 value5040904595856015355560103028

46 Visualizing Bottom-Up Initialization N = 14 counter = 2 fixDown(counter, N): 46 position1*234*5*567891011121314 value5095904540856015355560103028

47 Visualizing Bottom-Up Initialization N = 14 counter = 2 fixDown(counter, N): 47 position1*234567891011121314 value5095904560856015355540103028

48 Visualizing Bottom-Up Initialization N = 14 counter = 1 fixDown(counter, N): 48 position*1234567891011121314 value5095904560856015355540103028

49 Visualizing Bottom-Up Initialization N = 14 counter = 1 fixDown(counter, N): 49 position*1*2*234567891011121314 value9550904560856015355540103028

50 Visualizing Bottom-Up Initialization N = 14 counter = 1 fixDown(counter, N): 50 position*1234*5*567891011121314 value9560904550856015355540103028

51 Visualizing Bottom-Up Initialization N = 14 counter = 1 fixDown(counter, N): 51 position*1234567891011121314 value9560904555856015355040103028

52 Visualizing Bottom-Up Initialization N = 14 counter = 1 DONE!!! The heap condition is now satisfied. 52 position1234567891011121314 value9560904555856015355040103028

53 Running Time How can we analyze the running time? To simplify, suppose that N = 2 n - 1. The counter starts at value ???. At that point, we call fixDown on a heap of size ???. 53 Heap bottom_up_heap_init(Item * array, int N) for counter = N/2,..., 1 fixDown(array, counter, N). Heap result = (Heap)malloc(sizeof(*result)). result.array = array. result.N = N. return result.

54 Running Time How can we analyze the running time? To simplify, suppose that N = 2 n - 1. The counter starts at value 2 n-1 - 1. At that point, we call fixDown on a heap of size 3 (= 2 2 - 1). For counter values between 2 n-1 - 1 and 2 n-2, we call fixDown on a heap of size 2 2 - 1. 54 Heap bottom_up_heap_init(Item * array, int N) for counter = N/2,..., 1 fixDown(array, counter, N). Heap result = (Heap)malloc(sizeof(*result)). result.array = array. result.N = N. return result.

55 Running Time For counter values between 2 n-1 - 1 and 2 n-2, we call fixDown on a heap of size 2 2 - 1. For counter values between 2 n-2 - 1 and 2 n-3, we call fixDown on a heap of size ???. … For counter value 2 0 we call fixDown on a heap of size ???. 55 Heap bottom_up_heap_init(Item * array, int N) for counter = N/2,..., 1 fixDown(array, counter, N). Heap result = (Heap)malloc(sizeof(*result)). result.array = array. result.N = N. return result.

56 Running Time For counter values between 2 n-1 - 1 and 2 n-2, we call fixDown on a heap of size 2 2 - 1. For counter values between 2 n-2 - 1 and 2 n-3, we call fixDown on a heap of size 7 (= 2 3 - 1). … For counter value 2 0 we call fixDown on a heap of size 2 n - 1. 56 Heap bottom_up_heap_init(Item * array, int N) for counter = N/2,..., 1 fixDown(array, counter, N). Heap result = (Heap)malloc(sizeof(*result)). result.array = array. result.N = N. return result.

57 Running Time 57 Counter: from Counter: to Number of Iterations Heap SizeTime per Iteration Time for All Iterations 2 n-1 - 12 n-2 2 2 - 1O(2)O(2 n-2 * 2) 2 n-2 - 12 n-3 2 3 - 1O(3)O(2 n-3 * 3) 2 n-3 - 12 n-4 2 4 - 1O(4)O(2 n-4 * 4) … 2 1 - 1=12 0 = 1 2 n - 1O(n)O(2 0 * n)

58 Bottom-Up Versus Top-Down Top-down initialization does not touch the input array. – Instead, it creates a new heap, where it inserts the data. – Thus, it needs O(N) extra space, in addition to the space already taken by the input array. Bottom-up initialization, instead, changes the input array. – The heap does not allocate memory for a new array. – Instead, the heap uses the input array as its own array. – Consequently, it needs O(1) extra space, in addition to the space already taken by the input array. 58

59 Heapsort Use a heap to sort an array, by repeatedly: Swap top (largest element) and last element. Remove last element (current largest). In the code below: Where is the sorted data? How does the heap get shorter? void heapsort(Item a[], int N) bottom_up_heap_init(a, N) for counter = N, …, 2 exch(a[1], a[counter]) fixDown(a, 1, counter-1) 59

60 Finding top k largest elements 60

61 Finding top k largest elements Using a max-heap – Build max-heap of size N from all elements, then – remove k Using a min-heap – Build min-heap of size k from first k elements. – Continue with (N-k) set of insert and then delete. – Left with a min-heap of size k where all the elements left are the largest ones. – advantage: less space (constant space: k) 61

Download ppt "Priority Queues, Heaps, and Heapsort CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1."

Similar presentations

COL 106 Shweta Agrawal and Amit Kumar

COL 106 Shweta Agrawal and Amit Kumar
Transform and Conquer Chapter 6. Transform and Conquer Solve problem by transforming into: a more convenient instance of the same problem (instance simplification)

Transform and Conquer Chapter 6. Transform and Conquer Solve problem by transforming into: a more convenient instance of the same problem (instance simplification)
BY Lecturer: Aisha Dawood. Heapsort  O(n log n) worst case like merge sort.  Sorts in place like insertion sort.  Combines the best of both algorithms.

BY Lecturer: Aisha Dawood. Heapsort  O(n log n) worst case like merge sort.  Sorts in place like insertion sort.  Combines the best of both algorithms.
Analysis of Algorithms

Analysis of Algorithms
CS 46101 Section 600 CS 56101 Section 002 Dr. Angela Guercio Spring 2010.

CS Section 600 CS Section 002 Dr. Angela Guercio Spring 2010.
David Luebke 1 5/20/2015 CS 332: Algorithms Quicksort.

David Luebke 1 5/20/2015 CS 332: Algorithms Quicksort.
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu.

Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu.
Binary Heaps CSE 373 Data Structures Lecture 11. 2/5/03Binary Heaps - Lecture 112 Readings Reading ›Sections 6.1-6.4.

Binary Heaps CSE 373 Data Structures Lecture 11. 2/5/03Binary Heaps - Lecture 112 Readings Reading ›Sections
More sorting algorithms: Heap sort & Radix sort. Heap Data Structure and Heap Sort (Chapter 7.6)

More sorting algorithms: Heap sort & Radix sort. Heap Data Structure and Heap Sort (Chapter 7.6)
TCSS 343, version 1.1 Algorithms, Design and Analysis Transform and Conquer Algorithms Presorting HeapSort.

TCSS 343, version 1.1 Algorithms, Design and Analysis Transform and Conquer Algorithms Presorting HeapSort.
Version 1.0 1 TCSS 342, Winter 2006 Lecture Notes Priority Queues Heaps.

Version TCSS 342, Winter 2006 Lecture Notes Priority Queues Heaps.
Comp 122, Spring 2004 Heapsort. heapsort - 2 Lin / Devi Comp 122 Heapsort  Combines the better attributes of merge sort and insertion sort. »Like merge.

Comp 122, Spring 2004 Heapsort. heapsort - 2 Lin / Devi Comp 122 Heapsort  Combines the better attributes of merge sort and insertion sort. »Like merge.
3-Sorting-Intro-Heapsort1 Sorting Dan Barrish-Flood.

3-Sorting-Intro-Heapsort1 Sorting Dan Barrish-Flood.
UMass Lowell Computer Science 91.404 Analysis of Algorithms Prof. Karen Daniels Spring, 2007 Heap Lecture Chapter 6 Use NOTES feature to see explanation.

UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Spring, 2007 Heap Lecture Chapter 6 Use NOTES feature to see explanation.
Lecture 11 Binary Heap King Fahd University of Petroleum & Minerals College of Computer Science & Engineering Information & Computer Science Department.

Lecture 11 Binary Heap King Fahd University of Petroleum & Minerals College of Computer Science & Engineering Information & Computer Science Department.
David Luebke 1 7/2/2015 Merge Sort Solving Recurrences The Master Theorem.

David Luebke 1 7/2/2015 Merge Sort Solving Recurrences The Master Theorem.
Heaps and heapsort COMP171 Fall 2005 Part 2. Sorting III / Slide 2 Heap: array implementation 1 2 4 8 9 10 5 3 7 6 Is it a good idea to store arbitrary.

Heaps and heapsort COMP171 Fall 2005 Part 2. Sorting III / Slide 2 Heap: array implementation Is it a good idea to store arbitrary.
UMass Lowell Computer Science 91.404 Analysis of Algorithms Prof. Karen Daniels Fall, 2001 Heap Lecture 2 Chapter 7 Wed. 10/10/01 Use NOTES feature to.

UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2001 Heap Lecture 2 Chapter 7 Wed. 10/10/01 Use NOTES feature to.
Heapsort CIS 606 Spring 2010. Overview Heapsort – O(n lg n) worst case—like merge sort. – Sorts in place—like insertion sort. – Combines the best of both.

Heapsort CIS 606 Spring Overview Heapsort – O(n lg n) worst case—like merge sort. – Sorts in place—like insertion sort. – Combines the best of both.
1 Binary Heaps What is a Binary Heap? Array representation of a Binary Heap MinHeap implementation Operations on Binary Heaps: enqueue dequeue deleting.

1 Binary Heaps What is a Binary Heap? Array representation of a Binary Heap MinHeap implementation Operations on Binary Heaps: enqueue dequeue deleting.

Similar presentations

Ads by Google