Computer Algorithms Lecture 9 Heapsort Ch. 6, App. B.5 Some of these slides are courtesy of D. Plaisted et al, UNC and M. Nicolescu, UNR.

Slides:



Advertisements
Similar presentations
BY Lecturer: Aisha Dawood. Heapsort  O(n log n) worst case like merge sort.  Sorts in place like insertion sort.  Combines the best of both algorithms.
Advertisements

Analysis of Algorithms
CS Section 600 CS Section 002 Dr. Angela Guercio Spring 2010.
David Luebke 1 5/20/2015 CS 332: Algorithms Quicksort.
September 19, Algorithms and Data Structures Lecture IV Simonas Šaltenis Nykredit Center for Database Research Aalborg University
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu.
CS 253: Algorithms Chapter 6 Heapsort Appendix B.5 Credit: Dr. George Bebis.
Analysis of Algorithms CS 477/677
Comp 122, Spring 2004 Heapsort. heapsort - 2 Lin / Devi Comp 122 Heapsort  Combines the better attributes of merge sort and insertion sort. »Like merge.
Data Structures, Spring 2004 © L. Joskowicz 1 Data Structures – LECTURE 7 Heapsort and priority queues Motivation Heaps Building and maintaining heaps.
Heapsort Chapter 6. Heaps A data structure with  Nearly complete binary tree  Heap property: A[parent(i)] ≥ A[i] eg. Parent(i) { return } Left(i) {
1 Priority Queues A priority queue is an ADT where: –Each element has an associated priority –Efficient extraction of the highest-priority element is supported.
3-Sorting-Intro-Heapsort1 Sorting Dan Barrish-Flood.
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu.
David Luebke 1 7/2/2015 Merge Sort Solving Recurrences The Master Theorem.
Heapsort CIS 606 Spring Overview Heapsort – O(n lg n) worst case—like merge sort. – Sorts in place—like insertion sort. – Combines the best of both.
Sorting Algorithms (Part II) Slightly modified definition of the sorting problem: input: A collection of n data items where data item a i has a key, k.
2IL50 Data Structures Spring 2015 Lecture 3: Heaps.
Ch. 6: Heapsort n nodes. Heap -- Nearly binary tree of these n nodes (not just leaves) Heap property If max-heap, the max-heap property is that for every.
David Luebke 1 10/3/2015 CS 332: Algorithms Solving Recurrences Continued The Master Theorem Introduction to heapsort.
Heaps, Heapsort, Priority Queues. Sorting So Far Heap: Data structure and associated algorithms, Not garbage collection context.
Binary Heap.
2IL50 Data Structures Fall 2015 Lecture 3: Heaps.
September 29, Algorithms and Data Structures Lecture V Simonas Šaltenis Aalborg University
Computer Sciences Department1. Sorting algorithm 3 Chapter 6 3Computer Sciences Department Sorting algorithm 1  insertion sort Sorting algorithm 2.
David Luebke 1 6/3/2016 CS 332: Algorithms Heapsort Priority Queues Quicksort.
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 9.
Data Structure & Algorithm Lecture 5 Heap Sort & Binary Tree JJCAO.
CS 2133: Data Structures Quicksort. Review: Heaps l A heap is a “complete” binary tree, usually represented as an array:
Introduction to Algorithms
1 Analysis of Algorithms Chapter - 03 Sorting Algorithms.
1 Algorithms CSCI 235, Fall 2015 Lecture 14 Analysis of Heap Sort.
David Luebke 1 12/23/2015 Heaps & Priority Queues.
S. Raskhodnikova and A. Smith. Based on slides by C. Leiserson and E. Demaine. 1 Adam Smith L ECTURES Priority Queues and Binary Heaps Algorithms.
Heapsort. What is a “heap”? Definitions of heap: 1.A large area of memory from which the programmer can allocate blocks as needed, and deallocate them.
ΔΟΜΕΣ ΔΕΔΟΜΕΝΩΝ & ΑΡΧΕΙΩΝ
CSC 413/513: Intro to Algorithms Solving Recurrences Continued The Master Theorem Introduction to heapsort.
Chapter 6: Heapsort Combines the good qualities of insertion sort (sort in place) and merge sort (speed) Based on a data structure called a “binary heap”
Lecture 8 : Priority Queue Bong-Soo Sohn Assistant Professor School of Computer Science and Engineering Chung-Ang University.
1 Heap Sort. A Heap is a Binary Tree Height of tree = longest path from root to leaf =  (lgn) A heap is a binary tree satisfying the heap condition:
David Luebke 1 2/5/2016 CS 332: Algorithms Introduction to heapsort.
Analysis of Algorithms CS 477/677 Lecture 8 Instructor: Monica Nicolescu.
COSC 3101A - Design and Analysis of Algorithms 3 Recurrences Master’s Method Heapsort and Priority Queue Many of the slides are taken from Monica Nicolescu’s.
CS6045: Advanced Algorithms Sorting Algorithms. Heap Data Structure A heap (nearly complete binary tree) can be stored as an array A –Root of tree is.
Heapsort A minimalist's approach Jeff Chastine. Heapsort Like M ERGE S ORT, it runs in O(n lg n) Unlike M ERGE S ORT, it sorts in place Based off of a.
ECOE 556: Algorithms and Computational Complexity Heapsort Serdar Taşıran.
Sept Heapsort What is a heap? Max-heap? Min-heap? Maintenance of Max-heaps -MaxHeapify -BuildMaxHeap Heapsort -Heapsort -Analysis Priority queues.
Heapsort Lecture 4 Asst. Prof. Dr. İlker Kocabaş.
6.Heapsort. Computer Theory Lab. Chapter 6P.2 Why sorting 1. Sometimes the need to sort information is inherent in a application. 2. Algorithms often.
1 Algorithms CSCI 235, Fall 2015 Lecture 13 Heap Sort.
Sorting Cont. Quick Sort As the name implies quicksort is the fastest known sorting algorithm in practice. Quick-sort is a randomized sorting algorithm.
"Teachers open the door, but you must enter by yourself. "
Heaps, Heapsort, and Priority Queues
Heaps, Heap Sort and Priority Queues
Heapsort Chapter 6 Lee, Hsiu-Hui
Heapsort.
Heap Sort Example Qamar Abbas.
Introduction to Algorithms
CS 583 Analysis of Algorithms
Heaps, Heapsort, and Priority Queues
CS200: Algorithm Analysis
Ch 6: Heapsort Ming-Te Chi
Heapsort.
Lecture 3 / 4 Algorithm Analysis
"Teachers open the door, but you must enter by yourself. "
Design and Analysis of Algorithms
HEAPS.
Heapsort Sorting in place
Solving Recurrences Continued The Master Theorem
Asst. Prof. Dr. İlker Kocabaş
Presentation transcript:

Computer Algorithms Lecture 9 Heapsort Ch. 6, App. B.5 Some of these slides are courtesy of D. Plaisted et al, UNC and M. Nicolescu, UNR

Heapsort Combines the better attributes of merge sort and insertion sort. –Like merge sort, but unlike insertion sort, running time is O(n lg n). –Like insertion sort, but unlike merge sort, sorts in place. Introduces an algorithm design technique –Create data structure (heap) to manage information during the execution of an algorithm. The heap has other applications beside sorting. –Priority Queues

Background on Trees Def: Binary tree = structure composed of a finite set of nodes that either: –Contains no nodes, or –Is composed of three disjoint sets of nodes: a root node, a left subtree and a right subtree root Right subtree Left subtree

Special Types of Trees Def: Full binary tree = a binary tree in which each node is either a leaf or has degree (number of children) exactly 2. Def: Complete binary tree = a binary tree in which all leaves have the same depth and all internal nodes have degree 2. Full binary tree Complete binary tree

Definitions Height of a node = the number of edges on a longest simple path from the node down to a leaf Depth of a node = the length of a path from the root to the node Height of tree = height of root node =  lgn , for a heap of n elements Height = 3 Depth of (10)= 2 Height of root = 3

Complete k-ary tree all leaves have the same depth all internal nodes have degree k degree k=2 – binary tree How many leaves does the k-ary tree of height h have? –The root has k children –Each of the root’s children has k children at depth 2 –Number of leaves/nodes at depth h is k h –The height of a complete k-ary tree with n leaves is log k n –The number of internal nodes of complete binary tree of height h k=2

Data Structure Binary Heap Def: A heap is a nearly complete binary tree with the following two properties: –Structural property: all levels are full, except possibly the last one, which is filled from left to right –Order (max-heap) property: for any node x Parent(x) ≥ x Heap It doesn’t matter that 4 in level 1 is smaller than 5 in level 2 2

Data Structure Binary Heap (2) –Root – A[ 1 ] –Left[i] – A[ 2i ] How can you do this operation using one instruction? –Right[i] – A[ 2i+1 ] How can you do this operation? –Parent[i] – A[  i/2  ] How can you do this operation? length[A] – number of elements in array A. heap-size[A] – number of elements in heap stored in A. –heap-size[A]  length[A] Array viewed as a nearly complete binary tree. –Physically – linear array. –Logically – binary tree, filled on all levels (except lowest.) Map from array elements to tree nodes and vice versa

Max-heap: example Max-heap as an array. Last row filled from left to right. Max-heap as a binary tree.

Heap Property (Max and Min) Max-Heap –For every node excluding the root, value is at most that of its parent: A[parent[i]]  A[i] Largest element is stored at the root. In any subtree, no values are larger than the value stored at subtree root. Min-Heap –For every node excluding the root, value is at least that of its parent: A[parent[i]]  A[i] Smallest element is stored at the root. In any subtree, no values are smaller than the value stored at subtree root

Operations on Heaps Maintain the max-heap property – MAX-HEAPIFY Create a max-heap from an unordered array – BUILD-MAX-HEAP Sort an array in place – HEAPSORT Priority queue operations

Height Height of a node in a tree: the number of edges on the longest simple downward path from the node to a leaf. Height of a tree: the height of the root. Height of a heap:  lg n  –Basic operations on a heap run in O(lg n) time No. of leaves =  n/2  No. of nodes of height h   n/2 h+1 

Maintaining the Heap Property Assume: –A node i is smaller than a child –Left and Right subtrees of i are max-heaps –The heap condition is violated only at that node To eliminate the violation: –Exchange with larger child –Move down the tree –Recursively fix the children until all of them satisfy the max-heap property.

MaxHeapify – Example MaxHeapify(A, 2)

Procedure MaxHeapify MaxHeapify(A, i) 1. l  left(i) 2. r  right(i) 3. if l  heap-size[A] and A[l] > A[i] 4. then largest  l 5. else largest  i 6. if r  heap-size[A] and A[r] > A[largest] 7. then largest  r 8. if largest  i 9. then exchange A[i]  A[largest] 10. MaxHeapify(A, largest) MaxHeapify(A, i) 1. l  left(i) 2. r  right(i) 3. if l  heap-size[A] and A[l] > A[i] 4. then largest  l 5. else largest  i 6. if r  heap-size[A] and A[r] > A[largest] 7. then largest  r 8. if largest  i 9. then exchange A[i]  A[largest] 10. MaxHeapify(A, largest) Assumption: Left(i) and Right(i) are max-heaps.

MAX-HEAPIFY Running Time Intuitively: –A heap is an almost complete binary tree  must process O(lgn) levels, with constant work at each level Running time of MAX-HEAPIFY is O(lgn) Can be written in terms of the height of the heap, as being O(h) –Since the height of the heap is  lgn 

Running Time for MaxHeapify MaxHeapify(A, i) 1. l  left(i) 2. r  right(i) 3. if l  heap-size[A] and A[l] > A[i] 4. then largest  l 5. else largest  i 6. if r  heap-size[A] and A[r] > A[largest] 7. then largest  r 8. if largest  i 9. then exchange A[i]  A[largest] 10. MaxHeapify(A, largest) MaxHeapify(A, i) 1. l  left(i) 2. r  right(i) 3. if l  heap-size[A] and A[l] > A[i] 4. then largest  l 5. else largest  i 6. if r  heap-size[A] and A[r] > A[largest] 7. then largest  r 8. if largest  i 9. then exchange A[i]  A[largest] 10. MaxHeapify(A, largest) Time to fix node i and its children =  (1) Time to fix the subtree rooted at one of i’s children = T(size of subree at largest) PLUS

Building a heap Use MaxHeapify to convert an array A into a max-heap. How? Call MaxHeapify on each element in a bottom-up manner. BuildMaxHeap(A) 1. heap-size[A]  length[A] 2. for i   length[A]/2  downto 1 3. do MaxHeapify(A, i) BuildMaxHeap(A) 1. heap-size[A]  length[A] 2. for i   length[A]/2  downto 1 3. do MaxHeapify(A, i) Why length[A]/2

BuildMaxHeap – Example Input Array: Initial Tree: (not max-heap)

BuildMaxHeap – Example MaxHeapify(  10/2  = 5) 36 MaxHeapify(4) MaxHeapify(3) MaxHeapify(2) MaxHeapify(1)

Correctness of BuildMaxHeap Loop Invariant: –At the start of each iteration of the for loop, each node i+1, i+2, …, n is the root of a max-heap. Initialization: –Before first iteration i =  n/2  –Nodes  n/2  +1,  n/2  +2, …, n are leaves and hence roots of max- heaps. Maintenance: –By LI, subtrees at children of node i are max heaps. –Hence, MaxHeapify(i) renders node i a max heap root (while preserving the max heap root property of higher-numbered nodes). –Decrementing i reestablishes the loop invariant for the next iteration.

Running Time of BuildMaxHeap Loose upper bound: –Cost of a MaxHeapify call  No. of calls to MaxHeapify –O(lg n)  O(n) = O(nlg n) Heapify costs O(n lg n) There are n such calls Tighter bound: –Cost of a call to MaxHeapify at a node depends on the height, h, of the node – O(h). –Height of most nodes smaller than n. –Height of nodes h ranges from 0 to  lg n . –No. of nodes of height h is  n/2 h+1 

Running Time of BuildMaxHeap Tighter Bound for T(BuildMaxHeap): Heapify time at a node depends on the node’s height Heights of many nodes are small In n-element heap there are at most nodes of height h T(BuildMaxHeap) Can build a heap from an unordered array in linear time

Heaps in Sorting Use max-heaps for sorting. The array representation of max-heap is not sorted. Steps in sorting –Convert the given array of size n to a max-heap (BuildMaxHeap) –Swap the first and last elements of the array. Now, the largest element is in the last position – where it belongs. That leaves n – 1 elements to be placed in their appropriate locations. However, the array of first n – 1 elements is no longer a max- heap. Float the element at the root down one of its subtrees so that the array remains a max-heap (MaxHeapify) Repeat step 2 until the array is sorted.

25 Heapsort Goal: –Sort an array using heap representations Idea: –Build a max-heap from the array –Swap the root (the maximum element) with the last element in the array –“Discard” this last node by decreasing the heap size –Call MAX-HEAPIFY on the new root –Repeat this process until only one node remains

26 Alg: HEAPSORT(A) HeapSort(A) 1. Build-Max-Heap(A) 2. for i  length[A] downto 2 3. do exchange A[1]  A[i] 4. heap-size[A]  heap-size[A] – 1 5. MaxHeapify(A, 1) HeapSort(A) 1. Build-Max-Heap(A) 2. for i  length[A] downto 2 3. do exchange A[1]  A[i] 4. heap-size[A]  heap-size[A] – 1 5. MaxHeapify(A, 1) O(n) O(lg n) n-1 times Running time: ?? O(n lg n)

27 Example: MAX-HEAPIFY(A, 1, 4) MAX-HEAPIFY(A, 1, 3) MAX-HEAPIFY(A, 1, 2) MAX-HEAPIFY(A, 1, 1) input A (maxHeapyfied) output A (sorted)

Heapsort Sort by maintaining the as yet unsorted elements as a max-heap. Start by building a max-heap on all elements in A. –Maximum element is in the root, A[1]. Move the maximum element to its correct final position. –Exchange A[1] with A[n]. Discard A[n] – it is now sorted. –Decrement heap-size[A]. Restore the max-heap property on A[1..n–1]. –Call MaxHeapify(A, 1). Repeat until heap-size[A] is reduced to 2.

Algorithm Analysis In-place Not Stable –A sorting algorithm is stable, if it leaves the order of equal elements unchanged. Build-Max-Heap takes O(n) and each of the n-1 calls to Max-Heapify takes time O(lg n). Therefore, T(n) = O(n lg n) HeapSort(A) 1. Build-Max-Heap(A) 2. for i  length[A] downto 2 3. do exchange A[1]  A[i] 4. heap-size[A]  heap-size[A] – 1 5. MaxHeapify(A, 1) HeapSort(A) 1. Build-Max-Heap(A) 2. for i  length[A] downto 2 3. do exchange A[1]  A[i] 4. heap-size[A]  heap-size[A] – 1 5. MaxHeapify(A, 1)

Heap Procedures for Sorting MaxHeapify O(lg n) BuildMaxHeap O(n) HeapSort O(n lg n)

Priority Queue Popular & important application of heaps. Max and min priority queues. –Maintains a dynamic set S of elements. –Each set element has a key – an associated value. –Goal is to support insertion and extraction efficiently. Applications: Ready list of processes in operating systems by their priorities – the list is highly dynamic In event-driven simulators to maintain the list of events to be simulated in order of their time of occurrence.

Basic Operations Operations on a max-priority queue: –Insert(S, x) - inserts the element x into the set S S  S  {x}. –Maximum(S) - returns the element of S with the largest key. –Extract-Max(S) - removes and returns the element of S with the largest key. –Increase-Key(S, x, k) – increases the value of element x’s key to the new value k. Min-priority queue supports Insert, Minimum, Extract-Min, and Decrease-Key. Heap gives a good compromise between fast insertion but slow extraction and vice versa.

Heap Property (Max and Min) Max-Heap –For every node excluding the root, value is at most that of its parent: A[parent[i]]  A[i] Largest element is stored at the root. In any subtree, no values are larger than the value stored at subtree root. Min-Heap –For every node excluding the root, value is at least that of its parent: A[parent[i]]  A[i] Smallest element is stored at the root. In any subtree, no values are smaller than the value stored at subtree root

HEAP-MAXIMUM Goal: –Return the largest element of the heap Alg: HEAP-MAXIMUM(A) 1.return A[1] Running time: O(1) Heap A: Heap-Maximum(A) returns 7

HEAP-EXTRACT-MAX Goal: –Extract the largest element of the heap (i.e., return the max value and also remove that element from the heap Idea: –Exchange the root element with the last –Decrease the size of the heap by 1 element –Call MAX-HEAPIFY on the new root, on a heap of size n-1 Heap A: Root is the largest element

HEAP-EXTRACT-MAX Alg: HEAP-EXTRACT-MAX(A, n) 1. if n < 1 2. then error “heap underflow” 3. max ← A[1] 4. A[1] ← A[n] 5. MAX-HEAPIFY(A, 1, n-1) remakes heap 6. return max Running time: O(lg n)

Example: HEAP-EXTRACT-MAX max = Heap size decreased with Call MAX-HEAPIFY (A, 1, n-1)

Heap-Extract-Max(A) 1. if heap-size[A] < 1 2. then error “heap underflow” 3. max  A[1] 4. A[1]  A[heap-size[A]] 5. heap-size[A]  heap-size[A] MaxHeapify(A, 1) 7. return max Heap-Extract-Max(A) 1. if heap-size[A] < 1 2. then error “heap underflow” 3. max  A[1] 4. A[1]  A[heap-size[A]] 5. heap-size[A]  heap-size[A] MaxHeapify(A, 1) 7. return max Running time : Dominated by the running time of MaxHeapify = O(lg n) Implements the Extract-Max operation.

Heap-Insert(A, key) 1. heap-size[A]  heap-size[A] i  heap-size[A] 4. while i > 1 and A[Parent(i)] < key 5. do A[i]  A[Parent(i)] 6. i  Parent(i) 7. A[i]  key Heap-Insert(A, key) 1. heap-size[A]  heap-size[A] i  heap-size[A] 4. while i > 1 and A[Parent(i)] < key 5. do A[i]  A[Parent(i)] 6. i  Parent(i) 7. A[i]  key Running time is O(lg n) The path traced from the new leaf to the root has length O(lg n)

Heap-Increase-Key(A, i, key) 1 If key < A[i] 2 then error “new key is smaller than the current key” 3 A[i]  key 4 while i > 1 and A[Parent[i]] < A[i] 5 do exchange A[i]  A[Parent[i]] 6 i  Parent[i] Heap-Increase-Key(A, i, key) 1 If key < A[i] 2 then error “new key is smaller than the current key” 3 A[i]  key 4 while i > 1 and A[Parent[i]] < A[i] 5 do exchange A[i]  A[Parent[i]] 6 i  Parent[i] Heap-Insert(A, key) 1 heap-size[A]  heap-size[A] A[heap-size[A]]  –  3 Heap-Increase-Key(A, heap-size[A], key) Heap-Insert(A, key) 1 heap-size[A]  heap-size[A] A[heap-size[A]]  –  3 Heap-Increase-Key(A, heap-size[A], key)