HeapSort 25 March 2003

2 HeapSort Heaps or priority queues provide a means of sorting: 1.Construct a heap, 2.Add each item to it (maintaining the heap property!), 3.When all items have been added, remove them one by one (restoring the heap property as each one is removed). Addition and deletion are both O(logn) operations. We need to perform n additions and deletions, leading to an O(nlogn) algorithm.

3 Binary Trees A binary tree is completely full if it is of height, h, and has 2 h+1 -1 nodes. A binary tree of height, h, is complete iff 1.it is empty or 2.its left subtree is complete of height h-1 and its right subtree is completely full of height h-2 or 3.its left subtree is completely full of height h-1 and its right subtree is complete of height h-1. A complete tree is filled from the left: 1.all the leaves are on the same level or 2.two adjacent ones and 3.All nodes at the lowest level are as far to the left as possible

4 Heaps (Priority Queues) A binary tree has the heap property iff 1.it is empty or 2.the key in the root is larger than that in either child and (recursively) both subtrees have the heap property. The value of the heap structure is that we can both extract the highest item and insert a new one in O(logn) time.

5 Remove Root Start with a heap. A deletion will remove the T at the root.

6 Bring Item to Root To maintain the heap property, use the fact that a complete tree is filled from the left. So that the position which must become empty is the one occupied by the “M”. Put it in the vacant root position.

7 Check Heap Property M at the root violates the condition that the root must be greater than each of its children. So interchange the M with the larger of its children.

8 Recheck Heap Property The left subtree has now lost the heap property. So again interchange the M with the larger of its children. This tree is now a heap again, so we're finished.

9 Complexity We need to make at most h interchanges of a root of a subtree with one of its children to fully restore the heap property. Thus deletion from a heap is O(h) or O(logn).

10 Add Item to Heap To add an item to a heap, we follow the reverse procedure. Place it in the next leaf position and move it up. Again, we require at most O(h) or O(logn) exchanges.

11 Storing Heaps in an Array If we number the nodes from 1 at the root and place: 1.the left child of node k at position 2k 2.the right child of node k at position 2k+1 Then the ‘fill from the left’ nature of the complete tree ensures that the heap can be stored in consecutive locations in an array.

12 2k and 2k+1 Are Children of k k+1 2k N = 9

13 HeapSort Animation heapsort.html

14 Empirical Studies Empirical studies show that generally QuickSort is considerably faster than HeapSort. The following counts of compare and exchange operations were made for three different sorting algorithms running on the same data:

15 QuickSort is Method of Choice Thus, when an occasional “blowout” to O(n 2 ) is tolerable, we can expect that, on average, QuickSort provides considerably better performance - especially if one of the modified pivot choice procedures is used. Most commercial applications would use QuickSort for its better average performance: they can tolerate an occasional long run in return for shorter runs most of the time.

16 Nothing is Guaranteed QuickSort should never be used in applications which require a guarantee of response time, unless it is treated as an O(n 2 ) algorithm in calculating the worst-case response time. If you have to assume O(n 2 ) time, then - if n is small, you're better off using insertion sort - which has simpler code and therefore smaller constant factors.

17 HeapSort is an Alternative If n is large, use heap sort, for it is guaranteed O(nlog n) time. Life-critical (medical monitoring, life support) and mission-critical (monitoring and control in plants handling dangerous materials, control for aircraft, defense, etc.) software will generally have response time as part of the system specifications. In such systems, it is not acceptable to design based on average performance, you must allow for the worst case, and thus treat QuickSort as O(n 2 ).

18 Priority Queue A priority queue is a collection of zero or more elements. Each element has a priority or value. – Find an element – Insert a new element – Remove an element min priority queue max priority queue

19 Example of a Priority Queue Operating system is handing out CPU time slices to multiple users of the system Certain tasks (especially those involving I/O) have to be serviced before the data is lost Order requests for service by priority Use a priority queue to decide which request receives service next

20 Speeding up QuickSort The recursive calls in QuickSort are generally expensive on most architectures - the overhead of any procedure call is significant and reasonable improvements can be obtained with equivalent iterative algorithms. Two things can be done to “eke” a little more performance out of your processor when sorting. (See next slide)

21 Fine Tuning 1.QuickSort- in it's usual recursive form - has a reasonably high constant factor relative to a simpler sort such as insertion sort. Thus, when the partitions become small (n < ~10), a switch to insertion sort for the small partition will usually show a measurable speed-up. (The point at which it becomes effective to switch to the insertion sort is extremely sensitive to architectural features and needs to be determined for any target processor: although a value of ~10 is a reasonable guess!) 2.Write the whole algorithm in an iterative form.

22 Sorting Summary BubbleO(n 2 )No additional space Loop check for early exit HeapO(nlgn)No additional space InsertionO(n 2 )No additional space Loop check for early exit MergeO(nlgn)Duplicate array Stack for recursion Can be used with linked lists QuickO(nlgn) on average, O(n 2 ) worst Stack for recursion SelectionO(n 2 )No additional space O(n) exchanges ShellBetween O(n(lgn) 2 ) and O(n 1.5 ) No additional space