CS200: Algorithm Analysis

HEAPSORT General characteristics : Sorts in place. θ(nlgn) best, average, and worst case run-time. Uses a data structure called a HEAP (array). HEAP: an array, A, that is organized as a complete binary tree. (Complete-> leaves differ by no more than one level and levels are filled left to right). A Heap of n elements has a height of θ(log n).

The HEAP is completely filled except for possibly the lowest level The HEAP is completely filled except for possibly the lowest level. The root is at A[1] and given the index i of a node, its Parent(i) can be computed as: Root of tree at position i = 1 Parent(i) = ? return i / 2 {floor(i/2)} and its LeftChild(i) as: Left(i) = ? return 2i; and its RightChild(i) as: Right(i) = ? return 2i + 1;

A MAX HEAP must always satisfy the following property for all nodes i in the array. Min Heap defined analogously. A[Parent(i)] >= A[i] => largest value in HEAP is stored in root and the value stored in the root of each subtree is >= the value in all descended nodes. The height of a HEAP is the length of the longest simple path from the root to a leaf. (Show for example) => log n depth. The height of a node is the length of the longest simple path from the node to a leaf. (Show an example). Height of 2? Height of 14? Height of 16?

Heap Operations build_max_heap : produce a max- heap from an unordered array max_heapify : correct a single violation of the heap property in a subtree at its root insert, extract_max, heapsort

ALGORITHMS FOR HEAP 1. Heap Operation : Extracting the maximum element from a HEAP. ExtractMax(A) //n is size of heap, A is array temp = A[1] A[1] = A[n] n = n-1; Max_Heapify(A,1,n) return temp

The Heap is out of order after ExtractMax, and is reordered via the Max_Heapify algorithm. Run-time is O(1) + Max_Heapify time (discussed later). Extraction of elements is in reverse sorted order. Do example without Max_Heapify.

2. Heap Operation: to maintain HEAP property (elements float down to proper position in the HEAP). Max_Heapify(A, i, n) //position i, where reorder starts if (2i <= n) and (A[2i] > A[i]) then largest = 2i // 2i in bounds, left child largest else largest = i //current root largest if (2i+1 <= n) and (A[2i+1] > A[largest]) then largest = 2i+1 //2i+1 in bounds,right //child largest if largest != i then Swap(A[i],A[largest]); Max_Heapify(A,largest,n); Trace an example for algorithm next.

Trace Max_Heapify Run-time is intuitively = ? Why?

Build_Heap Operation : building a heap by converting an array. BuildHeap(A,n) for i = n / 2 downto 1 do Max_Heapify(A,i,n) Trace an example for algorithm, next slide set. Run-time is intuitively = ______________? Why? Is this a tight upper bound? Is it possible to do better?

Converts A[1...n] to a max heap BuildHeap(A,n) for i = n / 2 downto 1 do Max_Heapify(A,i,n) Why start at n/2? Because elements A[n/2 + 1 ... n] are all leaves of the tree 2i > n, for i > n/2 + 1. A single node is a max_heap! Time? O(n log n) via simple analysis

Build_Heap Analysis Converts A[1...n] to a max heap BuildHeap(A,n) for i = n / 2 downto 1 do Max_Heapify(A,i,n) Observe that Max_Heapify takes O(1) time for nodes that are one level above the leaves, and in general, O(L) for the nodes that are L levels above the leaves. We have n/4 nodes with level 1, n/8 with level 2, and so on till we have one root node that is log n levels above the leaves.

Build_Heap Analysis Converts A[1...n] to a max heap BuildHeap(A,n) for i = n / 2 downto 1 do Max_Heapify(A,i,n) Total amount of work in the for loop can be summed as: n/4 (1 c) + n/8 (2 c) + n/16 (3 c) + ... + 1 (log n c) Setting n/4 = 2k and simplifying we get: c 2k( 1/20 + 2/21 + 3/22 + ... (k+1)/2k ) The term in () is bounded by a constant! This means that Build_Max_Heap is O(n) . Complete analysis on next slide.

Time to run MAX-HEAPIFY is linear in the height of the node it’s run on, and most nodes have small heights. There are ≤ n / 2h+1 nodes of height h, and height of heap is log n. The time required by MAX-HEAPIFY when called on a node of height h is O(h), so the total cost of BUILD-MAX-HEAP is Evaluate the last summation by substituting x = 1/2 in the formula (A.8) The running time of BUILD-MAX-HEAP is O(n).

4. HEAPSORT HeapSort(A,n) BuildHeap(A,n) O(n) for i = n down to 2 do n-1 times swap(A[1], A[i] O(1) Max_Heapify(A,1, i-1) O(lgn) T(n) = O(nlgn) Trace an example of algorithm.

Variations on HEAPS: – min or max HEAP – HEAP maintained as a k-ary tree vs. binary tree where each interior node has k children and heap property is maintained for each subtree.

DYNAMIC SETS Set of elements that can grow, shrink with needs of an algorithm. Elements of a dynamic set contain a KEY field (for identification) and a satellite data field (unused in the set operations). An element can be viewed as a record with 2 fields: key and data. For the following operations assume that X is a pointer to an element record and that the record fields are already initialized.

DYNAMIC SET OPERATIONS Insert(S,x) S <– S U {x} Delete(S,x) S <– S - {x} Minimum(S) return x in S with smallest key field Search(S,k) return x in S with x->key = k The main issue is the data structure chosen to represent Dynamic Sets; different data structures will support and optimize different operations in the above list.

USING PRIORITY QUEUES AS DYNAMIC SETS A priority queue is a data structure for maintaining a dynamic set, S, of elements, each with an associated key value with ordering on the key. It supports: Insert(S,x) Minimum(S) ExtractMin(S) DecreaseKey(S,x,k) - discussed later for graph algorithms By examining priority queues it is obvious that this structure supports Insert and ExtractMin (Minimum set operation) but does not support the general delete operation. Ex: Max priority queue - process scheduler Min priority queue- event simulator by time of event

1. Simplest implementation of priority queue (ordered)? Runtime for ExtractMin = ________________? Runtime for Insert = _______________? 2. Simplest implementation of priority queue (unordered)? Actual implementation uses an array of pointers (indices) into the heap. The heap maintains backward pointers to the array of indices.

A more interesting implementation of a priority queue is via a HEAP. A min-HEAP with minimum maintained at root provides an excellent representation of a priority queue. This provides a good compromise between insertion and extraction because both can be done in ___________? time. Runtime of Minimum(S) _______? Runtime of ExtractMin(S) ______? How does this work? How do we implement a general insert into a priority queue as a heap? What is its runtime?

Insertion into a Max_Heap as a Priority Queue Insert(S, x) n = n+1 i = n while(i > 1) and (S[i / 2] > x) do S[i] = S[i/2] i = i / 2 A[i] = x T(n) = _____________?

Insert of key element is like insertion sort as elements are moved down tree to open up the right spot for insertion. In worst case Insert traverses log n nodes as does heapify, but it compares parent only to 1 child not to both as in Heapify.

Summary Heap Heap algorithms Heap analysis HeapSort Priority Queues Heap as a priority queue