Sorting Algorithms and Average Case Time Complexity Simple Sorts O(n2) Insertion Sort Selection Sort Bubble Sort More Complex Sorts O(nlgn) Heap Sort Quick Sort Merge Sort

Heap Sort Remember the heap data structure . . . Binary heap = a binary tree that is Complete satisfies the heap property

Heap Sort (cont’d) Given an array of integers: First, use MakeHeap to convert the array into a max-heap. Then, repeat the following steps until there are no more unsorted elements: Take the root (maximum) element off the heap by swapping it into its correct place in the array at the end of the unsorted elements Heapify the remaining unsorted elements (This puts the next-largest element into the root position)

Heap Sort (cont’d) HeapSort(A, n) MakeHeap(A) for i = n downto 2 exchange A[1] with A[i] n = n-1 Heapify(A,1)

Heapsort Example 2 8 6 1 10 15 12 11

Heap Sort (cont’d) Time complexity? First, MakeHeap(A) Then: Easy analysis: O(nlgn) since Heapify is O(lgn) More exact analysis: O(n) Then: n-1 swaps = O(n) n-1 calls to Heapify = O(nlgn)  O(n) + O(n) + O(nlgn) = O(nlgn)

Other Sorting Ideas Divide and Conquer!

Quicksort Recursive in nature. First Partition the array, and recursively quicksort the subarray separately until the whole array is sorted This process of partitioning is carried down until there are only one-cell arrays that do not need to be sorted at all

Quicksort Algorithm Given an array of integers: If array only contains one element, return Else Pick one element to use as the pivot Partition elements into 2 subarrays: Elements less than or equal to the pivot Elements greater than or equal to the pivot Quicksort the two subarrays

Quicksort Example

Quicksort Example There are a number of ways to pick the pivot element. Here, we will use the first element in the array

Quicksort Partition Given a pivot, partition the elements of the array such that the resulting array consists of: One sub-array that contains elements <= pivot Another sub-array that contains elements >= pivot The sub-arrays are stored in the original array Partitioning loops through, swapping elements below/above pivot

Quicksort Example: Partition Result:

QuickSort template<class T> void quicksort(T data[], int first, int last) { //partition such that left <= pivot, right >= pivot int lower = first+1, upper = last; swap(data[first],data[(first+last)/2]); T pivot = data[first]; while (lower <= upper) { //find the position of the first > pivot element on the left while (data[lower] < pivot) lower++; //find the position of the first < pivot element on the right while (pivot < data[upper]) upper--; if (lower < upper) swap(data[lower++],data[upper--]); else lower++; } swap(data[upper],data[first]); if (first < upper-1) quicksort (data,first,upper-1); //quicksort left if (upper+1 < last) quicksort (data,upper+1,last); //quicksort right

Quicksort - Recursion Recursion: Quicksort sub-arrays

Quicksort: Worst Case Complexity Worst case running time? Assume first element is chosen as pivot. Assume array is already in order: < – Recursion: 1. Partition splits array in two sub-arrays: • one sub-array of size 0 • the other sub-array of size n-1 2. Quicksort each sub-array – Depth of recursion tree? O(n) – Number of accesses per partition? O(n)

Quicksort: Best Case Complexity Assume that keys are random, uniformly distributed. Best case running time: O(n log2n)

Merge Sort Quicksort : worst case complexity in the worst case is O(n2) because it is difficult to control the partitioning process The partitioning depends on the choice of pivot, no guarantee that partitioning will result in arrays of approximately equal size Another strategy is to make partitioning as simple as possible and concentrate on merging sorted (sub)arrays

Merge Sort Also recursive in nature Given an array of integers, apply the divide-and-conquer paradigm Divide the n-element array into two subarrays of n/2 elements each Sort the two subarrays recursively using Merge Sort Merge the two subarrays to produce the sorted array The recursion stops when the array to be sorted has length 1

Merge Sort Merge Sort Divide array into two halves Recursively sort each half Merge two halves to make sorted whole

Merging Merging: Combine two pre-sorted lists into a sorted whole How to merge efficiently? Use temporary array Need index 1 (for array 1), index 2 (for array 2), 3 (for array tmp) 1 3 19 25 5 7 8 9 29 1 3 5 7

Merging Merge(A, left, middle, right) Create temporary array temp, same size as A i1 = left; i2 = middle; i3 = 0; while ((i1 != middle) && (i2 != right)) if (A[i1] < A[i2]) temp[i3++] = A[i1++] else temp[i3++] = A[i2++] copy into temp remaining elements of A copy into A the contents of temp

Merging Time Complexity Analysis T(n) = c1 if n =1 T(n) = 2 T(n/2) + c2n divide O(1) sort 2T(n/2) merge O(n)

Merge Sort MergeSort(A, left, right) if (left < right) { middle = (left + right)/2 MergeSort(A, left, middle) MergeSort(A, middle+1, right) Merge(A, left, middle+1, right) }