Algorithms: Design and Analysis 240-310, Semester 2, 2018-2019 3. Divide and Conquer Objective look at several divide and conquer examples (merge sort, quick sort)

1. Divide-and-Conquer Divide problem into two or more smaller instances Solve smaller instances recursively (conquer) Obtain solution to original (larger) problem by combining these solutions

Divide-and-Conquer Technique solve a problem of size n divide problem into smaller parts (often 2) a recursive algorithm solve subproblem 1 of size n/2 solve subproblem 2 of size n/2 a solution to subproblem 1 a solution to subproblem 2 combine solutions a solution to the original problem

2. A Faster Sort: Merge Sort Initial call: MergeSort(A, 1, n) MERGESORT( A, left, right) If left < right, // if left ≥ right, do nothing mid := floor(left+right)/2) MergeSort(A, left, mid) MergeSort( A, mid+1,right) Merge(A, left, mid, right) return 1 2 ... mid n

A faster sort: MergeSort input A[1 .. n] A[1 . . mid] A[mid+1 . . n] MERGESORT MERGESORT Sorted A[1 . . mid] Sorted A[mid+1 . . n] MERGE output

Tracing MergeSort() merge

Merging two sorted arrays 20 13 7 2 12 11 9 1 Time = one pass through each array = O(n) to merge a total of n elements (linear time).

Analysis of Merge Sort Statement Effort MergeSort(A, left, right) T(n) if (left < right) { O(1) mid = floor((left+right)/2); O(1) MergeSort(A, left, mid); T(n/2) MergeSort(A, mid+1, right); T(n/2) Merge(A, left, mid, right); O(n) } As shown on the previous slides

merge() Code merge(A, left, mid, right) Merges two adjacent subranges of an array A left == the index of the first element of the first range mid == the index of the last element of the first range right == to the index of the last element of the second range

with mergesort : double memory void merge(int[] A, int left, int mid, int right) { int[] temp = new int[right–left + 1]; int aIdx = left; int bIdx = mid+1; for (int i=0; i < temp.length; i++){ if(aIdx > mid) temp[i] = A[bIdx++]; // copy 2nd range else if (bIdx > right) temp[i] = A[aIdx++]; // copy 1st range else if (a[aIdx] <= a[bIdx]) temp[i] = A[aIdx++]; else temp[i] = A[bIdx++]; } // copy back into A for (int j = 0; j < temp.length; j++) A[left+j] = temp[j]; } the problem with mergesort : double memory

MergeSort Running Time Recursive T() equation: T(1) = O(1) T(n) = 2T(n/2) + O(n), for n > 1 Convert to algebra T(1) = a T(n) = 2T(n/2) + cn

T(n) = 2T(n/2) + cn 2(2T(n/2/2) + cn/2) + cn 22T(n/22) + cn2/2 + cn 22T(n/22) + cn(2/2 + 1) 22(2T(n/22/b) + cn/22) + cn(2/2 + 1) 23T(n/23) + cn(22/22) + cn(2/2 + 1) 23T(n/23) + cn(22/22 +2/2 + 1) … 2kT(n/2k) + cn(2k-1/2k-1 + 2k-2/2k-2 + … + 22/22 + 2/2 + 1)

So we have T(n) = 2kT(n/2k) + cn(2k-1/2k-1 + ... + 22/22 + 2/2 + 1) For k = log2 n n = 2k, so T() argument becomes 1 T(n) = 2kT(1) + cn(k-1+1) = na + cn(log2 n) = O(n) + O(n log2 n) = O(n log2 n) k-1 of these

Merge Sort vs Selection Sort • O(n log n) grows more slowly than O(n2). • In other words, merge sort is asymptotically faster (runs faster) than insertion sort in the worst case. • In practice, merge sort beats selection sort for n > 30 or so.

3. Quicksort Proposed by Tony Hoare in 1962. Voted one of top 10 algorithms of 20th century in science and engineering http://www.siam.org/pdf/news/637.pdf Sorts “in place” -- rearranges elements using only the array, as in insertion sort, but unlike merge sort which uses extra storage. Very practical (after some code tuning).

Divide and conquer Quicksort an n-element array: 1. Divide: Partition the array into two subarrays around a pivot x such that elements in lower subarray ≤ x ≤ elements in upper subarray. 2. Conquer: Recursively sort the two subarrays. 3. Combine: Nothing to do. Key: implementing a linear-time partitioning function

Pseudocode quicksort(int[] A, int left, int right) if (left < right) // If the array has 2 or more items pivot = partition(A, left, right) // recursively sort elements smaller than the pivot quicksort(A, left, pivot-1) // recursively sort elements bigger than the pivot quicksort(A, pivot+1, right)

Quicksort Diagram pivot

3.1. Partitioning Function PARTITION(A, p, q) // A[p . . q] x ← A[p] // pivot = A[p] Running time i ← p // index = O(n) for n for j ← p + 1 to q elements. if A[ j] ≤ x then i ← i + 1 // move the i boundary exchange A[i] ↔ A[ j] // switch big and small exchange A[p] ↔ A[i] return i // return index of pivot

Example of partitioning scan right until find something less than the pivot

Example of partitioning

Example of partitioning

Example of partitioning swap 10 and 5

Example of partitioning resume scan right until find something less than the pivot

Example of partitioning

Example of partitioning

Example of partitioning swap 13 and 3

Example of partitioning swap 10 and 2

Example of partitioning

Example of partitioning j runs to the end

Example of partitioning swap pivot and 2 so in the middle

3.2. Analysis of Quicksort The analysis is quite tricky. Assume all the input elements are distinct no duplicate values makes this code faster! there are better partitioning algorithms when duplicate input elements exist (e.g. Hoare's original code) Let T(n) = worst-case running time on an array of n elements.

Worst-case of quicksort QUICKSORT runs very slowly when its input array is already sorted (or is reverse sorted). almost sorted data is quite common in the real-world This is caused by the partition using the min (or max) element which means that one side of the partition will have has no elements. Therefore: T(n) = T(0) +T(n-1) + O(n) = O(1) +T(n-1) + O(n) = T(n-1) + O(n) = O(n2) no elements n-1 elements

Quicksort isn't Quick? In the worst case, quicksort isn't any quicker than selection sort. So why bother with quicksort? It's average case running time is very good, as we'll see.

Best-case Analysis If we’re lucky, PARTITION splits the array evenly: T(n) = 2T(n/2) + O(n) = O(n log n) (same as merge sort)

Good and Bad Suppose we alternate good, bad, good, bad, good, partitions …. G(n) = 2B(n/2) + O(n) good B(n) = G(n – 1) + O(n) bad Solving: G(n) = 2( G(n/2 – 1) + O(n/2) ) + O(n) = 2G(n/2 – 1) + O(n) = O(n log n) How can we make sure we choose good partitions? Good!

Randomized Quicksort IDEA: Partition around a random element. Running time is then independent of the input order. No assumptions need to be made about the input distribution. No specific input leads to the worst-case behavior. The worst case is determined only by the output of a random-number generator.

3.3. Quicksort in Practice Quicksort is a great general-purpose sorting algorithm. especially with a randomized pivot Quicksort can benefit substantially from code tuning Quicksort can be over twice as fast as merge sort Quicksort behaves well even with caching and virtual memory.

4. Timing Comparisons Running time estimates: Home PC executes 108 compares/second. Supercomputer executes 1012 compares/second selection Lesson 1. Good algorithms are better than supercomputers. Lesson 2. Great algorithms are better than good ones.