Chapter 4: Divide and Conquer Master Theorem, Mergesort, Quicksort, Binary Search, Binary Trees The Design and Analysis of Algorithms

2 Chapter 4. Chapter 4. Divide and Conquer Algorithms Basic Idea Merge Sort Quick Sort Binary Search Binary Tree Algorithms Closest Pair and Quickhull Conclusion

3 Basic Idea Divide instance of problem into two or more smaller instances Solve smaller instances recursively Obtain solution to original (larger) instance by combining these solutions

4 Basic Idea subproblem 2 of size n/2 subproblem 1 of size n/2 a solution to subproblem 1 a solution to the original problem a solution to subproblem 2 a problem of size n

5 General Divide-and-Conquer Recurrence T(n) = aT(n/b) + f (n) where f(n)   (n d ), d  0 Master Theorem: If a < b d, T(n)   (n d ) If a = b d, T(n)   (n d log n) If a > b d, T(n)   (n log b a ) Examples: T(n) = T(n/2) + n  T(n)   (n ) Here a = 1, b = 2, d = 1, a < b d T(n) = 2T(n/2) + 1  T(n)   (n log 2 2 ) =  (n ) Here a = 2, b = 2, d = 0, a > b d

6 Examples T(n) = T(n/2) + 1  T(n)   (log(n) ) Here a = 1, b = 2, d = 0, a = b d T(n) = 4T(n/2) + n  T(n)   (n log 2 4 ) =  (n 2 ) Here a = 4, b = 2, d = 1, a > b d T(n) = 4T(n/2) + n 2  T(n)   (n 2 log n) Here a = 4, b = 2, d = 2, a = b d T(n) = 4T(n/2) + n 3  T(n)   (n 3 ) Here a = 4, b = 2, d = 3, a < b d

Recurrence Examples

8 Recursion Tree for T(n)=3T(  n/4  )+  (n 2 ) T(n)T(n) (a) cn 2 T(n/4) (b) cn 2 c(n/4) 2 T(n/16) (c) cn 2 c(n/4) 2 c(n/16) 2 cn 2 (3/16)cn 2 (3/16) 2 cn 2 T(1)  (n log 4 3 ) 3 log4 n = n log 4 3 Total O(n 2 ) log 4 n (d)

9 Solution to T(n)=3T(  n/4  )+  (n 2 ) The height is log 4 n, #leaf nodes = 3 log 4 n = n log 4 3. Leaf node cost: T(1). Total cost T(n)=cn 2 +(3/16) cn 2 +(3/16) 2 cn 2 +  +(3/16) log 4 n -1 cn 2 +  (n log 4 3 ) =( 1+3/16+(3/16) 2 +  +(3/16) log 4 n -1 ) cn 2 +  (n log 4 3 ) <( 1+3/16+(3/16) 2 +  +(3/16) m +  ) cn 2 +  (n log 4 3 ) =(1/(1-3/16)) cn 2 +  (n log 4 3 ) =16/13cn 2 +  (n log 4 3 ) =O(n 2 ).

10 Recursion Tree of T(n)=T(n/3)+ T(2n/3)+O(n)

11 Mergesort Split array A[0..n-1] in two about equal halves and make copies of each half in arrays B and C Sort arrays B and C recursively Merge sorted arrays B and C into array A

12 Mergesort

13 Mergesort

14 Mergesort

Mergesort Code mergesort( int a[], int left, int right) { if (right > left) { middle = left + (right - left)/2; mergesort(a, left, middle); mergesort(a, middle+1, right); merge(a, left, middle, right); } 15

16 Analysis of Mergesort T(N) = 2T(N/2) + N All cases have same efficiency: Θ(n log n) Space requirement: Θ(n) (not in-place) Can be implemented without recursion (bottom-up)

Features of Mergesort All cases: Θ(n log n) Requires additional memory Θ(n) Recursive Not stable (does not preserve previous sorting) Never used for sorting in main memory, used for external sorting 17

Quicksort Let S be the set of N elements Quicksort(S): If N = 0 or N = 1, return Pick an element v - pivot, in S Partition S - {v} into two disjoint sets: S1 = {x є S - {v}| x ≤ v}, and S2 = {x є S - {v}| x ≥ v} Return {quicksort(S1), followed by v, followed by quicksort(S2)} 18

19 Quicksort Select a pivot (partitioning element) Median of three algorithm Take the first, the last and the middle elements and sort them (within their positions). Choose the median of these three elements. Hide the pivot – swap the pivot and the next to the last element. Example: After sorting Hide the pivot

Quicksort Rearrange the list so that all the elements in the first s positions are smaller than or equal to the pivot and all the elements in the remaining n-s-1 positions are larger than or equal to the pivot. Note: the pivot does not participate in rearranging the elements. Exchange the pivot with the first element in the second subarray — the pivot is now in its final position Sort the two subarrays recursively 20

left – the smallest valid index right – the largest valid index if( left + 10 <= right) { int i = left, j = right - 1; for ( ; ; ) { while (a[++i] < pivot) {} while (pivot < a[--j] ) {} if (i < j) swap (a[i],a[j]); else break; } swap (a[i], a[right-1]); quicksort ( a, left, i-1); quicksort (a, i+1, right); } else insertionsort (a, left, right); 21

22 Analysis of Quicksort Best case: split in the middle — Θ(n log n) T(N) = 2T(N/2) + N Worst case: sorted array — Θ(n 2 ) T(N) = T(N-1) + N Average case: random arrays — Θ(n log n) Quicksort is considered the method of choice for internal sorting of large files (n ≥ 10000)

Features of Quicksort Best and average case: Θ(n log n), Worst case: Θ(n 2 ), it happens when the pivot is the smallest of the largest element In-place sorting, additional memory only for swapping Recursive Not stable Never used for life-critical and mission- critical applications, unless you assume the worst-case response time 23

24 Binary Search Very efficient algorithm for searching in sorted array: K vs A[0]... A[m]... A[n-1] If K = A[m], stop (successful search); otherwise, continue searching by the same method in A[0..m-1] if K < A[m], and in A[m+1..n-1] if K > A[m]

25 Analysis of Binary Search Time efficiency T(N) = T(N/2) + 1  Worst case: O(log(n))  Best case: O(1) Optimal for searching a sorted array Limitations: must be a sorted array (not linked list)

26 Binary Tree Algorithms Binary tree is a divide-and-conquer ready structure Ex. 1: Classic traversals (preorder, inorder, postorder) Algorithm Inorder(T) if T   Inorder(Tleft) print(root of T) Inorder(Tright) Efficiency: Θ(n)

27 Binary Tree Algorithms Ex. 2: Computing the height of a binary tree h(T) = max{ h(TL), h(TR) } + 1 if T   and h(  ) = -1 Efficiency: Θ(n)

28 Recursion tree for T(n)=aT(n/b)+f(n)