Algorithms Merge Sort Solving Recurrences The Master Theorem

Asymptotic Notations l Upper Bound Notation: n f(n) is O(g(n)) if there exist positive constants c and n 0 such that f(n)  c  g(n) for all n  n 0 n Formally, O(g(n)) = { f(n):  positive constants c and n 0 such that f(n)  c  g(n)  n  n 0 l Big O fact: n A polynomial of degree k is O(n k )

Asymptotic Notations l Asymptotic lower bound: n f(n) is  (g(n)) if  positive constants c and n 0 such that 0  c  g(n)  f(n)  n  n 0 l Asymptotic tight bound: n f(n) is  (g(n)) if  positive constants c 1, c 2, and n 0 such that c 1 g(n)  f(n)  c 2 g(n)  n  n 0 n f(n) =  (g(n)) if and only if f(n) = O(g(n)) AND f(n) =  (g(n))

Merge Sort MergeSort(A, p, r) { if (p < r) { q = floor((p + r) / 2); MergeSort(A, p, q); MergeSort(A, q+1, r); Merge(A, p, q, r); } // Merge() takes two sorted subarrays of A and // merges them into a single sorted subarray of A // (how long should this take?)

MERGE (A, p, q, r ) 1. n 1 ← q − p n 2 ← r − q 3. Create arrays L[1.. n 1 + 1] and R[1.. n 2 + 1] 4. FOR i ← 1 TO n 1 5. DO L[i] ← A[p + i − 1] 6. FOR j ← 1 TO n 2 7. DO R[j] ← A[q + j ] 8. L[n 1 + 1] ← ∞ 9. R[n 2 + 1] ← ∞ 10. i ← j ← FOR k ← p TO r 13. DO IF L[i ] ≤ R[ j] 14. THEN A[k] ← L[i] 15. i ← i ELSE A[k] ← R[j] 17. j ← j + 1

Merge Sort: Example Show MergeSort() running on the array A = {10, 5, 7, 6, 1, 4, 8, 3, 2, 9};

Analysis of Merge Sort StatementEffort l So, T(n) =  (1) when n = 1, and 2T(n/2) +  (n) when n > 1 l So what (the value of) is T(n)? MergeSort(A, left, right) {T(n) if (left < right) {  (1) mid = (left + right) / 2;  (1) MergeSort(A, left, mid);T(n/2) MergeSort(A, mid+1, right);T(n/2) Merge(A, left, mid, right);  (n) }

Recurrences l The expression: is a recurrence. n Recurrence: an equation that describes a function in terms of its value on smaller functions

Recurrence Examples

Solving Recurrences l Substitution method l Iteration method l Master method

Solving Recurrences l The substitution method n The “making a good guess method” n Guess the form of the answer, then use induction to find the constants and show that solution works n Examples: u T(n) = 2T(n/2) +  (n)  T(n) =  (n lg n) u T(n) = 2T(  n/2  ) + n  ???

Solving Recurrences l The substitution method n The “making a good guess method” n Guess the form of the answer, then use induction to find the constants and show that solution works n Examples: u T(n) = 2T(n/2) +  (n)  T(n) =  (n lg n) u T(n) = 2T(  n/2  ) + n  T(n) =  (n lg n) u T(n) = 2T((  n/2  )+ 17) + n  ???

Solving Recurrences l The substitution method n The “making a good guess method” n Guess the form of the answer, then use induction to find the constants and show that solution works n Examples: u T(n) = 2T(n/2) +  (n)  T(n) =  (n lg n) u T(n) = 2T(  n/2  ) + n  T(n) =  (n lg n) u T(n) = 2T((  n/2  + 17) + n   (n lg n)

Solving Recurrences l Another option is the “iteration method” n Expand the recurrence n Work some algebra to express as a summation n Evaluate the summation

l s(n) = = c + s(n-1) = c + c + s(n-2) = 2c + s(n-2) = 2c + c + s(n-3) = 3c + s(n-3) … = kc + s(n-k) = ck + s(n-k)

l So far for n >= k we have n s(n) = ck + s(n-k) l What if k = n? n s(n) = cn + s(0) = cn

l So far for n >= k we have n s(n) = ck + s(n-k) l What if k = n? n s(n) = cn + s(0) = cn l So l Thus in general n s(n) = cn

l s(n) =n + s(n-1) =n + n-1 + s(n-2) =n + n-1 + n-2 + s(n-3) =n + n-1 + n-2 + n-3 + s(n-4) =…=… = n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k)

l s(n) =n + s(n-1) =n + n-1 + s(n-2) =n + n-1 + n-2 + s(n-3) =n + n-1 + n-2 + n-3 + s(n-4) =… = n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k) =

l So far for n >= k we have

l What if k = n?

l So far for n >= k we have l What if k = n?

l So far for n >= k we have l What if k = n? l Thus in general

l T(n) = = 2T(n/2) + c = 2(2T(n/2/2) + c) + c = 2 2 T(n/2 2 ) + 2c + c = 2 2 (2T(n/2 2 /2) + c) + 3c = 2 3 T(n/2 3 ) + 4c + 3c = 2 3 T(n/2 3 ) + 7c = 2 3 (2T(n/2 3 /2) + c) + 7c = 2 4 T(n/2 4 ) + 15c … = 2 k T(n/2 k ) + (2 k - 1)c

l So far for n > 2 k we have n T(n) = 2 k T(n/2 k ) + (2 k - 1)c l What if k = lg n? n T(n) = 2 lg n T(n/2 lg n ) + (2 lg n - 1)c = n T(n/n) + (n - 1)c = n T(1) + (n-1)c = nc + (n-1)c = (2n - 1)c