1. CSC 413/513: Intro to Algorithms Merge Sort Solving Recurrences

2. Updates l Homework 1 posted n Due on Sep-01 l Office hours have been updated n MW: 1:00-2:15 n Tues: 1:00-2:30 n Thurs: 2:00-3:00 l Note taker?

3. Review: Asymptotic Notation l Upper Bound Notation: n f(n) is O(g(n)) if there exist positive constants c and n 0 such that f(n)  c  g(n) for all n  n 0 n Formally, O(g(n)) = { f(n):  positive constants c and n 0 such that f(n)  c  g(n)  n  n 0 l Big O fact: n A polynomial of degree k is O(n k )

4. Review: Asymptotic Notation l Asymptotic lower bound: n f(n) is  (g(n)) if  positive constants c and n 0 such that 0  c  g(n)  f(n)  n  n 0 l Asymptotic tight bound: n f(n) is  (g(n)) if  positive constants c 1, c 2, and n 0 such that c 1 g(n)  f(n)  c 2 g(n)  n  n 0 n f(n) =  (g(n)) if and only if f(n) = O(g(n)) AND f(n) =  (g(n))

5. Other Asymptotic Notations l A function f(n) is o(g(n)) if for any positive constant c,  n 0 such that f(n) < c g(n)  n  n 0 l A function f(n) is  (g(n)) if for any positive constant c,  n 0 such that c g(n) < f(n)  n  n 0 l Intuitively,  o() is like <  O() is like    () is like >   () is like    () is like =

6. l How comfortable are you with recursion?

7. Merge Sort MergeSort(A, left, right) { if (left < right) { mid = floor((left + right) / 2); MergeSort(A, left, mid); MergeSort(A, mid+1, right); Merge(A, left, mid, right); } // Merge() takes two sorted subarrays of A and // merges them into a single sorted subarray of A // (how long should this take?)

8. How Merge works

9. Merge Sort: Example Show MergeSort() running on the array A = {10, 5, 7, 6, 1, 4, 8, 3, 2, 9};

10. Neat little example from the book

11. Analysis of Merge Sort StatementEffort l So T(n) =  (1) when n = 1, and 2T(n/2) +  (n) when n > 1 l So what (more succinctly) is T(n)? MergeSort(A, left, right) {T(n) if (left < right) {  (1) mid = floor((left + right) / 2);  (1) MergeSort(A, left, mid);T(n/2) MergeSort(A, mid+1, right);T(n/2) Merge(A, left, mid, right);  (n) }

12. Recurrences l The expression: is a recurrence. n Recurrence: an equation that describes a function in terms of itself on smaller inputs

13. Recurrence Examples

14. Solving Recurrences l Substitution method l Iteration method l Master method

15. Solving Recurrences l The substitution method (CLR 4.1) n A.k.a. the “making a good guess method” n Guess the form of the answer, then use induction to find the constants and show that solution works n Examples: u T(n) = 2T(n/2) +  (n)  T(n) =  (n lg n) u T(n) = 2T(  n/2  ) + n  ??? Tree method often used

16. Solving Recurrences l The substitution method (CLR 4.1) n A.k.a. the “making a good guess method” n Guess the form of the answer, then use induction to find the constants and show that solution works n Examples: u T(n) = 2T(n/2) +  (n)  T(n) =  (n lg n) u T(n) = 2T(  n/2  ) + n  T(n) =  (n lg n) u T(n) = 2T(  n/2  + 17) + n  ???

17. Solving Recurrences l The substitution method (CLR 4.1) n A.k.a. the “making a good guess method” n Guess the form of the answer, then use induction to find the constants and show that solution works n Examples: u T(n) = 2T(n/2) +  (n)  T(n) =  (n lg n) u T(n) = 2T(  n/2  ) + n  T(n) =  (n lg n) u T(n) = 2T(  n/2  + 17) + n   (n lg n)

18. Substitution examples l T(n) = 2T(n/2) + kn, T(1)=c’ (e.g., merge sort) n Guess T(n) <= c.nlgn does not work n Guess T(n) =c’, c>=k l T(n) = 2T(n/2) + k, T(1)=c’ (e.g., D&C search) n Guess T(n) <= cn does not work n Guess T(n) =k, c>=d+c’ l T(n) = 2T(√n) + k.lg n n Change variable first Must prove the exact form guessed; otherwise revise your guess

19. Solving Recurrences l Another option is what the book calls the “iteration method” n Expand the recurrence n Work some algebra to express as a summation n Evaluate the summation l We will show several examples

20. l s(n) = c + s(n-1) c + c + s(n-2) 2c + s(n-2) 2c + c + s(n-3) 3c + s(n-3) … kc + s(n-k) = ck + s(n-k)

21. l So far for n >= k we have n s(n) = ck + s(n-k) l What if n=k? n s(n) = cn + s(0) = cn

22. l So far for n >= k we have n s(n) = ck + s(n-k) l What if k = n? n s(n) = cn + s(0) = cn l So l Thus in general n s(n) = cn

23. l s(n) =n + s(n-1) =n + n-1 + s(n-2) =n + n-1 + n-2 + s(n-3) =n + n-1 + n-2 + n-3 + s(n-4) =…=… = n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k)

24. l s(n) =n + s(n-1) =n + n-1 + s(n-2) =n + n-1 + n-2 + s(n-3) =n + n-1 + n-2 + n-3 + s(n-4) =… = n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k) =

25. l So far for n >= k we have

26. l What if k = n?

27. l So far for n >= k we have l What if k = n?

28. l So far for n >= k we have l What if k = n? l Thus in general

29. l T(n) = 2T(n/2) + c 2(2T(n/2/2) + c) + c 2 2 T(n/2 2 ) + 2c + c 2 2 (2T(n/2 2 /2) + c) + 3c 2 3 T(n/2 3 ) + 4c + 3c 2 3 T(n/2 3 ) + 7c 2 3 (2T(n/2 3 /2) + c) + 7c 2 4 T(n/2 4 ) + 15c … 2 k T(n/2 k ) + (2 k - 1)c

30. l So far for n > 2 k-1 we have n T(n) = 2 k T(n/2 k ) + (2 k - 1)c l What if n=2 k, i.e., k = lg n? n T(n) = 2 lg n T(n/2 lg n ) + (2 lg n - 1)c = n T(n/n) + (n - 1)c = n T(1) + (n-1)c = nc + (n-1)c = (2n - 1)c