1. Merge Sort Solving Recurrences The Master Theorem
CS 332: Algorithms
Merge Sort
Solving Recurrences
The Master Theorem
David Luebke /28/2019

2. Merge Sort MergeSort(A, left, right) { if (left < right) {
mid = floor((left + right) / 2);
MergeSort(A, left, mid);
MergeSort(A, mid+1, right);
Merge(A, left, mid, right); }
// Merge() takes two sorted subarrays of A and
// merges them into a single sorted subarray of A
// (how long should this take?)
David Luebke /28/2019

3. Merge Sort: Example
Show MergeSort() running on the array A = {10, 5, 7, 6, 1, 4, 8, 3, 2, 9};
David Luebke /28/2019

4. Analysis of Merge Sort Statement Effort
So T(n) = (1) when n = 1, and T(n/2) + (n) when n > 1
So what (more succinctly) is T(n)?
MergeSort(A, left, right) { T(n)
if (left < right) { (1)
mid = floor((left + right) / 2); (1)
MergeSort(A, left, mid); T(n/2)
MergeSort(A, mid+1, right); T(n/2)
Merge(A, left, mid, right); (n) }
David Luebke /28/2019

5. Recurrences The expression: is a recurrence.
Recurrence: an equation that describes a function in terms of its value on smaller functions
David Luebke /28/2019

6. Recurrence Examples
David Luebke /28/2019

7. Solving Recurrences Substitution method Iteration method Master method
David Luebke /28/2019

8. Solving Recurrences The substitution method (CLR 4.1)
A.k.a. the “making a good guess method”
Guess the form of the answer, then use induction to find the constants and show that solution works
Examples:
T(n) = 2T(n/2) + (n)  T(n) = (n lg n)
T(n) = 2T(n/2 + n  ???
David Luebke /28/2019

9. Solving Recurrences The substitution method (CLR 4.1)
A.k.a. the “making a good guess method”
Guess the form of the answer, then use induction to find the constants and show that solution works
Examples:
T(n) = 2T(n/2) + (n)  T(n) = (n lg n)
T(n) = 2T(n/2) + n  T(n) = (n lg n)
T(n) = 2T(n/2 + 17) + n  ???
David Luebke /28/2019

10. Solving Recurrences The substitution method (CLR 4.1)
A.k.a. the “making a good guess method”
Guess the form of the answer, then use induction to find the constants and show that solution works
Examples:
T(n) = 2T(n/2) + (n)  T(n) = (n lg n)
T(n) = 2T(n/2) + n  T(n) = (n lg n)
T(n) = 2T(n/2 + 17) + n  T(n) = (n lg n)
David Luebke /28/2019

11. Solving Recurrences
Another option is what the book calls the “iteration method”
Expand the recurrence
Work some algebra to express as a summation
Evaluate the summation
We will show several examples
David Luebke /28/2019

12. s(n) = c + s(n-1) c + c + s(n-2) 2c + s(n-2) 2c + c + s(n-3)…
kc + s(n-k) = ck + s(n-k)
David Luebke /28/2019

13. So far for n >= k we have What if k = n?
s(n) = ck + s(n-k)
What if k = n?
s(n) = cn + s(0) = cn
David Luebke /28/2019

14. So far for n >= k we have What if k = n? So Thus in general
s(n) = ck + s(n-k)
What if k = n?
s(n) = cn + s(0) = cn So
Thus in general
s(n) = cn
David Luebke /28/2019

15. = n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k)
= n + s(n-1)
= n + n-1 + s(n-2)
= n + n-1 + n-2 + s(n-3)
= n + n-1 + n-2 + n-3 + s(n-4)
= …
= n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k)
David Luebke /28/2019

16. = n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k) =
= n + s(n-1)
= n + n-1 + s(n-2)
= n + n-1 + n-2 + s(n-3)
= n + n-1 + n-2 + n-3 + s(n-4)
= …
= n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k) =
David Luebke /28/2019

17. So far for n >= k we have
David Luebke /28/2019

18. So far for n >= k we have
What if k = n?
David Luebke /28/2019

19. So far for n >= k we have
What if k = n?
David Luebke /28/2019

20. So far for n >= k we have
What if k = n?
Thus in general
David Luebke /28/2019

21. T(n) = 2T(n/2) + c 2(2T(n/2/2) + c) + c 22T(n/22) + 2c + c…
2kT(n/2k) + (2k - 1)c
David Luebke /28/2019

22. So far for n > 2k we have What if k = lg n?
T(n) = 2kT(n/2k) + (2k - 1)c
What if k = lg n?
T(n) = 2lg n T(n/2lg n) + (2lg n - 1)c
= n T(n/n) + (n - 1)c
= n T(1) + (n-1)c
= nc + (n-1)c = (2n - 1)c
David Luebke /28/2019

23. a(aT(n/b/b) + cn/b) + cn a2T(n/b2) + cna/b + cn
a2(aT(n/b2/b) + cn/b2) + cn(a/b + 1)
a3T(n/b3) + cn(a2/b2) + cn(a/b + 1)
a3T(n/b3) + cn(a2/b2 + a/b + 1) …
akT(n/bk) + cn(ak-1/bk-1 + ak-2/bk-2 + … + a2/b2 + a/b + 1)
David Luebke /28/2019

24. So we have
T(n) = akT(n/bk) + cn(ak-1/bk a2/b2 + a/b + 1)
For k = logb n
n = bk
T(n) = akT(1) + cn(ak-1/bk a2/b2 + a/b + 1)
= akc + cn(ak-1/bk a2/b2 + a/b + 1)
= cak + cn(ak-1/bk a2/b2 + a/b + 1)
= cnak /bk + cn(ak-1/bk a2/b2 + a/b + 1)
= cn(ak/bk a2/b2 + a/b + 1)
David Luebke /28/2019

25. So with k = logb n What if a = b?
T(n) = cn(ak/bk a2/b2 + a/b + 1)
What if a = b?
T(n) = cn(k + 1)
= cn(logb n + 1)
= (n log n)
David Luebke /28/2019

26. So with k = logb n What if a < b?
T(n) = cn(ak/bk a2/b2 + a/b + 1)
What if a < b?
David Luebke /28/2019

27. So with k = logb n What if a < b?
T(n) = cn(ak/bk a2/b2 + a/b + 1)
What if a < b?
Recall that (xk + xk-1 + … + x + 1) = (xk+1 -1)/(x-1)
David Luebke /28/2019

28. So with k = logb n What if a < b?
T(n) = cn(ak/bk a2/b2 + a/b + 1)
What if a < b?
Recall that (xk + xk-1 + … + x + 1) = (xk+1 -1)/(x-1)
So:
David Luebke /28/2019

29. So with k = logb n What if a < b?
T(n) = cn(ak/bk a2/b2 + a/b + 1)
What if a < b?
Recall that (xk + xk-1 + … + x + 1) = (xk+1 -1)/(x-1)
So:
T(n) = cn ·(1) = (n)
David Luebke /28/2019

30. So with k = logb n What if a > b?
T(n) = cn(ak/bk a2/b2 + a/b + 1)
What if a > b?
David Luebke /28/2019

31. So with k = logb n What if a > b?
T(n) = cn(ak/bk a2/b2 + a/b + 1)
What if a > b?
David Luebke /28/2019

32. So with k = logb n What if a > b?
T(n) = cn(ak/bk a2/b2 + a/b + 1)
What if a > b?
T(n) = cn · (ak / bk)
David Luebke /28/2019

33. So with k = logb n What if a > b?
T(n) = cn(ak/bk a2/b2 + a/b + 1)
What if a > b?
T(n) = cn · (ak / bk)
= cn · (alog n / blog n) = cn · (alog n / n)
David Luebke /28/2019

34. So with k = logb n What if a > b?
T(n) = cn(ak/bk a2/b2 + a/b + 1)
What if a > b?
T(n) = cn · (ak / bk)
= cn · (alog n / blog n) = cn · (alog n / n)
recall logarithm fact: alog n = nlog a
David Luebke /28/2019

35. So with k = logb n What if a > b?
T(n) = cn(ak/bk a2/b2 + a/b + 1)
What if a > b?
T(n) = cn · (ak / bk)
= cn · (alog n / blog n) = cn · (alog n / n)
recall logarithm fact: alog n = nlog a
= cn · (nlog a / n) = (cn · nlog a / n)
David Luebke /28/2019

36. So with k = logb n What if a > b?
T(n) = cn(ak/bk a2/b2 + a/b + 1)
What if a > b?
T(n) = cn · (ak / bk)
= cn · (alog n / blog n) = cn · (alog n / n)
recall logarithm fact: alog n = nlog a
= cn · (nlog a / n) = (cn · nlog a / n)
= (nlog a )
David Luebke /28/2019

37. So…
David Luebke /28/2019

38. The Master Theorem if T(n) = aT(n/b) + f(n) then
David Luebke /28/2019

39. Using The Master Method
T(n) = 9T(n/3) + n
a=9, b=3, f(n) = n
nlogb a = nlog3 9 = (n2)
Since f(n) = O(nlog3 9 - ), where =1, case 1 applies:
Thus the solution is T(n) = (n2)
David Luebke /28/2019