CS200: Algorithm Analysis

ITERATION METHOD FOR SOLVING RECURRENCES This technique requires converting the recurrence to a summation and finding the closed form of the summation. This method is most easy to see with recurrence trees. If I ask you to solve a recurrence using iteration it should be possible to see the pattern by building the recurrence tree and without resorting to the following rather messy technique. Note that this method is not covered in text.

Expand recurrence until a pattern is found: ex. T(n) = n + 4T(n/2) = n + 4(n/2 + 4T(n/4)) = n +4(n/2 + 4((n/4)+4T(n/8))) = n + 2n + 4n + 8n +16n + .....+ ? Question is, how do we get base case of iteration? Base case of iteration occurs when n = 1 or for T(1). Using above example,

ex. n = 64 for T(n) = n + 4T(n/2) = 64 +4(32 + 4(16 + 4(8 + 4(4 + 4(2 +4(1)))))) 1 2 3 4 5 6 Notice that it requires 6 (+1) iterations to get to the base case of T(1), for n = 64, also notice that the base case = 46T(1) = 4lognT(1). thus the iteration results in = n + 2n + 4n + 8n +16n + .....+ 4lognT(1). = n ( S log n-1 2k) + Q (n2) k=0

n ( S log n-1 2k) + Q (n2) k=0 The n2 term results from the following log identity: a log b = b log a thus 4 log n = n log 4 = n2 The above summation is a geometric series: ( S t xk) = (xt+1 -1)/ (x-1) for x > 1

( S t xk) = (xt+1 -1)/ (x-1) for x > 1 k=0 So by substitution of 2 for x and log n for t, we get: ( S t xk) = (xt+1 -1)/ (x-1) for x > 1 k=0 T(n) = n ( S log n-1 2k) + Q (n2) = n(2log n -1/ (2-1)) + Q (n2) = n(n-1) + Q (n2) = Q (n2) You must know geometric and arithmetic series. The iteration method can be used to generate a guess for the substitution method.

MASTER METHOD (MM) FOR SOLVING RECURRENCES This method is not as general as substitution or iteration and can only be used for recurrences of the following form: T(n) = a T(n/b) + f(n) where a >= 1, b > 1 and f(n) > 0 Advantage is that it is a cookbook technique that can be easily applied to many recurrences. Look at general recursion tree from text. Explain intuition behind Master Method. Text proves Master Method (not discussed in class). Almost all recurrences in this course can be solved using MM.

General Recurrence Tree used in MASTER METHOD Case 3 Case2 Case 1 Note: Leaf work is alogbn(n/blogbn) alogbn = nlogba by log identity [a log b = b log a] and (n/blogbn) simplifies to (n/n) thus work at leaves is nlogba

Master Method Cases In this presentation, each case is stated as a ratio instead of a bound on f (as in text). This is easier to understand (I think so and it is recommended by the author of the book) and both presentations are equivalent. CASE1 (work at leaves)/(work at root): nlogba / f(n) = W(ne) for some constant e > 0. This specifies that nlogba is polynomially larger that f(n). Building a recursion tree for this case shows that the weight at each level increases geometrically and that each leaf contains a constant fraction of the total weight. Therefore T(n) = Q (n logba ).

Stated more intuitively : The leaf nodes of the recursion tree are where most of the work is done (by a polynomial order of magnitude). This means that the work done at the other tree levels does not contribute to growth rate as n approaches infinity. We can state case 1 as (work at root)/(work at leaves) so it has the same form as Case 2 and 3. f(n) / nlogba = O(n-e) for some constant e > 0. Ex. T(n) = 9T(n/3) + n, a = 9, b= 3, f(n) = n n / nlog39 = n/ n2 = O(n-1) , where e = 1, c =1, n0 =1 Thus T(n) = Q(nlog39 ) = Q(n2)

CASE2: f(n) / n logba = Q(logk n) for some constant k >= 0. This specifies that f(n) and n logba are the same within a polylogarithmic factor. The recursion tree for this case shows that the weight for each level decreases or remains constant. This means that we must incorporate each levels' work into the solution for our recurrence. Thus T(n) = Q(n logba * log k+1n). ex. T(n) = T(2n/3) + 1, a = 1, b = 3/2, f(n) = 1 1/n log3/21 = 1/n0 = 1/1 = log0n for k = 0, c1 =1, c2 = ½, n0 =1 Thus T(n) = Q(nlog3/21 * log1n) = Q(log n)

CASE3: f(n) / nlogba = W(ne) for some constant e > 0 (and af(n/b) <= cf(n) for some constant c < 1). This specifies that f(n) is polynomially larger than nlogba. The recursion tree for this case shows that the weight at each level is geometrically decreasing and the root contains a constant fraction of the total weight. Thus as n approaches infinity, the work done at the root defines growth rate. Thus T(n) = Q(f(n)). Note that the regularity condition must be proven.

ex. T(n) = 3T(n/4) + nlog n, a = 3, b = 4, f(n) =nlog n, nlog n / nlog43 = nlog n / n.793 = n.2 log n/1 for e  .2, c=1, n0 =1 and 3(n/4 log(n/4)) ≤cn log n for c = 3/4 Thus T(n) = Q(n log n). Now try some examples on your own (identify case and solve) 1. Mergesort T(n) = 2T(n/2) + n 2. Binsearch T(n) = T(n/2) + 1 3. Straussens T(n) = 7T(n/2) + n2 4. Mystery T(n) = 4T(n/2) + n3

Summary Iteration Method Master Method Three cases To solve, use ratio of root work to leaf work, simplify ratio, and specify one of 3 cases by proving it fits the bound of the case.