Mathematical Foundations (Solving Recurrence) Neelima Gupta Department of Computer Science University of Delhi ngupta@cs.du.ac.in people.du.ac.in/~ngupta

Recurrences The expression: is a recurrence. Recurrence: an equation that describes a function in terms of its value on smaller functions

Recurrence Examples

Solving Recurrences Substitution method Iteration method Master method

Substitution Method Guess and Check Method Guess the solution and Check by induction Example: T(n) = 2T(n/2) + n Guess T(n) = O(n lg n) i.e. there exist constants c >0 and m > 0 such that T(n) <= c n lg n

How do you guess? Master’s theorem for similar looking recurrence T(n) = 2T(n/2) + n  T(n) = O(n lg n) Recursion Tree

Check Assume that the claim holds for all m <= n/2 . Thus, T(n) ≤ 2(c n/2 lg(n/2 ) + n ≤ cn lg(n/2) + n = cn lg n – cn lg 2 + n = cn lg n – cn + n ≤ cn lg n where , the last step holds as long as c≥ 1.

Base Case : For n = 1, T(1) = 1 and c lg1=0 Thus T(1) < = c 1lg 1 does not hold Try for n =2, T(2) = 2 T(1) + 2 = 4 and c2 lg 2 = 2c Thus, T(2) < = c 2 lg 2 holds whenever 4 <= 2c i.e c >= 2. Are we done? Can we say that we have proved T(n) < = 2 n lg n for all n>=2? …………..(1) Lets see: we have proved for n=2, have we proved for n = 3? T(3) = 2 T(1) + 3 ……what now? We have not proved for T(1), so we are stuck, we can’t claim (1) so far. Lets try to prove it explicitly for n = 3.

Are we done? T(3) = 2 T(1) + 3 = 5 and c3 log3 > 5 for c = 2 Are we done now? Can we now say that we have proved T(n) < = 2 n lg n for all n>=2? …………..(1) Lets see: we have proved for n=2 and n=3, have we proved for n = 4? Since it holds for n= 2, by induction hypothesis, it holds for n = 4 as well. n=5? Since we have proved for floor(5/2), by induction hypothesis, it holds for n = 5 as well. And so on.

Wrong Application of induction Given recurrence: T(n) = 2T(n/2) + 1 Guess: T(n) = O(n) Claim : Ǝ some constant c and n0 , such that T(n) <= cn , for all n >= n0 . Proof: Suppose the claim is true for all values <= n/2 then T(n)=2T(n/2)+1 (given) <= 2.c.(n/2) +1 (by induction hypothesis) = cn+1 <= (c+1) n , for all n>=1 Hence T(n) = O(n) ……..Wrong

So, T(n)= T(2logn)= (c+ log n)n = Ɵ (n log n) Why is it Wrong? Note that T(n/2)<=cn/2 => T(n)<=(c+1)n (this statement is true but does not help us in establishing a solution to the problem) T(1)<=c (when n=1) T(2)<=(c+1).2 T(22)<=(c+2).22 . . T(2i)<=(c+i).2i So, T(n)= T(2logn)= (c+ log n)n = Ɵ (n log n)

What if we have extra lower order terms? So, does that mean that the claim we initially made that T(n)=O(n) was wrong ? No. T(n)=2T(n/2)+1 (given) <= 2.c.(n/2) +1 (by induction hypothesis) = cn+1 Note that we have an extra lower order term in our inductive proof. So we subtract that lower order term from our claim- order remains the same.

T(n) <= cn - b for all n >= n0 Proof: Suppose the claim is true for all values <= n/2 then T(n)=2T(n/2)+1 <=2[c(n/2)-b]+1 <= cn-2b+1 <= cn-b+(1-b) <= cn-b , whenver b>=1 Thus, T(n/2) <= cn/2 – b => T(n) <= cn – b, whenever b>=1 Hence, by induction T(n) = O(n) we have proved that our claim was true. Note that this recurrence is well defined only when n is a power of 2, else we’ll have to use the floor function. Take T(1) anything <= c – b.

Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial Solving Recurrences using Recursion Tree Method Here while solving recurrences, we divide the problem into subproblems of equal size. For e.g., T(n) = a T(n/b) + f(n) where a > 1 ,b > 1 and f(n) is a given function . F(n) is the cost of splitting or combining the sub problems. n T n/b T n/b Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial 1) T(n) = 2T(n/2) + n The recursion tree for this recurrence is : n n/2 n/2 n log2 n : n T n/2 T n/2 n/22 n/22 n/22 n/22 1 1 1 1 1 1 1 Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial When we add the values across the levels of the recursion tree, we get a value of n for every level. We have n + n + n + …… log n times = n (1 + 1 + 1 + …… log n times) = n (log2 n) = Ɵ (n log n) T(n) = Ɵ (n log n) Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial II. Given : T(n) = 2T(n/2) + 1 Solution : The recursion tree for the above recurrence is 1 1 2 log n 4 1 : Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial Now we add up the costs over all levels of the recursion tree, to determine the cost for the entire tree : We get series like 1 + 2 + 22 + 23 + …… log n times which is a G.P. [ So, using the formula for sum of terms in a G.P. : a + ar + ar2 + ar3 + …… + ar n – 1 = a( r n – 1 ) r – 1 ] = 1 (2log n – 1) 2 – 1 = n – 1 = Ɵ (n – 1) (neglecting the lower order terms) = Ɵ (n) Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial III. Given : T(n) = T(n/3) + T(2n/3) + n Solution : The recursion tree for the above recurrence is n n/3 2n/3 n log3 n log3/2 n 1 n/3i : n n/3 2n/3 n 32 2 n 3 3 1 2n 3 3 n . (3/2)2 Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial When we add the values across the levels of the recursion tree , we get a value of n for every level. Since the shortest path from the root to the leaf is n → n → n → n → …. 1 3 32 33 we have 1 when n = 1 3i => n = 3i Taking log₃ on both the sides => log₃ n = i Thus the height of the shorter tree is log₃ n T(n) > n log₃ n … A Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial Similarly, the longest path from root to the leaf is n → 2 n → 2 2 n → … 1 3 3 So rightmost will be the longest when 2 k n = 1 3 or n = 1 (3/2)k => k = log3/2 n T(n) < n log3/2 n … B Since base does not matter in asymptotic notation , we guess from A and B T(n) = Ɵ (n log2 n) Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

Solving Recurrences Another option is “iteration method” Expand the recurrence Work some algebra to express as a summation Evaluate the summation We will show some examples

Assignment 4 Solve the following recurrence: T(n) = T(αn) + T(βn) + n, where 0 < α ≤ β < 1 Assume suitable initial conditions.

The Master Theorem Given: a divide and conquer algorithm An algorithm that divides the problem of size n into a subproblems, each of size n/b Let the cost of each stage (i.e., the work to divide the problem + combine solved subproblems) be described by the function f(n) Then, the Master Theorem gives us a cookbook for the algorithm’s running time:

The Master Theorem if T(n) = aT(n/b) + f(n) then

Using The Master Method T(n) = 9T(n/3) + n a=9, b=3, f(n) = n nlogb a = nlog3 9 = (n2) Since f(n) = O(nlog3 9 - ), where =1, case 1 applies: Thus the solution is T(n) = (n2)

More Examples of Master’s Theorem T(n) = 3T(n/5) + n T(n) = 2T(n/2) + n T(n) = 2T(n/2) + 1 T(n) = T(n/2) + n T(n) = T(n/2) + 1

When Master’s Theorem cannot be applied T(n) = 2T(n/2) + n logn T(n) = 2T(n/2) + n/ logn

Up Next Proving the correctness of Algorithms