1. MCA 202: Discrete Mathematics Instructor Neelima Gupta ngupta@cs.du.ac.in

2. Table Of Contents Solving Recurrences The Master Theorem

3. Recurrence Relations

4. Recurrences The expression: is a recurrence. –Recurrence: an equation that describes a function in terms of its value on smaller functions

5. Recurrence Examples

6. Solving Recurrences Substitution method Iteration method Master method

7. Solving Recurrences The substitution method (CLR 4.1) –“Making a good guess” method –Guess the form of the answer, then use induction to find the constants and show that solution works –Examples: T(n) = 2T(n/2) +  (n)  T(n) =  (n lg n) T(n) = 2T(  n/2  ) + n  ???

8. Solving Recurrences The substitution method (CLR 4.1) –“Making a good guess” method –Guess the form of the answer, then use induction to find the constants and show that solution works –Examples: T(n) = 2T(n/2) +  (n)  T(n) =  (n lg n) T(n) = 2T(  n/2  ) + n  T(n) =  (n lg n) T(n) = 2T(  n/2  )+ 17) + n  ???

9. Substitution method Guess the form of the solution. Use mathematical induction to find the constants and show that the solution works. The substitution method can be used to establish either upper or lower bounds on a recurrence.

10. An example (Substitution method ) T(n) = 2T(floor( n/2) ) +n We guess that the solution is T(n)=0(n lg n). i.e. to show that T(n) ≤ cn lg n, for some constant c> 0 and n ≥ m. Assume that this bound holds for [n/2]. So, we get T(n) ≤ 2(c floor (n/2) lg(floor(n/2))) + n ≤ cn lg(n/2) + n = cn lg n – cn lg 2 + n = cn lg n – cn + n ≤ cn lg n where, the last step holds as long as c≥ 1.

11. Boundary conditions : Suppose, T(1)=1 is the sole boundary condition of the recurrence. then, for n=1, the bound T(n)≤ c n lg n yields T(1)≤ c lg1=0, which is at odds with T(1)=1. Thus,the base case of our inductive proof fails to hold. To overcome this difficulty, we can take advantage of the asymptotic notation which only requires us to prove T(n)≤c n lg n for n≥ m. The idea is to remove the difficult boundary condition T(1)= 1 from consideration. Thus, we can replace T(1) by T(2) as the base cases in the inductive proof, letting m=2.

12. Contd.... From the recurrence, with T(1) = 1, we get T(2)=4 We require T(2)≤ c 2 lg 2 It is clear that, any choice of c≥2 suffices for the base cases

13. Are we done? Have we proved the case for n =3. Have we proved that T(3) ≤ c 3 lg 3. No. Since floor(3/2) = 1 and for n =1, it does not hold. So induction does not apply on n= 3. From the recurrence, with T(1) = 1, we get T(3) = 5. The inductive proof that T(n) ≤ c n lg n for some constant c≥1 can now be completed by choosing c large enough that T(3)≤c 3 lg 3 also holds. It is clear that, any choice of c ≥ 2 is sufficient for this to hold. Thus we can conclude that T(n) ≤ c n lg n for any c ≥ 2 and n ≥ 2.

14. Wrong Application of induction Given recurrence: T(n) = 2T(n/2) + 1 Guess: T(n) = O(n) Claim : Ǝ some constant c and n 0, such that T(n) = n 0. Proof: Suppose the claim is true for all values <= n/2 then T(n)=2T(n/2)+1 (given) <= 2.c.(n/2) +1 (by induction hypothesis) = cn+1 =1

15. Note that T(n/2) T(n)<=(c+1)n (this statement is true but does not help us in establishing a solution to the problem) T(1)<=c (when n=1) T(2)<=(c+1).2 T(2^2)<=(c+2).2^2.. T(2^i)<=(c+i).2^I So, T(n)= T(2^logn)= (c+ log n)n = Ɵ (n log n) Why is it Wrong?

16. What if we have extra lower order terms? So, does that mean that the claim we initially made that T(n)=O(n) was wrong ? No. Recall: T(n)=2T(n/2)+1 (given) <= 2.c.(n/2) +1 (by induction hypothesis) = cn+1 Note that in the proof we have an extra lower order term in our inductive proof.

17. Given Recurrence: T(n) = 2T(n/2) + 1 Guess: T(n) = O(n) Claim : Ǝ some constant c and n 0, such that T(n) = n 0 Proof: Suppose the claim is true for all values <= n/2 then T(n)=2T(n/2)+1 <=2[c(n/2)-b]+1 <= cn-2b+1 <= cn-b+(1-b) =1 Thus, T(n/2) <= cn/2 – b => T(n) =1 Hence, by induction T(n) = O(n), i.e. Our claim was true. Hence proved.

18. Another Example (Substitution method ) T(n) = 2T(  n/2 + 17  ) + n, T(34) < c.34.lg 34 for an appropriate constant c. Since it looks similar to T(n) = T(n/2) + n, We guess that the solution is T(n) = O(n lg n). Thanks to Manoj Patial, Roll No. - 20 (MCA 2012)

19. Let us assume that it holds ∀ m ≤  n/2 + 17  > T(m) ≤ cm lg m ∀ m ≤  n/2 + 17  Hence we can write T(n) = 2T(  n/2 + 17  ) + n ≤ 2c (  n/2+17  )lg (  n/2 +17  ) + n ≤ 2c (n/2+17)lg(n/2 +17) + n ≤ 2c(n/2+17) lg(n/2+n/4) + n ∀ n/4 ≥ 17 = (cn + 34c) lg(3n/4) + n = cn lgn – cn lg 4/3 + 34c lg n – 34c lg 4/3 + n ≤ cn lgn – cn lg 4/3 + 34c lg n + n As lg n = 0(n) therefore 34 c lgn ≤ kn ∀ n ≥ n o hence T(n)≤ cn lg n + n(1- c lg 4/3) + kn ∀ n ≥ max {n o, 68} = cn lg n + n(k+1-c lg 4/3) …………..(1) now we can always choose c such that k+1 – c lg 4/3 ≤ 0 Thanks to Manoj Patial, Roll No. - 20 (MCA 2012)

20. > k+1 ≤ c lg 4/3 > c ≥ (k+1)/ lg(4/3) = k T(n) ≤ cn lg n for some constant c ≥ k and n ≥ max {n o, 68} …………….(Hence proved) Boundary Conditions: n = 35, it holds since it holds for  35/2 + 17  =34, therefore it holds for 35 by induction. n = 36, it holds since it holds for  36/2 + 17  =35, therefore it holds for 36 by induction.. And so on. Thanks to Manoj Patial, Roll No. - 20 (MCA 2012)

21. Solving Recurrences using Recursion Tree Method Here while solving recurrences, we divide the problem into subproblems of equal size. For e.g., T(n) = a T(n/b) + f(n) where a > 1,b > 1 and f(n) is a given function. F(n) is the cost of splitting or combining the sub problems. n T n/b Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

22. n T n/2 n/2 2 1) T(n) = 2T(n/2) + n The recursion tree for this recurrence is : n n/2 n/2 n log 2 n : 1111111 Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

23. When we add the values across the levels of the recursion tree, we get a value of n for every level. We have n + n + n + …… log n times = n (1 + 1 + 1 + …… log n times) = n (log 2 n) = Ɵ (n log n) T(n) = Ɵ (n log n) Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

24. II. Given : T(n) = 2T(n/2) + 1 Solution : The recursion tree for the above recurrence is 1 1 2 log n 4 1 :::::: Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

25. Now we add up the costs over all levels of the recursion tree, to determine the cost for the entire tree : We get series like 1 + 2 + 2 2 + 2 3 + …… log n times which is a G.P. [ So, using the formula for sum of terms in a G.P. : a + ar + ar 2 + ar 3 + …… + ar n – 1 = a( r n – 1 ) r – 1 ] = 1 (2 log n – 1) 2 – 1 = n – 1 = Ɵ (n – 1) (neglecting the lower order terms) = Ɵ (n) Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

26. III. Given : T(n) = T(n/3) + T(2n/3) + n Solution : The recursion tree for the above recurrence is n n/3 2n/3 n log 3 n log 3/2 n n 1 n/3 i : n n/3 2n/3 n 3 2 2 n 3 3 1 2n 3 3 n. (3/2) 2 Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

27. When we add the values across the levels of the recursion tree, we get a value of n for every level. Since the shortest path from the root to the leaf is n → n → n → n → …. 1 3 3 2 3 3 we have 1 when n = 1 3 i => n = 3 i Taking log ₃ on both the sides => log ₃ n = i Thus the height of the shorter tree is log ₃ n T(n) > n log ₃ n … A Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

28. Similarly, the longest path from root to the leaf is n → 2 n → 2 2 n → … 1 3 3 So rightmost will be the longest when 2 k n = 1 3 or n = 1 (3/2) k => k = log 3/2 n T(n) < n log 3/2 n … B Since base does not matter in asymptotic notation, we guess from A and B T(n) = Ɵ (n log 2 n) Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

29. Solving Recurrences Another option is “iteration method” –Expand the recurrence –Work some algebra to express as a summation –Evaluate the summation We will show some examples

30. Solve the following recurrence: T(n) = T(αn) + T(βn) + n, where 0 < α ≤ β < 1 Assume suitable initial conditions. Assignment 4

31. The Master Theorem Given: a divide and conquer algorithm –An algorithm that divides the problem of size n into a subproblems, each of size n/b –Let the cost of each stage (i.e., the work to divide the problem + combine solved subproblems) be described by the function f(n) Then, the Master Theorem gives us a cookbook for the algorithm’s running time:

32. The Master Theorem if T(n) = aT(n/b) + f(n) then

33. Using The Master Method T(n) = 9T(n/3) + n –a=9, b=3, f(n) = n –n log b a = n log 3 9 =  (n 2 ) –Since f(n) = O(n log 3 9 -  ), where  =1, case 1 applies: –Thus the solution is T(n) =  (n 2 )

34. More Examples of Master’s Theorem T(n) = 3T(n/5) + n T(n) = 2T(n/2) + n T(n) = 2T(n/2) + 1 T(n) = T(n/2) + n T(n) = T(n/2) + 1

35. When Master’s Theorem cannot be applied T(n) = 2T(n/2) + n logn T(n) = 2T(n/2) + n/ logn

36. Up Next Proving the correctness of Algorithms