1. Mathematical Foundations (Solving Recurrence)
Neelima Gupta
Department of Computer Science
University of Delhi
people.du.ac.in/~ngupta

2. Recurrences The expression: is a recurrence.
Recurrence: an equation that describes a function in terms of its value on smaller functions

3. Recurrence Examples

4. Solving Recurrences
Substitution method
Iteration method
Master method

5. Substitution Method Guess and Check Method Guess the solution and
Check by induction
Example:
T(n) = 2T(n/2) + n
Guess T(n) = O(n lg n) i.e. there exist constants
c >0 and m > 0 such that
T(n) <= c n lg n

6. How do you guess? Master’s theorem for similar looking recurrence
T(n) = 2T(n/2) + n  T(n) = O(n lg n)
Recursion Tree

7. Check Assume that the claim holds for all m <= n/2 . Thus,
T(n) ≤ 2(c n/2 lg(n/2 ) + n
≤ cn lg(n/2) + n
= cn lg n – cn lg 2 + n
= cn lg n – cn + n
≤ cn lg n
where , the last step holds as long as c≥ 1.

8. Base Case :
For n = 1, T(1) = 1 and c lg1=0
Thus T(1) < = c 1lg 1 does not hold
Try for n =2,
T(2) = 2 T(1) + 2 = 4 and c2 lg 2 = 2c
Thus, T(2) < = c 2 lg 2 holds whenever 4 <= 2c i.e c >= 2.
Are we done?
Can we say that we have proved
T(n) < = 2 n lg n for all n>=2? …………..(1)
Lets see:
we have proved for n=2, have we proved for n = 3?
T(3) = 2 T(1) + 3
……what now? We have not proved for T(1), so we are stuck, we can’t claim (1) so far.
Lets try to prove it explicitly for n = 3.

9. Are we done? T(3) = 2 T(1) + 3 = 5 and c3 log3 > 5 for c = 2
Are we done now?
Can we now say that we have proved
T(n) < = 2 n lg n for all n>=2? …………..(1)
Lets see:
we have proved for n=2 and n=3, have we proved for n = 4?
Since it holds for n= 2, by induction hypothesis, it holds for n = 4 as well.
n=5? Since we have proved for floor(5/2), by induction hypothesis, it holds for n = 5 as well.
And so on.

10. Wrong Application of induction
Given recurrence: T(n) = 2T(n/2) + 1
Guess: T(n) = O(n)
Claim : Ǝ some constant c and n0 , such that
T(n) <= cn , for all n >= n0 .
Proof: Suppose the claim is true for all values <= n/2
then T(n)=2T(n/2)+1 (given)
<= 2.c.(n/2) +1 (by induction hypothesis)
= cn+1
<= (c+1) n , for all n>=1
Hence T(n) = O(n) ……..Wrong

11. So, T(n)= T(2logn)= (c+ log n)n = Ɵ (n log n)
Why is it Wrong?
Note that T(n/2)<=cn/2 => T(n)<=(c+1)n (this statement is true but does not help us in establishing a solution to the problem)
T(1)<=c (when n=1)
T(2)<=(c+1).2
T(22)<=(c+2) .
T(2i)<=(c+i).2i
So, T(n)= T(2logn)= (c+ log n)n
= Ɵ (n log n)

12. What if we have extra lower order terms?
So, does that mean that the claim we initially made that T(n)=O(n) was wrong ?
No.
T(n)=2T(n/2)+1 (given)
<= 2.c.(n/2) +1 (by induction hypothesis)
= cn+1
Note that we have an extra lower order term in our inductive proof.
So we subtract that lower order term from our claim- order remains the same.

13. T(n) <= cn - b for all n >= n0
Proof: Suppose the claim is true for all values <= n/2
then
T(n)=2T(n/2)+1
<=2[c(n/2)-b]+1
<= cn-2b+1
<= cn-b+(1-b)
<= cn-b , whenver b>=1
Thus, T(n/2) <= cn/2 – b
=> T(n) <= cn – b, whenever b>=1
Hence, by induction T(n) = O(n) we have proved that our claim was true.
Note that this recurrence is well defined only when n is a power of 2, else we’ll have to use the floor function.
Take T(1) anything <= c – b.

14. Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial
Solving Recurrences using Recursion Tree Method
Here while solving recurrences, we divide the problem into
subproblems of equal size.
For e.g., T(n) = a T(n/b) + f(n) where a > 1 ,b > 1 and f(n) is a given
function .
F(n) is the cost of splitting or combining the sub problems. n
T n/b
T n/b
Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

15. Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial
1) T(n) = 2T(n/2) + n
The recursion tree for this recurrence is : n
n/ n/ n
log2 n : n
T n/2
T n/2
n/22
n/22
n/22
n/22 1 1 1 1 1 1 1
Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

16. Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial
When we add the values across the levels of the recursion tree, we get a value of n for every level. We have n + n + n + …… log n times = n ( …… log n times) = n (log2 n) = Ɵ (n log n) T(n) = Ɵ (n log n)
Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

17. Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial
II. Given : T(n) = 2T(n/2) + 1 Solution : The recursion tree for the above recurrence is log n 4 1 :
Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

18. Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial
Now we add up the costs over all levels of the recursion tree, to determine the cost for the entire tree : We get series like …… log n times which is a G.P. [ So, using the formula for sum of terms in a G.P. : a + ar + ar2 + ar3 + …… + ar n – 1 = a( r n – 1 ) r – 1 ] = 1 (2log n – 1) 2 – 1 = n – 1 = Ɵ (n – 1) (neglecting the lower order terms) = Ɵ (n)
Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

19. Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial
III.
Given : T(n) = T(n/3) + T(2n/3) + n
Solution : The recursion tree for the above recurrence is n
n/ n/ n
log3 n log3/2 n
n/3i : n
n/3
2n/3 n 32
2 n
3 3
1 2n
3 3
n .
(3/2)2
Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

20. Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial
When we add the values across the levels of the recursion tree , we get a value of n for every level. Since the shortest path from the root to the leaf is n → n → n → n → … we have 1 when n = 1 3i => n = 3i Taking log₃ on both the sides => log₃ n = i Thus the height of the shorter tree is log₃ n T(n) > n log₃ n … A
Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

21. Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial
Similarly, the longest path from root to the leaf is n → 2 n → 2 2 n → … So rightmost will be the longest when 2 k n = 1 3 or n = 1 (3/2)k => k = log3/2 n T(n) < n log3/2 n … B Since base does not matter in asymptotic notation , we guess from A and B T(n) = Ɵ (n log2 n)
Thanks : MCA 2012 Anurag Aggarwal and Aniruddh Jarial

22. Solving Recurrences Another option is “iteration method”
Expand the recurrence
Work some algebra to express as a summation
Evaluate the summation
We will show some examples

23. Assignment 4 Solve the following recurrence: T(n) = T(αn) + T(βn) + n,
where 0 < α ≤ β < 1
Assume suitable initial conditions.

24. The Master Theorem Given: a divide and conquer algorithm
An algorithm that divides the problem of size n into a subproblems, each of size n/b
Let the cost of each stage (i.e., the work to divide the problem + combine solved subproblems) be described by the function f(n)
Then, the Master Theorem gives us a cookbook for the algorithm’s running time:

25. The Master Theorem
if T(n) = aT(n/b) + f(n) then

26. Using The Master Method
T(n) = 9T(n/3) + n
a=9, b=3, f(n) = n
nlogb a = nlog3 9 = (n2)
Since f(n) = O(nlog3 9 - ), where =1, case 1 applies:
Thus the solution is T(n) = (n2)

27. More Examples of Master’s Theorem
T(n) = 3T(n/5) + n
T(n) = 2T(n/2) + n
T(n) = 2T(n/2) + 1
T(n) = T(n/2) + n
T(n) = T(n/2) + 1

28. When Master’s Theorem cannot be applied
T(n) = 2T(n/2) + n logn
T(n) = 2T(n/2) + n/ logn

29. Up Next
Proving the correctness of Algorithms