1. Advanced Algorithms Analysis and Design By Dr. Nazir Ahmad Zafar Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

2. Lecture No. 9 Further Techniques Solving Recurrence Relations (Algorithms Analysis Techniques) Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

3. If a recurrence is of the form Then the characteristics polynomial is It has one factor for left hand side for homogenous part, and factor corresponding to each term on right hand side for non-homogenous part. Once the characteristics polynomial is obtained the recurrence can be solved as before. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Review of Previous Lecture

4. In this lecture following will be covered –Assumptions in solving recurrence –The Substitution Method –The Recursion Tree Method –The Master Theorem –Conclusion Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Today Covered

5. Neglect certain technical details solve recurrences Assume integer arguments to functions –Because running time T (n) is always defined when n is an integer Consequently, for convenience, we shall omit statements of boundary conditions of recurrences and assume that T (n) is constant for small n Recurrence for worst-case time of MERGE-SORT Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Assumption in Solving Recurrence Relation

6. We do not give any explicit value for small n Because changing n, T (1) changes solution to recurrence. Solution typically doesn’t change by more than a constant factor, so order of growth is unchanged Solve recurrences, often omit floors, ceilings, etc. First analyze without these details and later determine whether such assumptions matter or not. Usually it dose not matter but it is important to know when it does and when it does not Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Assumption in Solving Recurrence Relation

7. The Substitution Method Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Methods Solving Recurrence Relation

8. Substitution method has two steps Guess the form of the solution Use mathematical induction to find constants and show that the solution does work The name Substitution comes from the substitution of guessed answer for the function when the inductive hypothesis is applied to smaller values. Method is powerful, but it can be applied only in cases when it is easy to guess the form of answer The substitution method can be used to establish either upper or lower bounds on a recurrence. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Substitution Method

9. Prove that Proof let us suppose that Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Some Important Rules used in this Section

10. Prove that Proof Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Some Important Rules used in this Section

11. Solve the recurrence relation given below. Solution: Dr Nazir A. Zafar Advanced Algorithms Analysis and Design The Substitution Method

12. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design The Substitution Method

13. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design The Substitution Method

14. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design The Substitution Method

15. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design The Substitution Method

16. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design A Hard Example

17. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design A Hard Example

18. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design A Hard Example

19. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design A Hard Example

20. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design A Hard Example

21. Solve the recurrence relation given below. Solution: Dr Nazir A. Zafar Advanced Algorithms Analysis and Design More Hard Example using Substitution Method

22. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Contd..

23. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Contd..

24. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Contd..

25. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Contd..

26. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Contd..

27. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Contd..

28. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Observations

29. Recursion Tree Method Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

30. Although substitution method can provide a sufficient proof that a solution to a recurrence is correct, sometimes difficult to give a good guess. Drawing out a recursion tree, is a straight forward way to devise a good guess. In recursion tree, nodes represent costs of a sub- problems in the set of recursive function invocations. We sum costs within each level of the tree to obtain a set of per-level costs. And then we sum all per-level costs to determine the total cost of all levels of the recursion. Recursion trees are particularly useful when recurrence describes running time of divide and conquer algos. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Recursion Tree Method

31. Recursion tree is best one used to generate a good guess, which is then verified by substitution method When using a recursion tree to generate a good guess, we can often tolerate a small amount of sloppiness since we have to verify it later on. If we are careful when drawing out a recursion tree and summing costs, then we can use a recursion tree as a direct proof of a solution to any recurrence of any problem. Here, we will use recursion trees directly to prove theorem that forms the basis of the master method. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Recursion Tree Method

32. Example: Solve the following recurrence using recurrence tree method Solution: The above recurrence can be written in the form Assumption : We assume that n is exact power of 4. The recurrence tree is given in the next slide Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Recursion Tree Method

33. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Recursion Tree Method

34. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Recursion Tree Method

35.  (n log 4 3) Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Recursion Tree Method

36. Now the total computation cost would be = Cost of Childs + Cost of tree excluding childes = Cost of Child x total number of Childs + Cost of all levels excluding childes level = total number of Childs + sum of costs at each level excluding childes level. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Recursion Tree Method

37. Now total computational cost can be calculated as Where Hence Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Recursion Tree Method

38. Master Theorem Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

39. Lemma 1: Let a  1, b > 1 be constants, f(n) a non-negative function defined on exact power of b by recurrence where i is a positive integer. Then Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Master Theorem

40. Lemma 2: Let a  1, b > 1 be constants, let f(n) be a non-negative function defined on exact power of b. A function g(n) defined over exact powers of b by can be bounded asymptotically for exact powers of b as 1.If 2.If 3.If a.f(n/b) ≤ c.f(n), for some constant c < 1 and for all n  b, then g(n) =  (f(n)) Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Master Theorem

41. Proof: Case 1: Which implies that (1) We are given that:(2) Substituting value from Equation (1) in Equation (2) Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Master Theorem

42. Consider Assume that: Taking log on both sides: Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Master Theorem

43. It is a geometric series with first term 1, common ratio b  and number of terms as log b n. Hence g(n), Hence proved Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Master Theorem

44. Case 2: Which implies that: (3) We are given that:(4) Substituting value from Equation (3) in Equation (4) Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Master Theorem

45. case is proved Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Master Theorem We have already proved that

46. Case 3: Given that: In general: Equivalently: Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Master Theorem

47. We are given that:(5) We have proved that: (6) From equation (5) and (6) (7) Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Master Theorem

48. Since f(n) appears in definition of g(n) and all terms of g(n) are non-negative, we can conclude easily that (8) We have already proved, equation (7), that (9) From Equations (8) and (9) Hence it proves the lemma Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Master Theorem

49. Lemma 3 Let a  1, b > 1 be constants, let f(n) be a non- negative function defined on exact power of b. Define T(n) on exact powers of b by the recurrence where is a positive integer. Then T(n) can be bounded asymptotically for exact powers of b as 1.If 2.If 3.If for some  > 0, and a.f(n/b) ≤ c.f(n) for some constant c < 1and sufficiently large n, then T(n) =  (f(n)) Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Master Theorem

50. Proof: Case 1 Given that By Lemma 4.2: By Lemma 4.3: Hence for case 1 it is proved Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Master Theorem

51. Case 2 Again given that By Lemma 4.2: By Lemma 4.3: Hence for case 2 it is also proved Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Master Theorem

52. Case 3 By Lemma 4.2: By Lemma 4.3: Since Hence This proves the Lemma 3 Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Master Theorem

53. First Order homogenous linear recurrence is in fact a geometric sequence We studied characteristics of second and higher order linear homogenous recurrence relations with constant coefficients Then non-homogenous recurrence relations with constant coefficients Different methods to solve such recurrences As most of the algorithms are recursive therefore to give analysis of such algorithms, we have discussed this powerful technique. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Conclusion