1. Asymptotics and Recurrence Equations Prepared by John Reif, Ph.D. Analysis of Algorithms

2. Asymptotics and Recurrence Equations a)Computational Complexity of a Program b)Worst Case and Expected Bounds c)Definition of Asymptotic Equations d)Solution of Recurrence Notation

3. Readings Main Reading Selection: CLR, Chapter 3, 4 and Appendix A

4. Goal To estimate and compare growth rates of functions Ignore constant factors of growth

5. “f(n) is asymptotically equal to g(n)”

6. ”f(n) is little o g(n) ” f(n) is o(g(n)) if

7. ”f(n) is big O g(n) ” f(n) is O(g(n)) if

8. Example of f(n) is O(g(n))

9. “f(n) is order at least g(n)” f(n) is (g(n)) if

10. Example of f(n) is  (g(n))

11. “f(n) is order tight with g(n)” f(n) is (g(n)) if

12. Suppose my algorithm runs in time O(n) Don’t say: “his runs in time O(n 2 ) so is worse” But prove: “his runs in time (n 2 ) so is worse” Must find a worst case input of length n for which his algorithm takes time > cn 2 for all n > n 0

13. Use of O Notation N is O (n 2 ) sometimes written n = O (n 2 ) But n 2 is not O(n) so can’t use identities!  The two sides of the equality do not play a symmetric role

14. Use of Asymptotic Notation Write As Example

15. Convergent Power Sum A polynomial is asymptotically equal to its leading term as x  

16. Sums of Powers for n   Or equivalently

17. Examples 2 nd order asymptotic expansion

18. Asymptotic Expansion of f(n) as n  n 0 If and

19. Bounding Sums by Integrals

20. Bounding Sums by Integrals (cont’d) So Example then

21. Bounding Sums by Integrals (cont’d) So Since So

22. Other Approximations Derived from Integrals Harmonic Numbers

23. Stirling’s Approximation for Factorial Factorial n! = 1· 2 · 3 · · · (n-1) · n So

24. Recurrence Equations (over integers) Homogenous of degree d n > d Given constant coefficients a 1, …, a d initial valuesx 1, x 2, …, x d

25. Example: Fibonacci Sequence n > 2 F n = F n-1 + F n-2 F 0 = 0, F 1 =1

26. Solution of Fibonacci Sequence “golden ratio” where

27. Solution of Fibonacci Sequence (cont’d) Hence

28. A Useful Theorem c > 0, d > 0 If then

29. Proof Is solution

30. Cases

31. Example: Mergesort input list L of length N if N=1 then return L else do let L 1 be the first elements of L let L 2 be the last elements of L M 1  Mergesort (L 1 ) M 2  Mergesort (L 2 ) return Merge (M 1, M 2 )

32. Time Bounds of Mergesort Initial Value T(1) = c 1 for N > 1 for some constants c 1, c 2 > 1

33. Time Bound (cont’d) N > 1 guess Holds if Solution

34. Time Bound (cont’d) N > 1 Transform Variables Recurrence equation:

35. Solve by usual methods for recurrence equations

36. Advanced Material Exact Solution of Recurrence Relations

37. Homogenous Recurrence Relations (no constant additive term) Solve: try Multiply by Get characteristic equation:

38. Case of Distinct Roots Distinct Roots r 1, r 2, …, r d Where r i is dominant root

39. Other Case Roots are not distinct r 1 = r 2 = r 3 Then solutions not independent, so additional terms:

40. Inhomogenous Recurrence Equations Nonzero constant term Solution Method (1) Solve homogenous equation

41. Solution Method 1)Solve homogenous equation 2)Case, add particular solution

42. Solution Method (cont’d) Case, add particular solution 3)Add particular and homogeneous solutions, and solve for constants This is all we usually need!!

43. Asymptotics and Recurrence Equations Prepared by John Reif, Ph.D. Analysis of Algorithms