Asymptotics and Recurrence Equations Prepared by John Reif, Ph.D. Analysis of Algorithms

Asymptotics and Recurrence Equations a)Computational Complexity of a Program b)Worst Case and Expected Bounds c)Definition of Asymptotic Equations d)Solution of Recurrence Notation

Readings Main Reading Selection: CLR, Chapter 3, 4 and Appendix A

Goal To estimate and compare growth rates of functions Ignore constant factors of growth

“f(n) is asymptotically equal to g(n)”

”f(n) is little o g(n) ” f(n) is o(g(n)) if

”f(n) is big O g(n) ” f(n) is O(g(n)) if

Example of f(n) is O(g(n))

“f(n) is order at least g(n)” f(n) is (g(n)) if

Example of f(n) is  (g(n))

“f(n) is order tight with g(n)” f(n) is (g(n)) if

Suppose my algorithm runs in time O(n) Don’t say: “his runs in time O(n 2 ) so is worse” But prove: “his runs in time (n 2 ) so is worse” Must find a worst case input of length n for which his algorithm takes time > cn 2 for all n > n 0

Use of O Notation N is O (n 2 ) sometimes written n = O (n 2 ) But n 2 is not O(n) so can’t use identities!  The two sides of the equality do not play a symmetric role

Use of Asymptotic Notation Write As Example

Convergent Power Sum A polynomial is asymptotically equal to its leading term as x  

Sums of Powers for n   Or equivalently

Examples 2 nd order asymptotic expansion

Asymptotic Expansion of f(n) as n  n 0 If and

Bounding Sums by Integrals

Bounding Sums by Integrals (cont’d) So Example then

Bounding Sums by Integrals (cont’d) So Since So

Other Approximations Derived from Integrals Harmonic Numbers

Stirling’s Approximation for Factorial Factorial n! = 1· 2 · 3 · · · (n-1) · n So

Recurrence Equations (over integers) Homogenous of degree d n > d Given constant coefficients a 1, …, a d initial valuesx 1, x 2, …, x d

Example: Fibonacci Sequence n > 2 F n = F n-1 + F n-2 F 0 = 0, F 1 =1

Solution of Fibonacci Sequence “golden ratio” where

Solution of Fibonacci Sequence (cont’d) Hence

A Useful Theorem c > 0, d > 0 If then

Proof Is solution

Cases

Example: Mergesort input list L of length N if N=1 then return L else do let L 1 be the first elements of L let L 2 be the last elements of L M 1  Mergesort (L 1 ) M 2  Mergesort (L 2 ) return Merge (M 1, M 2 )

Time Bounds of Mergesort Initial Value T(1) = c 1 for N > 1 for some constants c 1, c 2 > 1

Time Bound (cont’d) N > 1 guess Holds if Solution

Time Bound (cont’d) N > 1 Transform Variables Recurrence equation:

Solve by usual methods for recurrence equations

Advanced Material Exact Solution of Recurrence Relations

Homogenous Recurrence Relations (no constant additive term) Solve: try Multiply by Get characteristic equation:

Case of Distinct Roots Distinct Roots r 1, r 2, …, r d Where r i is dominant root

Other Case Roots are not distinct r 1 = r 2 = r 3 Then solutions not independent, so additional terms:

Inhomogenous Recurrence Equations Nonzero constant term Solution Method (1) Solve homogenous equation

Solution Method 1)Solve homogenous equation 2)Case, add particular solution

Solution Method (cont’d) Case, add particular solution 3)Add particular and homogeneous solutions, and solve for constants This is all we usually need!!

Asymptotics and Recurrence Equations Prepared by John Reif, Ph.D. Analysis of Algorithms