Chapter 3 Growth of Functions

(defined as a set of functions) Used to denote worst case, as in insertion sort T(n) = O(n2) Allows for possibility that f(n) = cg(n) for large n To prove T(n) = O(g(n)), find a combination of c and n0 (may not be unique) for which definition is satisfied.

Used to denote best case run time Allows for possibility that f(n) = cg(n) for large n To prove T(n) = W(g(n)), find a combination of c and n0 (may not be unique) for which definition is satisfied.

Theorem 3.1: T(n) = Q(g(n)) iff T(n) = O(g(n)) and T(n) = W(g(n)) At large n, f(n) and g(n) are equal within a constant factor To prove T(n) = Q(g(n)), use Theorem 3.1,

Important points Constants in the definitions of O, W, and Q may depend on the choice of n0 but for n > n0 they cannot depend on n. Even though O(g(n)) is formally defined by a set of functions, we write f(n) = O(g(n)) to mean that f(n) is a member of the set. Same convention used for W, and Q

More important points For all forms of asymptotic notation, f(n)=O(g(n)) f(n)=W(g(n)) f(n)=Q(g(n)) f(n)=o(g(n)) to be defined f(n)=w(g(n)) to be defined the argument of asymptotic notation is the bound and the quantity it is equal to is the function being bounded.

Informal proofs of Q notation If we have an equation for f(n), and g(n) is the dominant term in that equation at large values of n, then f(n) = Q(g(n)) Example: Show by informal proof that (n+a)b = Q(nb) real a & b a not zero b > 0 Binomial expansion: (n+a)b = Cbnb + Cb-1nb-1+…+C0 Since nb is dominant term at large n, (n+a)b = Q(nb) by informal proof

 Assignment 3: due 1/14/19 Use Stirling’s approximation (Eq(3.18) text p57) to give an informal proof that lg(n!)=Q(nlgn). Show all steps.

Ex 3.1-2 text p52 Formal proof that (n+a)b = Q(nb) real a and b a not zero b > 0 Strategy: formal proof of (n+a) = O(n) Generalize to (n+a)b = O(nb) by algebra Similarly for (n+a)b = W(nb) Use theorem 3.1

To prove (n+a)=O(n), find c>0 and n0 such that 0 < (n+a) < cn for all n>n0 Need an inequality with n+a < something a < |a| n+a < n+|a| < cn This string of inequalities says that if we find c>0 and n0 such that n+|a| < cn for all n > n0, then they will also work for n+a and the condition (n+a) < cn in the definition of bigO is satisfied. Hence we focus on inequality n+|a| < cn in the search c and n0. |a| < cn-n |a| < (c-1)n To use values of n to make this inequality true, c > 1 If we choose c = 2, then |a| < (c-1)n becomes n > |a| n0 = |a| is large enough to make (n+a) > 0 if a is negative. Conditions for Big O are satisfied

To prove (n+a)=W(n), find c>0 and n0 such that 0 < cn < (n+a) for all n>n0 Need an inequality with n+a > something a > -|a| n+a > n-|a| > cn This string of inequalities says that if we find c>0 and n0 such that n-|a| > cn for all n > n0, then they will also work for n+a and the condition (n+a) > cn in the definition of big W is satisfied. Hence we focus on inequality n+|a| > cn in the search c and n0. n-cn > |a| n(1-c) > |a| To use values of n to make this inequality true, c < 1 If we choose c = 1/2, then n(1-c) > |a| becomes n > 2|a| Both c and n are positive, so cn > 0 Conditions for Big W are satisfied.

There exist c=2 such that 0 < (n+a) < cn for all n>n0 = |a| Since b>0, 0 < (n+a)b < cbnb There exist d=2b such that 0 < (n+a)b < dnb for all n>n0 = |a| Therefore (n+a)b = O(nb) There exist c=1/2 such that 0 < cn < (n+a) for all n>n0 =2|a| Since b>0, 0 < cbnb < (n+a)b There exist d=(1/2)b such that 0 < dnb < (n+a)b for all n>n0 = 2|a| Therefore (n+a)b = W(nb) Therefore (n+a)b = Q(nb) by Theorem 3.1

Assignment 4: due 1/16/19 Ex 3.1-1 text p52 If f(n) and g(n) are asymptotically non-negative, show by the definitions of big O and W and theorem 3.1 that max(f(n),g(n)) = Q(f(n)+g(n)). Note: at any n, max(f(n),g(n)) is the larger of f(n) and g(n) Address both inequalities in the definitions of big O and W Use given information to define n0 Use non-negative property of f(n) and g(n) to find constants in the definitions of big O and W

Antonio’s Office Hours Monday 2-4pm in TFLO 145 TuTh 2-4pm TFLO 252

Prove that Sk (O(fk(n))) = O(Sk fk(n)) when fk(n) is asymptotically non-negative for all k. {fk(n)} is a set of functions of n that are non-negative for all n>n0 Range of k-values is not specified. What is the bound? What is being bounded? If true, result say “big O passes through sums” (i.e. a sum of big Os is big O of the sum of the bounds) Let gk(n)=O(fk(n)) and use definition of Big O(fk(n))

Little o notation f(n) = o(g(n)) means f(n) is a member of a set of function such that for any c > 0 (no matter how small) 0 < f(n) < cg(n) all n > n0 Can only be true if f(n) is insignificant relative to g(n) at large n Equivalent to limit f(n)/g(n) = 0 n →∞ g(n) is strictly an asymptotic upper bound on f(n) Asymptotic equality (within a constant) of f(n) and g(n) is excluded 2n = o(n2) 2n2 is not o(n2) (see text p 50)

Prove that (lg(n))k = o(ne) What is f(n)? What is g(n)? What is limit f(n)/g(n)? n →∞

Prove that nk = o(cn) What is f(n)? What is g(n)? What is limit f(n)/g(n)? n →∞

Polynomials are exponentially dominated

Little omega notation f(n) = w(g(n)) means f(n) is a member of a set of function such that for any c > 0 (no matter how large) 0 < cg(n) < f(n) all n > n0 Can only be true if g(n) is insignificant relative to f(n) at large n Equivalent to limit g(n)/f(n) = 0 n →∞ g(n) is strictly an asymptotic lower bound on f(n) Asymptotic equality of f(n) and g(n) is excluded n2/2 = w(n) n2/2 is not w(n2) text p51

Prove that 2n = w(2n/2) What is f(n)? What is g(n)? What is limit g(n)/f(n)? n →∞

Homework Assignment 5: due 1/23/19 Use Stirling’ approximation, Eq(3.18) p 57, to show that n! = o(nn) and n! = w(2n)

Prove f(n) = o(g(n)) implies f(n) = O(g(n)) If for any c > 0, 0 < f(n) < cg(n) then there exist some c > 0 such that 0 < f(n) < cg(n) Similarly f(n) = w(g(n)) implies f(n) = W(g(n)) What is wrong with these arguments?

Little o notation f(n) = o(g(n)) means f(n) is a member of a set of function such that for any c > 0 (no matter how small) 0 < f(n) < cg(n) all n > n0 Can only be true if f(n) is insignificant relative to g(n) at large n Equivalent to limit f(n)/g(n) = 0 n →∞ g(n) is strictly an asymptotic upper bound on f(n) Asymptotic equality (within a constant) of f(n) and g(n) is excluded 2n = o(n2) 2n2 is not o(n2) text p 50

prob 3-2 p 61: work in class A->f(n) B->g(n) Which is true O, o, W, w, Q Can be more than 1 or none

Properties of asymptotic notations (text pp51-52)

Important math from Section 3.2 Text pp53-58

See text p54 for useful properties Floors and Ceilings Greatest integer less than or equal to x Least integer greater than or equal to x See text p54 for useful properties

Asymptotic notation in equations Alone on right side: n = o(n2) set definition Part of an equation: f(n) = 2n2 + Q(n) = Q(n2) In recursive relation T(n) = T(n-1) + Q(n) says overhead is linear