O, , and  Notations Lecture 39 Section 9.2 Wed, Mar 30, 2005

 Notation Let g : R  R be a function. A function f : R  R is “of order at least g,” written “f(x) is  (g(x)),” if there exist real numbers M, x 0 such that  f(x)   M  g(x)  for all x  x 0.

“Big-Oh” Notation A function f : R  R is “of order at most g,” written “f(x) is  O(g(x)),” if there exist real numbers M, x 0 such that  f(x)   M  g(x)  for all x  x 0.

 Notation A function f : R  R is “of order g,” written “f(x) is  (g(x)),” if there exist real numbers M 1, M 2, x 0  R such that M 1  g(x)    f(x)   M 2  g(x)  for all x  x 0.

Growth Rates If f(x) is O(g(x)), then the growth rate of f is no greater than the growth rate of g, and maybe less. If f(x) is  (g(x)), then the growth rate of f is no less than the growth rate of g, and maybe greater.

Growth Rates Theorem: If f(x) is O(g(x)) and g(x) is O(f(x)), then f(x) is  (g(x)). Proof: If f(x) is O(g(x)), then there exist M 1, x 1  R such that  f(x)   M 1  g(x)  for all x  x 1. If g(x) is O(f(x)), then there exist M 2, x 2  R such that  g(x)   M 2  f(x)  for all x  x 2.

Growth Rates Let x 3 = max(x 1, x 2 ). Then (1/M 2 )  g(x)    f(x)   M 1  g(x)  for all x  x 3. Therefore, f(x) is  (g(x)). We call  (f(x)) the growth rate of f.

Growth Rates Theorem: f(x) is  (g(x)) if and only if f(x) is  (g(x)) and f(x) is O(g(x)). Proof: Show that if f(x) is  (g(x)), then g(x) is O(f(x)). Then apply the previous theorem.

Common Growth Rates Growth RateExample  (1) Access an array element  (log 2 x) Binary search (x)(x) Sequential search  (x log 2 x) Quick sort (x2)(x2) Bubble sort  (2 x ) Factor an integer  (x!) Traveling salesman problem

Transitivity of O(f) Theorem: Let f, g, and h be functions from R to R. If f(x) is O(g(x)) and g(x) is O(h(x)), then f(x) is O(h(x)). Proof: If f(x) is O(g(x)), then  f(x)   M 1  g(x)  for all x  x 1, for some M 1 and x 1. If g(x) is O(h(x)), then  g(x)   M 2  h(x)  for all x  x 2, for some M 2 and x 2.

Transitivity of O(f) Let x 3 = max(x 1, x 2 ). Then  f(x)   M 1 M 2  h(x) , for all x  x 3. Therefore, f(x) is O(h(x)).

Power Functions Theorem: If 0  a  b, then x a is O(x b ), but x b is not O(x a ). Proof: Since a  b, for all x > 1, x a – b < 1. Therefore, x a 1. On the other hand, suppose x b  Mx a, for some M and for all x > x 0, for some x 0.

Power Functions Then, x b – a  M, for all x > x 0. But b – a > 0, so x b – a increases without bound. This is a contradiction, so x b is not O(x a ).

Application Any polynomial of degree d is  (x m ), if m < d.  (x d ). O(x n ), if n > d. For example, 16x 3 – 10x 2 + 3x – 12 is  (x 2 ),  (x 3 ), and O(x 4 ).

Logarithmic Functions Lemma: Proof: It true by the definition of the log a function. Also, it follows from the fact that the functions f(x) = a x and g(x) = log a x are inverses of each other.

Logarithmic Functions Theorem: For all a, b  1, log a x is  (log b x). Proof:

Application Since all logarithmic functions are of the same order, it doesn’t matter which base we use.

Logarithmic Functions vs. Power Functions Theorem: For all a > 1, b > 0, log a x is O(x b ), but x b is not O(log a x). Proof (using Calc II):

Application Every polynomial function has a faster growth rate than all logarithmic functions. If a function mixes polynomial terms and logarithmic terms, then the polynomial tems “dominate.” 3x log 10 (5x + 2) is  (x 2 ).

Exponential Functions Theorem: If 1 < a < b, then a x is O(b x ), but b x is not O(a x ). Proof: Clearly, a x < b x for all x  1. Therefore, a x is O(b x ). We must show that b x is not O(a x ). Suppose that it is. Then b x  Ma x for all x > x 0 for some M, x 0.

Exponential Functions Then But this contradicts that assumption that the inequality holds for all x > x 0.

Application Any exponential function with base a is  (b x ), if b < a.  (a x ). O(b x ), if b > a. For example, 5  8 x is  (2 x ),  (8 x ), and O(10 x ).

Power Functions vs. Exponential Functions Theorem: For all a > 0, b > 1, x a is O(b x ), but b x is not O(x a ).

Application If a function is a mix of polynomial terms, logarithmic terms, and exponential terms, the exponential term with the highest base “dominates.” 5x log 2 x  3 x + 28  5 x is  (5 x ).

Benefits of O, , and  Clearly, the  -notation is useful because it singles out the term that best represents the growth rate of the function (for large x). The O-notation is useful when we can’t pin down the exact growth rate, but we can put an upper bound on it. Similarly, the  -notation expresses a lower bound.

Multiplicativity of O(f) Theorem: Let f, g, h, and k be functions from R to R. If f(x) is O(h(x)) and g(x) is O(k(x)), then f(x)g(x) is O(h(x)k(x)). Proof: Suppose  f(x)   M 1  h(x)  for all x  x 1 and  g(x)   M 2  k(x)  for all x  x 2, for some M 1, M 2, x 1, x 2. Then  f(x)g(x)   M 1 M 2  h(x)k(x)  for all x  max(x 1, x 2 ).

Application We know that log x is O(x) and that x is O(x), so x log x is O(x 2 ). More generally, x d is O(x d log x). x d log x is O(x d + 1 ).