Lecture 44 Section 9.2 Mon, Apr 23, 2007 O, , and  Notations Lecture 44 Section 9.2 Mon, Apr 23, 2007

 Notation Let g : R  R be a function. A function f : R  R is of order at least g, written f(x) is(g(x)), if there exist real numbers M, x0 such that f(x)  Mg(x) for all x  x0.

“Big-Oh” Notation A function f : R  R is of order at most g, written f(x) isO(g(x)), if there exist real numbers M, x0 such that f(x)  Mg(x) for all x  x0.

M1g(x)  f(x)  M2g(x)  Notation A function f : R  R is of order g, written f(x) is(g(x)), if there exist real numbers M1, M2, x0  R such that M1g(x)  f(x)  M2g(x) for all x  x0.

Growth Rates If f(x) is O(g(x)), then the growth rate of f is no greater than the growth rate of g, and maybe less. If f(x) is (g(x)), then the growth rate of f is no less than the growth rate of g, and maybe greater.

Growth Rates Theorem: If f(x) is O(g(x)) and g(x) is O(f(x)), then f(x) is (g(x)). Proof: If f(x) is O(g(x)), then there exist M1, x1  R such that f(x)  M1g(x) for all x  x1. If g(x) is O(f(x)), then there exist M2, x2  R such that g(x)  M2f(x) for all x  x2.

(1/M2)g(x)  f(x)  M1g(x) Growth Rates Let x3 = max(x1, x2). Then (1/M2)g(x)  f(x)  M1g(x) for all x  x3. Therefore, f(x) is (g(x)). We call (f(x)) the growth rate of f.

Growth Rates Theorem: f(x) is (g(x)) if and only if f(x) is (g(x)) and f(x) is O(g(x)). Proof: Show that if f(x) is (g(x)), then g(x) is O(f(x)). Then apply the previous theorem.

Common Growth Rates Growth Rate Example (1) Access an array element (log2 x) Binary search (x) Sequential search (x log2 x) Quick sort (x2) Bubble sort (2x) Factor an integer (x!) Traveling salesman problem

Transitivity of O(f) Theorem: Let f, g, and h be functions from R to R. If f(x) is O(g(x)) and g(x) is O(h(x)), then f(x) is O(h(x)). Proof: If f(x) is O(g(x)), then f(x)  M1g(x) for all x  x1, for some M1 and x1. If g(x) is O(h(x)), then g(x)  M2h(x) for all x  x2, for some M2 and x2.

Transitivity of O(f) Let x3 = max(x1, x2). Then f(x)  M1M2h(x), for all x  x3. Therefore, f(x) is O(h(x)).

Power Functions Theorem: If 0  a  b, then xa is O(xb), but xb is not O(xa). Proof: Since a  b, for all x > 1, xa – b < 1. Therefore, xa < xb for all x > 1. On the other hand, suppose xb  Mxa, for some M and for all x > x0, for some x0.

Power Functions Then, xb – a  M, for all x > x0. But b – a > 0, so xb – a increases without bound. This is a contradiction, so xb is not O(xa).

Application Any polynomial of degree d is For example, O(xn), if d < n. (xn), if n = d. (xn), if d > n. For example, 16x3 – 10x2 + 3x – 12 is (x2), (x3), and O(x4).

Helpful Facts from Calculus Theorem: If then f(x) is (g(x)).

Helpful Facts from Calculus Theorem: If then f(x) is O(g(x)).

Helpful Facts from Calculus Theorem: If then f(x) is (g(x)).

L’Hôpital’s Rule Theorem: If and then

Logarithmic Functions Lemma: Theorem: For all a, b  1, loga x is (logb x). Proof:

Application Since all logarithmic functions are of the same order, it doesn’t matter which base we use.

Logarithmic Functions vs. Power Functions Theorem: For all a > 1, b > 0, loga x is O(xb), but xb is not O(loga x). Proof:

Application Every polynomial function has a faster growth rate than all logarithmic functions. If a function mixes polynomial terms and logarithmic terms, then the polynomial terms “dominate.” For example, 3x2 + 20 log10 (5x + 2) is (x2).

Exponential Functions Theorem: If 1 < a < b, then ax is O(bx), but bx is not O(ax). Proof: It is clear that

Application Any exponential function with base a is (bx), if b < a. (bx), if b = a. O(bx), if b > a. For example, 5(8x) is (2x), (8x), and O(10x). How does 2x compare to 21.1x?

Power Functions vs. Exponential Functions Theorem: For all a > 0, b > 1, xa is O(bx), but bx is not O(xa).

Application If a function is a mix of polynomial terms, logarithmic terms, and exponential terms, the exponential term with the highest base “dominates.” For example, 5x3 + 2 log2 x + 100(3x) + 28(5x) is (5x).

Benefits of O, , and  Clearly, the -notation is useful because it singles out the term that best represents the growth rate of the function (for large x). The O-notation is useful when we can’t pin down the exact growth rate, but we can put an upper bound on it. Similarly, the -notation expresses a lower bound.

Multiplicativity of O(f) Theorem: Let f, g, h, and k be functions from R to R. If f(x) is O(h(x)) and g(x) is O(k(x)), then f(x)g(x) is O(h(x)k(x)). Proof: Suppose f(x)  M1h(x) for all x  x1 and g(x)  M2k(x) for all x  x2, for some M1, M2, x1, x2. Then f(x)g(x)  M1 M2h(x)k(x) for all x  max(x1, x2).

Application We know that log x is O(x) and that x is O(x), so x log x is O(x2). More generally, xd is O(xd log x). xd log x is O(xd + 1).