1. Lecture 44 Section 9.2 Mon, Apr 23, 2007
O, , and  Notations
Lecture 44
Section 9.2
Mon, Apr 23, 2007

2.  Notation Let g : R  R be a function.
A function f : R  R is of order at least g, written f(x) is(g(x)), if there exist real numbers M, x0 such that
f(x)  Mg(x)
for all x  x0.

3. “Big-Oh” Notation
A function f : R  R is of order at most g, written f(x) isO(g(x)), if there exist real numbers M, x0 such that
f(x)  Mg(x)
for all x  x0.

4. M1g(x)  f(x)  M2g(x)
 Notation
A function f : R  R is of order g, written f(x) is(g(x)), if there exist real numbers M1, M2, x0  R such that
M1g(x)  f(x)  M2g(x)
for all x  x0.

5. Growth Rates
If f(x) is O(g(x)), then the growth rate of f is no greater than the growth rate of g, and maybe less.
If f(x) is (g(x)), then the growth rate of f is no less than the growth rate of g, and maybe greater.

6. Growth Rates
Theorem: If f(x) is O(g(x)) and g(x) is O(f(x)), then f(x) is (g(x)).
Proof:
If f(x) is O(g(x)), then there exist M1, x1  R such that f(x)  M1g(x) for all x  x1.
If g(x) is O(f(x)), then there exist M2, x2  R such that g(x)  M2f(x) for all x  x2.

7. (1/M2)g(x)  f(x)  M1g(x)
Growth Rates
Let x3 = max(x1, x2).
Then
(1/M2)g(x)  f(x)  M1g(x)
for all x  x3.
Therefore, f(x) is (g(x)).
We call (f(x)) the growth rate of f.

8. Growth Rates
Theorem: f(x) is (g(x)) if and only if f(x) is (g(x)) and f(x) is O(g(x)).
Proof:
Show that if f(x) is (g(x)), then g(x) is O(f(x)).
Then apply the previous theorem.

9. Common Growth Rates Growth Rate Example (1) Access an array element
(log2 x)
Binary search
(x)
Sequential search
(x log2 x)
Quick sort
(x2)
Bubble sort
(2x)
Factor an integer
(x!)
Traveling salesman problem

10. Transitivity of O(f)
Theorem: Let f, g, and h be functions from R to R. If f(x) is O(g(x)) and g(x) is O(h(x)), then f(x) is O(h(x)).
Proof:
If f(x) is O(g(x)), then f(x)  M1g(x) for all x  x1, for some M1 and x1.
If g(x) is O(h(x)), then g(x)  M2h(x) for all x  x2, for some M2 and x2.

11. Transitivity of O(f) Let x3 = max(x1, x2).
Then f(x)  M1M2h(x), for all x  x3.
Therefore, f(x) is O(h(x)).

12. Power Functions
Theorem: If 0  a  b, then xa is O(xb), but xb is not O(xa).
Proof:
Since a  b, for all x > 1, xa – b < 1.
Therefore, xa < xb for all x > 1.
On the other hand, suppose xb  Mxa, for some M and for all x > x0, for some x0.

13. Power Functions Then, xb – a  M, for all x > x0.
But b – a > 0, so xb – a increases without bound.
This is a contradiction, so xb is not O(xa).

14. Application Any polynomial of degree d is For example,
O(xn), if d < n.
(xn), if n = d.
(xn), if d > n.
For example,
16x3 – 10x2 + 3x – 12
is (x2), (x3), and O(x4).

15. Helpful Facts from Calculus
Theorem: If
then f(x) is (g(x)).

16. Helpful Facts from Calculus
Theorem: If
then f(x) is O(g(x)).

17. Helpful Facts from Calculus
Theorem: If
then f(x) is (g(x)).

18. L’Hôpital’s Rule
Theorem: If
and
then

19. Logarithmic Functions
Lemma:
Theorem: For all a, b  1, loga x is (logb x).
Proof:

20. Application
Since all logarithmic functions are of the same order, it doesn’t matter which base we use.

21. Logarithmic Functions vs. Power Functions
Theorem: For all a > 1, b > 0, loga x is O(xb), but xb is not O(loga x).
Proof:

22. Application
Every polynomial function has a faster growth rate than all logarithmic functions.
If a function mixes polynomial terms and logarithmic terms, then the polynomial terms “dominate.”
For example,
3x log10 (5x + 2)
is (x2).

23. Exponential Functions
Theorem: If 1 < a < b, then ax is O(bx), but bx is not O(ax).
Proof:
It is clear that

24. Application Any exponential function with base a is
(bx), if b < a.
(bx), if b = a.
O(bx), if b > a.
For example, 5(8x) is (2x), (8x), and O(10x).
How does 2x compare to 21.1x?

25. Power Functions vs. Exponential Functions
Theorem: For all a > 0, b > 1, xa is O(bx), but bx is not O(xa).

26. Application
If a function is a mix of polynomial terms, logarithmic terms, and exponential terms, the exponential term with the highest base “dominates.”
For example,
5x3 + 2 log2 x + 100(3x) + 28(5x)
is (5x).

27. Benefits of O, , and 
Clearly, the -notation is useful because it singles out the term that best represents the growth rate of the function (for large x).
The O-notation is useful when we can’t pin down the exact growth rate, but we can put an upper bound on it.
Similarly, the -notation expresses a lower bound.

28. Multiplicativity of O(f)
Theorem: Let f, g, h, and k be functions from R to R. If f(x) is O(h(x)) and g(x) is O(k(x)), then f(x)g(x) is O(h(x)k(x)).
Proof:
Suppose f(x)  M1h(x) for all x  x1 and g(x)  M2k(x) for all x  x2, for some M1, M2, x1, x2.
Then f(x)g(x)  M1 M2h(x)k(x) for all x  max(x1, x2).

29. Application
We know that log x is O(x) and that x is O(x), so x log x is O(x2).
More generally,
xd is O(xd log x).
xd log x is O(xd + 1).