1. Asymptotic Analysis Motivation Definitions Common complexity functions
Example problems

2. Motivation
Lets agree that we are interested in performing a worst case analysis of algorithms
Do we need to do an exact analysis?

3. Exact Analysis is Hard! What can we really say about this function?
We’re more interested in how fast a function increases than what its exact value is.
As long as the exact speeds are reasonable for small n, it is more important to see what happens as n increases.
But this function jumps around too much!!!
[ draw close upper and lower bounds. ]
ALMOST N-Squared…. ALMOST.

4. Even Harder Exact Analysis

5. Simplifications
Ignore constants
Asymptotic Efficiency

6. Why ignore constants?
Implementation issues (hardware, code optimizations) can speed up an algorithm by constant factors
We want to understand how effective an algorithm is independent of these factors
Simplification of analysis
Much easier to analyze if we focus only on n2 rather than worrying about 3.7 n2 or 3.9 n2

7. Asymptotic Analysis
We focus on the infinite set of large n ignoring small values of n
Usually, an algorithm that is asymptotically more efficient will be the best choice for all but very small inputs.
infinity

8. “Big Oh” Notation O(f(n)) =
{g(n) : there exists positive constants c and n0 such that 0 <= g(n) <= c f(n) }
What are the roles of the two constants?
n0: c:

9. Set Notation Comment O(f(n)) is a set of functions.
However, we will use one-way equalities like
n = O(n2)
This really means that function n belongs to the set of functions O(n2)
Incorrect notation: O(n2) = n
Analogy
“A dog is an animal” but not “an animal is a dog”

10. Three Common Sets
g(n) = O(f(n)) means c  f(n) is an Upper Bound on g(n)
g(n) = (f(n)) means c  f(n) is a Lower Bound on g(n)
g(n) = (f(n)) means c1  f(n) is an Upper Bound on g(n)
and c2  f(n) is a Lower Bound on g(n)
These bounds hold for all inputs beyond some threshold n0.
Asymptotic or Big-O notation.
O Omega Sigma

11. O(f(n))

12. (f(n))

13. (f(n))

14. (f(n))

15. O(f(n)) and (f(n))

16. Example Function
f(n) = 3n n + 6

17. Quick Questions c n0 3n2 - 100n + 6 = O(n2) 3n2 - 100n + 6 = O(n3)
[ Do on blackboard!!!!! ]
f(n) = 3n n + 6

18. “Little Oh” Notation o(g(n)) =
{f(n) : for any positive constant c >0,
there exists a constant n0 > 0 such that
0 <= f(n) < cg(n) for all n >= n0}
Intuitively, limn f(n)/g(n) = 0
f(n) < c g(n)

19. Two Other Sets
g(n) = o(f(n)) means c  f(n) is a strict upper bound on g(n)
g(n) = w(f(n)) means c  f(n) is a strict lower bound on g(n)
These bounds hold for all inputs beyond some threshold n0 where n0 is now dependent on c.
Asymptotic or Big-O notation.
O Omega Sigma

20. Common Complexity Functions
n 110-5 sec 210-5 sec 310-5 sec 410-5 sec 510-5 sec 610-5 sec
n sec sec sec sec sec sec
n sec sec sec sec sec sec
n sec sec sec min min min
2n sec sec min days years cent
3n 0.59sec min years cent 2108cent 1013cent
log2 n 310-6 sec 410-6 sec 510-6 sec 510-6 sec 610-6 sec 610-6 sec
n log2 n 310-5 sec 910-5 sec sec sec sec sec

21. Complexity Graphs
log(n)

22. Complexity Graphs
n log(n) n
log(n)

23. Complexity Graphs
n10 n3 n2
n log(n)

24. Complexity Graphs (log scale)3n nn
n20 2n
n10
1.1n

25. Logarithms blogbx = x logab = b loga
Properties:
bx = y  x = logby
blogbx = x
logab = b loga
logax = c logbx (where c = 1/logba)
Questions:
* How do logan and logbn compare?
* How can we compare n logn with n2?

26. Example Problems 1. What does it mean if:
f(n)  O(g(n)) and g(n)  O(f(n)) ???
2. Is 2n+1 = O(2n) ?
Is 22n = O(2n) ?
3. Does f(n) = O(f(n)) ?
4. If f(n) = O(g(n)) and g(n) = O(h(n)),
can we say f(n) = O(h(n)) ?