Not all algorithms are created equally Insertion of words from a dictionary into a sorted list takes a very long time. Insertion of the same words into a trie takes very little time. How can we express this difference?

Empirical studies We can gather empirical data by running both algorithms on different data sets and comparing their performance.

Issue #1 Empirical data reflects performance on individual cases: the more data points, the better, but no general understanding of algorithm performance is gained

Issue #2 You must code the algorithms to be compared. This can be a non-trivial task.

Issue #3 Empirical performance measures depend on many factors: –implementation language –compiler –execution hardware

Desiderata (things desired as essential – on-line Webster) We want a measure of algorithm performance which: –gives performance bounds as problem size grows large, –is implementation independent, –describes performance of algorithm in the general case, not just specific cases, and –allows performance of different algorithms to be compared.

Asymptotic notation There are many flavors of asymptotic notation. We will study one: the big-Oh notation. big-Oh gives an upper bound typically used to express an upper-bound on the worst-case performance of an algorithm

Definition Given two functions, f and g, mapping natural numbers into non-negative reals, we say that f(n) = O(g(n)) if there exist positive constants c and n 0, such that f(n) n 0

What does this mean? It means that f can’t grow faster than g. We’re interested only in what happens when the input size of the problem is large. g(n) is a bound on the running time of an algorithm whose actual (unknown) runtime is f(n). We guarantee that the time required by the algorithm grows no more quickly than g, as the problem size gets large.

Basic examples f(n)g(n)Is f(n)=O(g(n))? n2n2 n3n3 YES n3n3 n2n2 NO n2n2 n2n2 YES 17n 2 n2n2 YES

A little tougher If you know that f(n) = O(n 3 ), what can you say about f(n) = O(n 2 )? Is it impossible, possible, or necessary?

A little tougher If you know that f(n) = O(n 3 ), what can you say about f(n) = O(n 2 )? Is it impossible, possible, or necessary? It is possible.

How about… If you know that f(n) = O(n 3 ), what can you say about f(n) = O(n 4 )? Is it impossible, possible, or necessary?

How about… If you know that f(n) = O(n 3 ), what can you say about f(n) = O(n 4 )? Is it impossible, possible, or necessary? It is necessary.

One last example… If you know that f(n) = O(n 3 ), what can you say about f(n) = n 4 ? Is it impossible, possible, or necessary?

One last example… If you know that f(n) = O(n 3 ), what can you say about f(n) = n 4 ? Is it impossible, possible, or necessary? It is impossible.