1. Computational Complexity
CSC221 – Data Structures
September 5, 2007

2. How to Determine Computational Complexity of an Algorithm
Step 1: Estimate the number of steps in the most computationally-intensive parts of the program
Loops
Recursion
(Note: We’re estimating worst-case complexity.)

3. How to Determine Computational Complexity of an Algorithm
Step 2: Compare this number of steps to one of the “standard” functions that we use to describe computational complexity
some constant c constant time
N linear
logN logarithmic
NlogN N times logarthmic
N2 quadratic
N3 cubic
2N exponential

4. How to Determine Computational Complexity of an Algorithm
Step 3: Describe the complexity as precisely as possible, using O, θ, o, or Ω.
If T(N) is an estimate of the number of steps needed to execute your function and f(N) is a “standard” function against which you’re comparing f(N), then
T(N) is O(f(N)) means, loosely, that T(N) grows at about the same rate or more slowly than f(N). (“more slowly” is better)
T(N) is θ(f(N)) means, loosely, that T(N) grows at about the same rate as f(N).
T(N) is o(f(N)) means, loosely, that T(N) grows more slowly than f(N).
T(N) is Ω(f(N)) means, loosely, that T(N) grows at about the same rate or more quickly than f(N).

5. Examples What can you say about T(N) = 2N – 3? T(N) is θ(N)
T(N) is O(N)
T(N) is Ω(N)
T(N) is o(N2)
T(N) is Ω(2)
T(N) is O(N2)
T(N) is Ω(N2) false
T(N) is o(2) false

6. What is the most precise thing you can say about T(N) = 2N – 3?
T(N) is θ(N) is the most precise thing we can say.
Often, we say T(N) is O(N), even though T(N) is θ(N) is more precise.

7. What do these statements mean, graphically?
2N-3 grows at about the same rate as N.

8. What do these statements mean, graphically?
N2 grows faster than N.

9. How do you prove your claims, using the definitions of O, θ, Ω, and o?
T(N) = O(f(N)) if there exist positive constants c and n0 such that T(N) ≤ cf(N) when N ≥ n0.
Say that T(N) is 2N – 3 and f(N) is N.
Then the above definition holds true for c = 2 and n0 = 0.
(Note: There isn’t just one c or n0 to make the definition true, but you only need to show one such pair.)

10. Show that N2 + 2N is O(N2) using the definition
The definition holds true for c = 2 and n0 = 2
N2 + 2N = *2 = 8 for N = 2
2 N2 = 2* 22 = 8 for N = 2
N2 + 2N = *3 = 15 for N = 3
2 N2 = 2* 32 = 18 for N = 3

11. You can see that N2 + 2N grows at about the same rate or more slowly than 2 N2
Choose an no after the point where the functions cross. After that point, the values of T(N) are smaller than those of cf(N).

12. You can also prove your claim by using limits.
Determine the
If the limit is
then
T(N) is o(f(N))
finite
T(N) is O(f(N))
Nonzero finite
T(N) is θ(f(N))
nonzero
T(N) is Ω(f(N))

13. Say T(N) = 2N – 3 and f(N) = N
Nonzero finite – therefore,
2N-3 = θ(N)
It is also true that 2N-3 = O(N)
It is also true that 2N-3 = Ω(N)