1. Estimating Algorithm Performance
How to determine how much time an algorithm A uses to solve problem X?
Depends on input; use input length as parameter. Determine function f(n) representing cost.
Code it and use timer running on many inputs
Analyze steps of algorithm, estimating amount of work each step takes (Algorithm Analysis).
Coding often impractical at early design stages.
Algorithm Analysis: determining how long an algorithm will take to solve a problem USING A MODEL.
TCSS 342 Intro Alg. Anal. v 1.0

2. RAM Model for Algorithm Analysis
Typically use a simple model for basic operation costs.
RAM (Random Access Machine) model
RAM model has all the basic operations: +, -, *, / , =, comparisons.
fixed sized integers (e.g., 32-bit)
infinite memory
All (basic) operations take exactly one time unit to execute.
analysis= determining run-time efficiency.
model = estimate, not meant to represent everything in real-world
TCSS 342 Intro Alg. Anal. v 1.0

3. Critique of the model Weaknesses: Strengths:
not all operations take the same amount of time in a real machine
does not account for page faults, disk accesses, etc.
Strengths:
simple
easier to prove things about the model than the real machine
can estimate algorithm behavior on any hardware/software
model = approximation of real world.
can predict run-time of algorithm on machine.
TCSS 342 Intro Alg. Anal. v 1.0

4. Why models? model real world
Idea: (Useful) statements using the model translate into useful statement about real computers
TCSS 342 Intro Alg. Anal. v 1.0

5. Relative rates of growth
Distinguish between fast- and slow-growing functions.
100n
n3 + 2n2
Motivation:
we usually care only about algorithm performance when there are large number of inputs.
We usually don’t care about small changes in run-time performance. (inaccuracy of estimates
make small changes less relevant). Consider algorithms with slow growth rate better than
those with fast growth rates.
TCSS 342 Intro Alg. Anal. v 1.0

6. Big-Oh notation
Defn: T(N) = O(f(N)) if there are positive constants c and n0 such that T(N)  c f(N) for all N  n0 .
Idea: We are concerned with how the function grows when N is large. We are not picky about constant factors: coarse distinctions among functions.
Lingo: “T(N) grows no faster than f(N).”
Important (not in Lewis & Chase book). Write on board. (for next slide).
TCSS 342 Intro Alg. Anal. v 1.0

7. Examples n = O(2n) ? 2n = O(n) ? n = O(n2) ? n2 = O(n) ? n = O(1) ?
10 log n = O(n) ?
214n + 34 = O(2n2 + 8n) ?
TCSS 342 Intro Alg. Anal. v 1.0

8. Preferred big-Oh usage
If f(N) = 5N, then
f(N) = O(N5)
f(N) = O(N3)
f(N) = O(N)  preferred
f(N) = O(N log N)
Ignore constant factors and low order terms.
T(N) = O(N), not T(N) = O(5N).
T(N) = O(N3), not T(N) = O(N3 + N2 + N log N).
Bad style: f(N)  O(g(N)). Wrong: f(N)  O(g(N)).
TCSS 342 Intro Alg. Anal. v 1.0

9. figure 1.11 asymptotic complexity of selected functions
TCSS 342 Intro Alg. Anal. v 1.0

10. figure 1.12 Increase in problem size with a ten-fold increase in processor speed
Book has errata, including on this figure. 2^n = exponential complexity. 2n
TCSS 342 Intro Alg. Anal. v 1.0

11. Big Omega, Theta
Defn: T(N) = (g(N)) if there are positive constants c and n0 such that T(N)  c g(N) for all N  n0 .
Lingo: “T(N) grows no slower than g(N).”
Defn: T(N) = (h(N)) if and only if T(N) = O(h(N)) and T(N) = (h(N)).
Big-Oh, Omega, and Theta establish a relative order among all functions of N.
TCSS 342 Intro Alg. Anal. v 1.0

12. Intuition, little-Oh
Defn: T(N) = o(p(N)) if T(N) = O(p(N)) and T(N)  (p(N)).
notation
intuition
O (Big-Oh) 
 (Big-Omega) 
 (Theta) =
o (little-Oh)
<
TCSS 342 Intro Alg. Anal. v 1.0

13. More about Asymptotics
Fact: If f(N) = O(g(N)), then g(N) = (f(N)).
Proof: Suppose f(N) = O(g(N)). Then there exist constants c and n0 such that f(N)  c g(N) for all N  n0.
Then g(N)  f(N) for all N  n0, and so g(N) = (f(N)).
TCSS 342 Intro Alg. Anal. v 1.0

14. More terminology Suppose T(N) = O(f(N)).
f(N) is an upper bound on T(N).
T(N) grows no faster than f(N).
Suppose T(N) = (g(N)).
g(N) is a lower bound on T(N).
T(N) grows at least as fast as g(N).
If T(N) = o(h(N)), then we say that T(N) grows strictly slower than h(N).
TCSS 342 Intro Alg. Anal. v 1.0

15. Facts about Big-Oh If T1(N) = O(f(N)) and T2(N) = O(g(N)), then
T1(N) + T2(N) = O(f(N) + g(N)).
T1(N) * T2(N) = O(f(N) * g(N)).
If T(N) is a polynomial of degree k, then T(N) = (Nk).
log k N = O(N), for any constant k.
TCSS 342 Intro Alg. Anal. v 1.0

16. Techniques Algebra ex. f(N) = N / log N, g(N) = log N.
same as asking which grows faster, N or log 2 N.
Evaluate .
limit is
Big-Oh relation
f(N) = o(g(N))
c  0
f(N) = (g(N)) 
g(N) = o(f(N))
no limit
no relation
TCSS 342 Intro Alg. Anal. v 1.0

17. Techniques, cont’d L’Hôpital’s rule: If and , then .
example: f(N) = N, g(N) = log N.
Use L’Hôpital’s rule.
f’(N) = 1, g’(N) = 1/N.
 g(N) = o(f(N)).
TCSS 342 Intro Alg. Anal. v 1.0

18. Kinds of Algorithm Analysis
Express the running time as f(N), where N is the size of the input.
Worst case: your enemy gets to pick the input.
Average case: need to assume a probability distribution on the inputs.
Amortized: your enemy gets to pick the inputs/operations, but you only have to guarantee speed over a large number of operations.
However, even with input size N, cost of an algorithm could vary on different input.
TCSS 342 Intro Alg. Anal. v 1.0

19. Search avg. case, if found: n/2 accesses
// return index if x is found
// in array a;
// or NOT_FOUND if not found
public int SequentialSearch(
int[] a, int x) {
int i;
for (i = 0; i < a.length; i++) {
if (a[i] == x)
return i; }
return NOT_FOUND;
worst case: O(n)
avg. case, if found: n/2 accesses
avg. case, if not found: n accesses
 constant work
 in loop
TCSS 342 Intro Alg. Anal. v 1.0

20. Binary Search  constant work  in loop // Given a sorted array a and
// a key x, return index if x is found
// in array a
// or NOT_FOUND if not found
public int BinarySearch(int[] a, int x){
int low = 0;
int high = a.length – 1;
while (low <= high) {
int mid = (low + high) / 2;
if (a[mid] < x) {
low = mid + 1;
} else if (a[mid] > x) {
high = mid – 1;
} else {
return mid; }
return NOT_FOUND;
 constant work
 in loop
TCSS 342 Intro Alg. Anal. v 1.0

21. Analysis Intuitive argument about the running time:
number of elements to consider (high – low + 1) is halved each iteration
high – low + 1 = n at start
The algorithm iterates at most log2 n.
 O(log n) steps to execute. …
mid
n-1
TCSS 342 Intro Alg. Anal. v 1.0