1. CS 201 Fundamental Structures of Computer Science
Algorithm Analysis
CS 201 Fundamental Structures of Computer Science

2. Introduction
Once an algorithm is given for a problem and decided to be correct, an important step is to determine how much in the way of resources (such as time or space) the algorithm will require
In this chapter, we shall discuss
How to estimate the time required for a program

3. Mathematical background
The idea of the following definitions is to establish a relative order among functions
Given two functions, there are usually points where one function is smaller than the other one
It does not make sense to claim f(N) < g(N)
Let us compare f(N) = 1000N and g(N) = N2
Thus, we will compare their relative rates of growth

4. Mathematical background
Definition 1:

5. Mathematical background
Definition 2:

6. Mathematical background
Definition 3:

7. Examples

8. Mathematical background
Rule 1:
Rule 2:
Rule 3:

9. Examples
Determine which of f(N) = N log N and g(N) = N1.5 grows faster
 Determine which of log N and N0.5 grows faster
 Determine which of log2 N and N grows faster
 Since N grows faster than any power of a log, g(N) grows faster than f(N)

10. Examples Consider the problem of downloading a file over Internet
Setting up the connection: 3 seconds
Download speed: 1.5 Kbytes/second
If a file is N kilobytes, the time to download is T(N) = N / , i.e., T(N) = O(N)
1500K file takes 1003 seconds
750K file takes 503 seconds
If the connection speed doubles, both times decrease, but downloading 1500K still takes approximately twice the time downloading 750K

11. Typical growth rates Function Name c Constant log N Logarithmic log2 N
Log-squared N
Linear
N log N N2
Quadratic N3
Cubic 2N
Exponential

12. Plots of various algorithms

13. Algorithm analysis
The most important resource to analyze is generally the running time
We assume that simple instructions (such as addition, comparison, and assignment) take exactly one unit time
Unlike the case with real computers (for example, I/O operations take more time compared to comparison and arithmetic operators)
Obviously, we do not have this assumption for fancy operations such as matrix inversion, list insertion, and sorting
We assume infinite memory
We do not include the time required to read the input

14. Algorithm analysis Typically, the input size is the main consideration
Worst-case performance represents a guarantee for performance on any possible input
Average-case performance often reflects typical behavior
Best-case performance is often of little interest
Generally, it is focused on the worst-case analysis
It provides a bound for all inputs
Average-case bounds are much more difficult to compute

15. Algorithm analysis
Although using Big-Theta would be more precise, Big-Oh answers are typically given
Big-Oh answer is not affected by the programming language
If a program is running much more slowly than the algorithm analysis suggests, there may be an implementation inefficiency

16. Algorithm analysis
If two programs are expected to take similar times, probably the best way to decide which is faster is to code them both up and run them!
On the other hand, we would like to eliminate the bad algorithmic ideas early by algorithm analysis
Although different instructions can take different amounts of time (these would correspond to constants in Big-Oh notation), we ignore this difference in our analysis and try to find the upper bound of the algorithm

17. A simple example Estimate the running time of the following function
int sum(int n){
int partialSum = 0;
for (int i = 0; i <= n; i++)
partialSum += i * i * i;
return partialSum; }

18. General rules
Rule 1 – for loops: The running time of a for loop is at most the running time of the statements inside the for loop (including tests) times the number of iterations
Rule 2 – nested loops: Analyze these inside out. The total running time of a statement inside a group of nested loops is the running time of the statement multiplied by the product of the sizes of all the loops
for (i = 0; i < n; i++)
for (j = i; j < n; j++)
k++;

19. General rules Rule 3 – consecutive statements: just add
for (i = 0; i < n; i++)
a[i] = 0;
for (j = 0; j < n; j++)
a[i] += a[j] + i + j;
Rule 4 – if/else: The running time is never more than that of the test plus the larger of that of S1 and S2
if (condition) S1
else S2

20. General rules If there are function calls, they must be analyzed first
If there are recursive functions, be careful about their analyses. For some recursions, the analysis is trivial. But for some others, it could be hard.
long factorial(int n){
if (n <= 1)
return 1;
return n * factorial(n - 1); }

21. General rules Let us analyze the following recursion long fib(int n){
if (n <= 1)
return 1;
return fib(n - 1) + fib(n - 2); }
By induction, it is possible to show that
So the running time grows exponentially. By using a for loop, the running time can be reduced substantially.

22. Sequential search index = -1; for (i = 0; i < N; i++)
if (A[i] == value){
index = i;
break; }
Worst-case: It is the last element or it is not found
N iterations  O(N)
Best-case: It is the first element
1 iteration  O(1)
Average case: We may have N different cases
(N + 1) / 2 iterations  O(N) (successful searches)
N iterations  O(N) (unsuccessful searches)

23. Searching a value in a sorted array using binary search
int binarySearch(int *A, int N, int value){
int low = 0, high = N;
while (low <= high){
int mid = (low + high) / 2;
if (a[mid] < value)
low = mid + 1;
else if (a[mid] > value)
high = mid - 1;
else
return mid; }
return -1;

24. Iterative solution of raising an integer to a power
long pow1(long x, long n){
long result = 1;
for (int i = 1; i <= n; i++)
result = result * x;
return result; }

25. Recursive solution of raising an integer to a power
long pow2(long x, long n){
if (n == 0)
return 1;
if (n == 1)
return x;
if (n % 2 == 0)
return pow2(x*x,n/2);
else
return pow2(x*x,n/2) * x; }