1. CS 213: Data Structures and Algorithms
Abhiram G. Ranade
Analysis of the time taken by an algorithm

2. We would like to write programs that are ...
Correct
Fast
Use less memory
Are easy to understand
Are easy to modify if needed
...
Today: Fast

3. Some terms (Computational) Problem:
Something like “Find GCD”, Sorting, matrix multiplication.
Map from set of acceptable input values to desired output values.
Acceptable input value: (Problem) Instance
Instances of “Find GCD”
all pairs of positive integers
Instances of Sorting: sequences of numbers
“Size” of the instance:
Formal definition: number of bits needed to specify the instance
Informal: a convenient parameter or set of parameters from which the size can be calculated.
Size(Sorting): number of keys
Size(Matrix multiplication): number of rows, columns of the matrices.

4. When do you call a program fast?
Compile and run – if it finishes quickly, it is fast!
It might run fast because the computer on which it runs is better.
Most programs run fast if you give as input a small instance.
Even among instances of the same length, some may take more time than others.

5. Our (very crude) estimate of time taken by a program
Non crude estimate of time taken by a program = number of operations performed by the program
The number of operations is measured as a function of n, the size of the instance.
Our estimate: We only say something like
Program X takes time proportional to n.
Program Y takes time proportional to n2
If for the same n different instances take different times, we consider the largest of those.
“Time taken by by Program Y (for any instance) is at most proportional to n2”

6. Notation “Time taken is O(n)” : Similarly O(n2), O(nlogn), ...
Time taken by the worst instance is at most kn for some k.
Time taken by all instances is at most kn for some k.
Similarly O(n2), O(nlogn), ...

7. Comparing algorithms
If one algorithm takes time proportional to n and the other to n2 then the former is considered better.
For large enough n, the time for the former will actually be small.
Example: 100n < n2 for n>100.
Computers solve large problem instances ⇒ so more interest in large n.

8. Estimating time taken: Example 1
Vector addition.
for(int i=0; i<n; i++)
c[i] = a[i] + b[i];
In each iteration some fixed number of operations are performed.
+, ==, [], <,
n iterations.
Total time: proportional to n, or O(n)

9. Example 2: Determine if an array A[0..n-1] contains a given number t
bool found = false;
for(int i=0; i<n; i++){
if(A[i] == t) {
found = true;
break; }
t is present in array at position i ⇒ i iterations executed.
t is not present in array ⇒ n iterations executed
(Worst) Time = what is needed for n iterations + time for statement 1.
= cn + c’
≤ c’’n, i.e. at most proportional to n, or O(n)

10. Example 3: multiplying nxn square matrices
for(int i=0; i<n; i++){
for(int j=0; i<n; j++){
c[i][j] = 0;
for(int k=0; k<n; k++){
c[i][j] += a[i][k] * b[k][j] }
Time = count how many times each statement is executed.
= n, n2, n3, n3 times
Each statement takes some fixed time to execute once
Time = an + bn2 + cn3
≤ (a+b+c)n3
= proportional to n3 = O(n3)

11. Example 4: what if there is a function call
// find if t is present in A[0..n-1]
for(int i=0; i<m; i++){
cin >> t;
cout << present(A,n,t); }
Time = m * (c + time for present(A,n,t))
We need to know time for present(A,n,t). say O(n)
So total time = O(mn)
Expressing time in terms of two parameters is OK.

12. Example 5: recursion
int factorial(int n){ if(n==0) return 1; else return n * factorial(n-1); } Standard idea: “Let T(n) denote time to compute factorial(n).” T(n) : we are defining an infinite number of variables. From the code: T(0) = fixed number, independent of n. T(n) = fixed number + time to execute factorial(n-1) = c + T(n-1) = c + c + T(n-2) = ... = cn + T(0) = cn + k ≤ (c+k)n, i.e. O(n)

13. Example 6: Recursive GCD
int gcd(int m, int n){ if(m % n == 0) return n; else return gcd(n, m%n); } T(n) = time for gcd with second argument n, no matter what first argument is. T(n) = c’ + T(n’), where n>n’ = m mod n. = 2c’ + T(n’’) where n’’ = n mod n’ n = kn’ + n’’ ... Definition of mod ≥ n’ + n’’ ... Because n > n’

14. n > 2n’’ ... Because n’’ < n’
If we resubstitute 2logn times in T(n), we get
T(n) = 2c log n + T(n) where n < n/2log n = 1.
n < 1.
But we never call gcd with second argument < 1.
So number of recursive calls < 2logn
T(n) < 2clog n, i.e. O(log n)

15. Example 7: Analysis of mergesort
Read chapter 16 of book.

16. Remarks
We only ask what function of n is time proportional to. (at most).
O(f(n)) = at most proportional to f(n)
In iterative programs to estimate time taken we just look at the statement that is executed most.
Recursive algorithms are analysed by writing down and solving recurrences.
Recurrence = Relationship between T(n) and T(smaller values), e.g. T(n) ≤ c + T(n-1)
We are only getting an upper bound on the time, the actual time can be lot smaller, which our analysis may not sometimes reveal.

17. Insertion sort
Basic idea: as you read, keep read values in an array in a sorted manner.
At the beginning of the i+1th iteration:
A[0..i-1] holds i keys in non-decreasing order.
In i+1th iteration
We read 1 key.
We rearrange keys and find a place to insert the new key so that the new contents are also sorted.
Write the code for insertion sort with invariants.
Analyse its running time.

18. Solution
The code, alongwith required invariants and additional comments, is also placed on the webpage.
It will be confusing how much of this to write; but at least the invariants should be written.
We could have broken up the inner loop into 2 steps, first find j where to insert x, then move A[j+1..i-1] to A[j+2..i], then store A[j] = x;
The code uses A[i..j] to mean elements A[i],A[i+1],...,A[j]
If i>j, then A[i..j] means the empty set.
If we write A[i..j] > x, we mean every element is larger.
If i > j, then A[i..j] is empty, and A[i..j] > x is trivially true.
Running time:
If the elements are given in order, then time will be O(n).
If elements are in reverse order, then time will be O(n2). No other input will take longer.
So overall O(n2).