Chapter 10 Algorithm Analysis

 Introduction  Generalizing Running Time  Doing a Timing Analysis  Big-Oh Notation  Analyzing Some Simple Programs – no Subprogram calls  Best-case, Worst-Case and Average Case Analysis  Analyzing Programs  Exercise

 We only analyze correct algorithms  An algorithm is correct ◦ If, for every input instance, it halts with the correct output  Incorrect algorithms ◦ Might not halt at all on some input instances ◦ Might halt with other than the desired answer  Analyzing an algorithm ◦ Predicting the resources that the algorithm requires ◦ Resources include  Memory  Communication bandwidth  Computational time (usually most important)

 Factors affecting the running time ◦ computer ◦ compiler ◦ algorithm used ◦ input to the algorithm  The content of the input affects the running time  typically, the input size (number of items in the input) is the main consideration  E.g. sorting problem  the number of items to be sorted  E.g. multiply two matrices together  the total number of elements in the two matrices  Machine model assumed ◦ Instructions are executed one after another, with no concurrent operations  Not parallel computers

Input Size: n (1)log nnn log nn² n³2ⁿ ³ Comparing the growth of the running time as the input grows to the growth of known functions.

1. n = read input from user 2. sum = 0 3. i = 0 4. while i < n 5. number = read input from user 6. sum = sum + number 7. i = i mean = sum / n T(n), or the running time of a particular algorithm on input of size n, is taken to be the number of times the instructions in the algorithm are executed. Pseudo code algorithm illustrates the calculation of the mean (average) of a set of n numbers: StatementNumber of times executed n+1 5n 6n 7n 81 The computing time for this algorithm in terms on input size n is: T(n) = 4n + 5.

Calculate Lines 1 and 4 count for one unit each Line 3: executed N times, each time four units Line 2: (1 for initialization, N+1 for all the tests, N for all the increments) total 2N + 2 total cost: 6N + 4  O(N)     

Big O notation is used in Computer Science to describe the performance or complexity of an algorithm. Big O specifically describes the worst-case scenario, and can be used to describe the execution time required or the space used (e.g. in memory or on disk) by an algorithm.

O(1) O(1) describes an algorithm that will always execute in the same time (or space) regardless of the size of the input data set.

O(N) O(N) describes an algorithm whose performance will grow linearly and in direct proportion to the size of the input data set.

O(N 2 ) O(N 2 ) represents an algorithm whose performance is directly proportional to the square of the size of the input data set. This is common with algorithms that involve nested iterations over the data set. Deeper nested iterations will result in O(N 3 ), O(N 4 ) etc.

O(2 N ) O(2 N ) denotes an algorithm whose growth will double with each additional element in the input data set. The execution time of an O(2 N ) function will quickly become very large.

Suppose f(n) = n 2 + 3n - 1. We want to show that f(n) = O(n 2 ). f(n) = n 2 + 3n - 1 < n 2 + 3n (subtraction makes things smaller so drop it) <= n 2 + 3n 2 (since n <= n 2 for all integers n) = 4n 2 Therefore, if C = 4, we have shown that f(n) = O(n 2 ).

Show: f(n) = 2n 7 + 6n n 2 – 5 We want to show that f(n) = O(n 7 ). f(n) < 2n 7 + 6n n 2 <= 2n 7 + 6n n 7 = 18n 7 thus, with C = 18 and we have shown that f(n) = O(n 7 )

Exercise 1: Find the Big O for this equation? T(n) = (n) + 4(n-1) + n(n+1)/2 – 1 + 3[n(n-1) / 2]

Algorithm Analysis with some examples Calculate Running Time, T(n) Analysis of Big O notation