Analysis of Algorithms (pt 2) (Chapter 4) COMP53 Oct 3, 2007

Best, Worst and Average Case For a particular problem size n, we can find: Best case: the input that can be solved the fastest Worst case: the input that will take the longest Average case: average time for all inputs of the same size.

Methods of Analysis Experimental Studies: Run experiments on implementations of algorithms and record actual time or operation counts. Theoretical Analysis: Determine time or operation counts from mathematical analysis of the algorithm – doesn’t require an implementation (or even a computer)

Theoretical Analysis Uses a high-level description of the algorithm instead of an implementation Characterizes running time as a function of the input size, n. Takes into account all possible inputs Allows us to evaluate the speed of an algorithm independent of the hardware/software environment

Seven Important Functions Seven functions that often appear in algorithm analysis: – Constant  1 – Logarithmic  log n – Linear  n – N-Log-N  n log n – Quadratic  n 2 – Cubic  n 3 – Exponential  2 n In a log-log chart, the slope of the line corresponds to the growth rate of the function

Growth Functions

Reasonable Time Assume GHz machine: 10 6 operations/second Clearly, any algorithm requiring more than operations is impractical MinuteHourDayMonthYear 10 8 ops10 9 ops10 11 ops10 12 ops10 13 ops

Growth Functions < day< month> year

Big-Oh Notation f(n) is O(g(n)) if there are positive constants c and n 0 such that f(n)  cg(n) for n  n 0 Example: 2n  10 є O(n) 2n  10  cn for c  3 and n >= n 0  10

Big-Oh Example Example: n 2 is not O(n) There is no value of c such that n 2  cn, as n  ∞

Comparing Growth Rates f(n) є O(g(n)) means that the growth rate of f(n) is no more than the growth rate of g(n) g(n) is an upper bound on the value of f(n)

Asymptotic Algorithm Analysis The asymptotic analysis of an algorithm determines the running time in big-Oh notation To perform the asymptotic analysis – We find the worst-case number of basic operations as a function of the input size – We express this function with big-Oh notation Example: – We determine that algorithm arrayMax executes at most 8n  2 primitive operations – We say that algorithm arrayMax “runs in O(n) time”

Example: Prefix Averages The i -th prefix average of an array X is average of the first (i  1) elements of X: A[i]  X[0]  X[1]  …  X[i])/(i+1) This has applications to financial analysis

Prefix Averages: Algorithm 1 Algorithm prefixAverages1(X, n) Input array X of n integers Output array A of prefix averages of X A  new array of n integers for i  0 to n  1 do s  X[0] for j  1 to i do s  s  X[j] A[i]  s  (i  1) return A basic operation: addition

Algorithm 1 Analysis Number of additions is 1  2  …  n-1 The sum of the first n-1 integers is (n-1)n  2 Algorithm prefixAverages1 є O(n 2 ) for i  0 to n  1 do for j  1 to i do s  s  X[j]

Prefix Averages (Algorithm 2) Algorithm prefixAverages2(X, n) Input array X of n integers Output array A of prefix averages of X A  new array of n integers s  0 for i  0 to n  1 do s  s  X[i] A[i]  s  (i  1) return A basic operation: addition

Algorithm 2 Analysis Number of additions is 1  1  …  1 = n Algorithm prefixAverages2 є O(n) for i  0 to n  1 do s  s + X[i]

Relatives of Big-Oh big-Omega f(n) є  (g(n)) if f(n)  cg(n) for n  n 0 big-Theta f(n) є  (g(n)) if c’g(n)  f(n)  c’’g(n) for n  n 0

Intuition for Asymptotic Notation f(n) є O(g(n)) f(n) is asymptotically smaller or equal to g(n) f(n) є  (g(n)) f(n) is asymptotically bigger or equal to g(n) f(n) є  (g(n)) f(n) is asymptotically the same to g(n)

Big-Oh

Big-Omega

Big-Theta

Example Uses  and  5n 2 is  (n 2 ) 5n 2 is  (n) 5n 2 is  (n 2 )