Analysys & Complexity of Algorithms Big Oh Notation

Complexity Two kinds of complexity –Space complexity Effects of data type choices on size of data Effects of amount of data (input and variables) –Time Complexity Effects of choices of program design Given 2 algorithms, which is "the best" 2

Big Oh (or Big O) A measure of complexity related to "n" (problem size=amount of data) e.g.: –# of records to process –#of files –# of numbers or times through a loop This is asymptotic analysis –associates n, the problem size, with t, the processing time required to solve the problem 3

Big O Examples x=x+1; –O(1) -> constant degradation Binary Search of a SORTED set of n elements –O(log n) for (i=1;i<n;i++) –O(n) -> linear degradation selection sort, compare two 2D arrays, find duplicates in an UNSORTED list –O(n 2 ) -> quadratic degradation Generate all premutations of n symbols –O(a n ) -> 'a' is some constant independtent of 'n' O(l) < O(log n) < O(n) < O(n log n) < O(n 2 ) < O(a n ) 4

3 cases Best case –minimum path lengths Average case –constant path length Worst case –maximum path length –most useful!! Leads to better design –answers question: will it be good enough tomorrow?? 5

Frequency Counting Examine a piece of code and predict the number of instructions to be executed e.g. predict how many times (max) each statement will run. 6 Inst #CodeFreq count 1for (int i=0; i< n ; i++)n+1 2 { printf ("%d",i);n 3 p = p + i;n }3n+1

Order of magnitude In the previous example: – best case = average case = worst case –Example is based on iteration limit: n To convert Frequency Count to order of magnitude: –pick the most significant term if polynomial –discard constant terms (like the +1) –disregard coefficients (like the 3) –yields worst case path through algorithm Big O (represented as O(n)) O(n) for the previous example 7

Common growth rates 8

Big Oh - Formal Definition f(n)=O(g(n)), Thus, g(n) is an upper bound on f(n) Note: f(n) = O(g(n)) –f(n) has a complexity of g(n)" this is NOT the same as O(g(n)) = f(n) The '=' is not the usual mathematical "=" operator (it is not reflexive) We will see more about this in chapter 6 9 iff {c, n 0 | f(n) <= c  g(n) for alln >= n 0 }