CS 307 Fundamentals of Computer Science 1 Asymptotic Analysis of Algorithms (how to evaluate your programming power tools) based on presentation material by Mike Scott Honors CS 102 Sept. 4, 2007

CS 307 Fundamentals of Computer Science 2 Algorithmic Analysis  A technique used to characterize the execution behavior of algorithms in a manner independent of a particular platform, compiler, or language.  Abstract away the minor variations and describe the performance of algorithms in a more theoretical, processor independent fashion.  A method to compare speed of algorithms against one another.

CS 307 Fundamentals of Computer Science 3 Different Algorithms  Why are the words in a dictionary in alphabetical order?  A brute force approach –linear search –worst case is (d x N)  Another way –divide and conquer –worst case is (c x log N)  Constants are unknown and largely irrelevant.

CS 307 Fundamentals of Computer Science 4 Big O  The most common method and notation for discussing the execution time of algorithms is "Big O”.  For the alphabetized dictionary the algorithm requires O(log N) steps.  For the unsorted list the algorithm requires O(N) steps.  Big O is the asymptotic execution time of the algorithm.

CS 307 Fundamentals of Computer Science 5 Formal Definition of Big O  T(N) is O( F(N) ) if there are positive constants c and N 0 such that T(N) N 0 –There is a point N 0 such that for all values of N that are past this point, T(N) is bounded by some multiple of F(N). –Thus if T(N) of the algorithm is O( N 2 ) then, ignoring constants, at some point we can bound the running time by a quadratic function of the input size. –Given a linear algorithm, it is technically correct to say the running time is O(N 2 ). O(N) is a more precise answer as to the Big O bound of a linear algorithm.

CS 307 Fundamentals of Computer Science 6 Big O Examples  3n 3 = O(n 3 )  3n = O(n 3 )  8n n * log(n) + 100n = O(n 2 )  3log(n) + 2n 1/2 = O(n 1/2 )  = O(1)  T linearSearch (n) = O(n)  T binarySearch (n) = O(log(n))

CS 307 Fundamentals of Computer Science 7 Other Algorithmic Analysis Tools  Big Omega T(N) is  ( F(N) ) if there are positive constants c and N 0 such that T(N) > cF( N )) when N > N 0 –Big O is similar to less than or equal, an upper bound. –Big Omega is similar to greater than or equal, a lower bound.  Big Theta T(N) is  ( F(N) ) if and only if T(N) is O( F(N) )and T( N ) is  ( F(N) ). –Big Theta is similar to equals.

CS 307 Fundamentals of Computer Science 8 Relative Rates of Growth Analysis Type Mathematical Expression Relative Rates of Growth Big OT(N) = O( F(N) )T(N) < F(N) Big  T(N) =  ( F(N) ) T(N) > F(N) Big  T(N) =  ( F(N) ) T(N) = F(N) "In spite of the additional precision offered by Big Theta, Big O is more commonly used, except by researchers in the algorithms analysis field" - Mark Weiss

CS 307 Fundamentals of Computer Science 9 What it All Means  T(N) is the actual growth rate of the algorithm.  F(N) is the function that bounds the growth rate. –may be upper or lower bound  T(N) may not equal F(N). –constants and lesser terms ignored because it is a bounding function

CS 307 Fundamentals of Computer Science 10 Assumptions in Big O Analysis  Once found accessing the value of a primitive is constant time x = y;  mathematical operations are constant time x = y * 5 + z % 3;  if statement: constant time if test and maximum time for each alternative are constants if( iMySuit ==DIAMONDS || iMySuit == HEARTS) return RED; else return BLACK;

CS 307 Fundamentals of Computer Science 11 Fixed-Size Loops  Loops that perform a constant number of iteration are considered to execute in constant time. They don't depend on the size of some data set for(int suit = Card.CLUBS; suit <= Card.SPADES; suit++) { for(int value = Card.TWO; value <= Card.ACE; value++) { myCards[cardNum] = new Card(value, suit); cardNum++; }

CS 307 Fundamentals of Computer Science 12 Loops That Work on a Data Set  Loops like on the previous slide are fairly rare.  Normally a loop operates on a data set which can vary is size. public double minimum(double[] values) { int n = values.length; double minValue = values[0]; for(int i = 1; i < n; i++) if(values[i] < minValue) minValue = values[i]; return minValue; }  The number of executions of the loop depends on the length of the array, values. The actual number of executions is (length - 1).  The run time is O(N).

CS 307 Fundamentals of Computer Science 13 Nested Loops  Number of executions? public void bubbleSort(double[] data) { int n = data.length; for(int i = n - 1; i > 0; i--) for(int j = 0; j data[j+1]) { double temp = data[j]; data[j] = data[j + 1]; data[j + 1] = temp; } }

CS 307 Fundamentals of Computer Science 14 Summing Execution Times  If an algorithm’s execution time is N 2 + N then it is said to have O(N 2 ) execution time, not O(N 2 + N).  When adding algorithmic complexities the larger value dominates.  Formally, a function f(N) dominates a function g(N) if there exists a constant value n 0 such that for all values N > N 0 it is the case that g(N) < f(N).

CS 307 Fundamentals of Computer Science 15 Example of Dominance  Suppose we go for precision and determine how fast an algorithm executes based on the number of items in the data set.  x 2 / x log 10 x  Is it plausible to say the x 2 term dominates even though it is divided by 10000?  What if we separate the equation into (x 2 /10000 ) and (2x log x )?

CS 307 Fundamentals of Computer Science 16 Summing Execution Times  For large values the x 2 term dominates so the algorithm is O(N 2 ).

CS 307 Fundamentals of Computer Science 17 FunctionCommon Name N!factorial 2N2N Exponential N d, d > 3Polynomial N3N3 Cubic N2N2 Quadratic N N log N NLinear NRoot - n log NLogarithmic 1Constant Ranking of Algorithmic Behaviors

CS 307 Fundamentals of Computer Science 18 Running Times  Assume N = 100,000 and processor speed is 1,000,000 operations per second FunctionRunning Time 2N2N over 100 years N3N years N2N2 2.8 hours N 31.6 seconds N log N1.2 seconds N0.1 seconds N3.2 x seconds log N1.2 x seconds

CS 307 Fundamentals of Computer Science 19 Graphical Results

CS 307 Fundamentals of Computer Science 20 Determining Big O  A DNA problem  Is a number prime?  Hand actions –inserting card –determining if card is in Hand –combining Hands –copying Hand –retrieving Card

CS 307 Fundamentals of Computer Science 21 Putting it in Context  Why worst-case analysis? Wouldn’t average-case analysis be more useful?  A1: No. E.g., doesn’t matter if a cashier’s terminal handles 2 transactions per second or 3, as long as it never takes more than 20.  A2: Well, OK, sometimes, yes. However, this often requires much more sophisticated mathematics. In fact, for some common algorithms, nobody has been able to do an average case analysis.