1. CS 307 Fundamentals of Computer Science 1 Asymptotic Analysis of Algorithms (how to evaluate your programming power tools) based on presentation material by Mike Scott Honors CS 102 Sept. 4, 2007

2. CS 307 Fundamentals of Computer Science 2 Algorithmic Analysis  A technique used to characterize the execution behavior of algorithms in a manner independent of a particular platform, compiler, or language.  Abstract away the minor variations and describe the performance of algorithms in a more theoretical, processor independent fashion.  A method to compare speed of algorithms against one another.

3. CS 307 Fundamentals of Computer Science 3 Different Algorithms  Why are the words in a dictionary in alphabetical order?  A brute force approach –linear search –worst case is (d x N)  Another way –divide and conquer –worst case is (c x log N)  Constants are unknown and largely irrelevant.

4. CS 307 Fundamentals of Computer Science 4 Big O  The most common method and notation for discussing the execution time of algorithms is "Big O”.  For the alphabetized dictionary the algorithm requires O(log N) steps.  For the unsorted list the algorithm requires O(N) steps.  Big O is the asymptotic execution time of the algorithm.

5. CS 307 Fundamentals of Computer Science 5 Formal Definition of Big O  T(N) is O( F(N) ) if there are positive constants c and N 0 such that T(N) N 0 –There is a point N 0 such that for all values of N that are past this point, T(N) is bounded by some multiple of F(N). –Thus if T(N) of the algorithm is O( N 2 ) then, ignoring constants, at some point we can bound the running time by a quadratic function of the input size. –Given a linear algorithm, it is technically correct to say the running time is O(N 2 ). O(N) is a more precise answer as to the Big O bound of a linear algorithm.

6. CS 307 Fundamentals of Computer Science 6 Big O Examples  3n 3 = O(n 3 )  3n 3 + 8 = O(n 3 )  8n 2 + 10n * log(n) + 100n + 10 20 = O(n 2 )  3log(n) + 2n 1/2 = O(n 1/2 )  2 100 = O(1)  T linearSearch (n) = O(n)  T binarySearch (n) = O(log(n))

7. CS 307 Fundamentals of Computer Science 7 Other Algorithmic Analysis Tools  Big Omega T(N) is  ( F(N) ) if there are positive constants c and N 0 such that T(N) > cF( N )) when N > N 0 –Big O is similar to less than or equal, an upper bound. –Big Omega is similar to greater than or equal, a lower bound.  Big Theta T(N) is  ( F(N) ) if and only if T(N) is O( F(N) )and T( N ) is  ( F(N) ). –Big Theta is similar to equals.

8. CS 307 Fundamentals of Computer Science 8 Relative Rates of Growth Analysis Type Mathematical Expression Relative Rates of Growth Big OT(N) = O( F(N) )T(N) < F(N) Big  T(N) =  ( F(N) ) T(N) > F(N) Big  T(N) =  ( F(N) ) T(N) = F(N) "In spite of the additional precision offered by Big Theta, Big O is more commonly used, except by researchers in the algorithms analysis field" - Mark Weiss

9. CS 307 Fundamentals of Computer Science 9 What it All Means  T(N) is the actual growth rate of the algorithm.  F(N) is the function that bounds the growth rate. –may be upper or lower bound  T(N) may not equal F(N). –constants and lesser terms ignored because it is a bounding function

10. CS 307 Fundamentals of Computer Science 10 Assumptions in Big O Analysis  Once found accessing the value of a primitive is constant time x = y;  mathematical operations are constant time x = y * 5 + z % 3;  if statement: constant time if test and maximum time for each alternative are constants if( iMySuit ==DIAMONDS || iMySuit == HEARTS) return RED; else return BLACK;

11. CS 307 Fundamentals of Computer Science 11 Fixed-Size Loops  Loops that perform a constant number of iteration are considered to execute in constant time. They don't depend on the size of some data set for(int suit = Card.CLUBS; suit <= Card.SPADES; suit++) { for(int value = Card.TWO; value <= Card.ACE; value++) { myCards[cardNum] = new Card(value, suit); cardNum++; }

12. CS 307 Fundamentals of Computer Science 12 Loops That Work on a Data Set  Loops like on the previous slide are fairly rare.  Normally a loop operates on a data set which can vary is size. public double minimum(double[] values) { int n = values.length; double minValue = values[0]; for(int i = 1; i < n; i++) if(values[i] < minValue) minValue = values[i]; return minValue; }  The number of executions of the loop depends on the length of the array, values. The actual number of executions is (length - 1).  The run time is O(N).

13. CS 307 Fundamentals of Computer Science 13 Nested Loops  Number of executions? public void bubbleSort(double[] data) { int n = data.length; for(int i = n - 1; i > 0; i--) for(int j = 0; j data[j+1]) { double temp = data[j]; data[j] = data[j + 1]; data[j + 1] = temp; } }

14. CS 307 Fundamentals of Computer Science 14 Summing Execution Times  If an algorithm’s execution time is N 2 + N then it is said to have O(N 2 ) execution time, not O(N 2 + N).  When adding algorithmic complexities the larger value dominates.  Formally, a function f(N) dominates a function g(N) if there exists a constant value n 0 such that for all values N > N 0 it is the case that g(N) < f(N).

15. CS 307 Fundamentals of Computer Science 15 Example of Dominance  Suppose we go for precision and determine how fast an algorithm executes based on the number of items in the data set.  x 2 /10000 + 2x log 10 x + 100000  Is it plausible to say the x 2 term dominates even though it is divided by 10000?  What if we separate the equation into (x 2 /10000 ) and (2x log x + 100000)?

16. CS 307 Fundamentals of Computer Science 16 Summing Execution Times  For large values the x 2 term dominates so the algorithm is O(N 2 ).

17. CS 307 Fundamentals of Computer Science 17 FunctionCommon Name N!factorial 2N2N Exponential N d, d > 3Polynomial N3N3 Cubic N2N2 Quadratic N N log N NLinear NRoot - n log NLogarithmic 1Constant Ranking of Algorithmic Behaviors

18. CS 307 Fundamentals of Computer Science 18 Running Times  Assume N = 100,000 and processor speed is 1,000,000 operations per second FunctionRunning Time 2N2N over 100 years N3N3 31.7 years N2N2 2.8 hours N 31.6 seconds N log N1.2 seconds N0.1 seconds N3.2 x 10 -4 seconds log N1.2 x 10 -5 seconds

19. CS 307 Fundamentals of Computer Science 19 Graphical Results

20. CS 307 Fundamentals of Computer Science 20 Determining Big O  A DNA problem  Is a number prime?  Hand actions –inserting card –determining if card is in Hand –combining Hands –copying Hand –retrieving Card

21. CS 307 Fundamentals of Computer Science 21 Putting it in Context  Why worst-case analysis? Wouldn’t average-case analysis be more useful?  A1: No. E.g., doesn’t matter if a cashier’s terminal handles 2 transactions per second or 3, as long as it never takes more than 20.  A2: Well, OK, sometimes, yes. However, this often requires much more sophisticated mathematics. In fact, for some common algorithms, nobody has been able to do an average case analysis.