1. Chapter 7 Analysis of Algorithms © 2006 Pearson Education Inc., Upper Saddle River, NJ. All rights reserved.

2. Overview ● 7.1 – Shows how to directly time programs and points out some problems with this approach. ● 7.2 – Powerful mathematical tools to simplify our reasoning about the efficiency of algorithms. ● 7.3 – The connections between this math and actual algorithms.

3. Overview ● 7.4 and 7.5 – Considering the average or the worst-case running time.

4. Timing ● To know which of two methods is faster, the most obvious approach is to time them. – System.currentTimeMillis() – The number of milliseconds since midnight, January 1, 1970, Greenwich mean time. – A modern computer is so fast that, on either of these data structures, the get() method takes less than one millisecond to run.

5. Timing

6. ● ArrayList: 0 milliseconds ● LinkedList: 0 milliseconds

7. Timing

8. ● Results Vary ● Variation is beyond our control. ● ArrayList is roughly twice as fast as the method from LinkedList.

9. Timing ● If we get element 50 instead of element 5. ● ArrayList, get() jumps right to the array element in question. ● LinkList traverse all the previous list nodes to find the one we want. ● Timing experiment provides empirical evidence. ● We can use some formal, mathematical tools to make statements about the efficiency of an algorithm.

10. Asymptotic Notation ● Method A is 10n² - 5 milliseconds to process n elements. ● Method B is 100n + 200 milliseconds.

11. Asymptotic Notation

12. ● The differences for small values of n are relatively insignificant. – What really concerns us is the asymptotic behavior of the running-time functions: ● What happens as n becomes very large?

13. Asymptotic Notation

14. ● Theorem: 10n² – 5 > 100n + 200 for any value of n ≥ 12. ● Proof: for any n ≥ 12: ● 10n² - 5 ≥ 10*12n - 5 – ≥ 120n - 5 – ≥ 100n + 20n – 5 ● ≥ 100n + 20*12 - 5 – ≥ 100n + 240 - 5 – > 100n + 200

15. Asymptotic Notation ● First and third inequalities follow because of the assumption that n ≥ 12. ● Actual running times will depend on a number of factors. ● Make statements about the speed of an algorithm in general, rather than a particular implementation of it. – This way, we don't have to redo our analysis if we change programming languages or buy a faster computer.

16. Asymptotic Notation ● To keep our running-time expressions general, we allow them to contain unspecified constants. – Algorithm A is an² + b – Algorithm B is cn + d ● a, b, c, and d are unspecified constants that depend on factors such as the speed of the hardware.

17. Asymptotic Notation ● Orders: – Functions can be classified into orders ● Monotonically nondecreasing: – f(n + 1) ≥ f(n) – Algorithm for which the running-time function did not fit into this category would be fairly strange.

18. Asymptotic Notation ● Θ (f): – “the order of f” or “order f” ● n² is one of the functions in Θ (n²) – Θ (2ⁿ) is the largest – Θ (1) is the smallest. ● For sufficiently large n, a function is asymptotically larger than any function in a lower order. ● Example: – Θ (n log n) is asymptotically larger than any function in Θ (n).

19. Asymptotic Notation

20. ● Multiplying or dividing a function by a positive constant doesn't change its order. 3n² and 0.2n² are both in Θ (n²) ● A function's order is not changed by adding or subtracting a function in a lower order. 2ⁿ – n + log n is in Θ (2ⁿ)

21. Asymptotic Notation ● Problem: f(n) = 5n³ + 3n² – 4n + 11 ● Answer: Θ (n³)

22. Asymptotic Notation

23. ● Column frobbing algorithm is in Θ (n²) ● Row zorching algorithm is in Θ (n). Θ (n) is a lower order, so we should choose row zorching.

24. Asymptotic Notation

25. ● Synchronized absquatulation takes time in Θ (n²)

26. Asymptotic Notation

27. ● ● At least ● Or

28. Asymptotic Notation ● Factorial decortication takes time which is at least in Θ (2ⁿ). ● Order Θ (n³), we should choose cubic flensing. Θ (n!) ≥ Θ (2ⁿ) – If Θ (f) is the set of functions which grow like f, then Ω (f) is the set of functions which grow like f or much more quickly. – We proved that n! is in Ω (2ⁿ).

29. Asymptotic Notation

30. A function in O(g) might actually be larger than g. –Example –3n² O(n²) –f O(g) if and only if, for some constant c>0, there is some n 0 ≥ 0 such that f(n) < cg(n) for any n ≥ n 0. –A threshold so that only large values of n are considered.

31. ● f Ω (g) if and only if, for some constant c > 0, there is some n 0 ≥ 0 such that f(n) > cg(n) for any n ≥ n 0. ● f Θ (g) if and only if f O(g) and f Ω (g). – Showing the f O(g) is called finding an asymptotic upper bound on f. – Showing that f Ω (g) is called finding an asymptotic lower bound. – Showing that f Θ (g) is called finding an asymptotically tight bound. Asymptotic Notation

32. ● For k > 1, the orders Θ (k n ) are called exponential orders. ● For k > 0, the orders Θ (n k ) are called polynomial orders.

33. Counting Steps ● When we analyze an algorithm, we're aiming for: – The order of the running time. ● Write the algorithm down in precise English or in any programming language, such as Java. ● Determine how many steps are accomplished by each line and how many times the line is executed. The time used by the line is the product of these two expressions. ● The total running time for the algorithm is the sum of the time required for each line. The order of this expression is the same as the order of the most time-consuming line.

34. Counting Steps ● Size of the list: n – A single step: ● Accessing or setting a variable or field, including an element of an array. ● Addition, subtraction, multiplication, division, and other arithmetic operators. ● Finding the length of an array or String. ● Comparisons using ==, <, etc. ● Any fixed number of single steps, such as two additions and a variable assignment. ● Operators count as single steps. – This is not necessarily true of methods in the Java library.

35. Counting Steps

36. ● Tally starts at 0 and ends up equal to n, there must be n passes through the loop. – Running time for size() is linear. ● 1 2 3 4 5 6 8 (lines) ● c + c + c + c(n + 1) + cn + cn + c = 3cn + 5c Θ (n)

37. Counting Steps ● r: the number of ranks. ● s: the number of suits. – Θ ((r + 1)s) = Θ (rs)

38. Counting Steps ● Deck constructor the most expensive step in the algorithm.

39. Counting Steps ● Adds up only the numbers for which j ≤ i.

40. Counting Steps ● Total number of passes in the inner loop: – 1+2+...+n

41. Counting Steps

42. ● A single for loop typically takes time in Θ (n) a doubly nested for loop typically takes time in Θ (n 2 ). – Do not overgeneralize the result about loops. – Enhanced for loops generally runs at most n times, where n is the number of elements in the data structure being traversed. – A loop may run less than n times if it is stopped early by a return or break statement or if it deals with more than one element on each pass.

43. Best, Worst, and Average Case ● Difficult to analyze because of the if statement.

44. Best, Worst, and Average Case ● We have to decide which kind of analysis we're doing. ● Best-case analysis – Tells us how fast the program runs if we get really lucky about the data. ● Contains() ● Θ (1)

45. Best, Worst, and Average Case ● Worst-case analysis – Contains() – This means assuming that target is not in the ArraList, giving a running time of Θ (n). ● Average-case analysis – Requires that we make some assumption about what the “average” data set looks like. – Average running time is:

46. Best, Worst, and Average Case ● We must always be careful to choose our events so that they are exhaustive – at least one of them will occur. ● Mutually exclusive – no more than one of them will occur. ● With n possible events, the probability of each event occurring is 1/n. – Assuming that they are equally likely.

47. Best, Worst, and Average Case ● In the contains() method,if target is at index – 0, there is 1 pass through the loop. – 1, there are 2 passes –...

48. Best, Worst, and Average Case ● Best case ≤ average case ≤ worst case

49. Amortized Analysis ● Average-case analysis – How much time does this algorithm take on a typical run? ● Worst-case analysis – How much time does this algorithm take on the worst possible run? ● Amortized analysis – If this algorithm is run several times, what is the average time per run, given the worst possible sequence of runs?

50. Amortized Analysis ● Amortized analysis does not have to make any assumptions about what a “typical” run looks life. – Often, the amortized running time is the same as the worst-case running time. – For some algorithms it is not possible for the worst run to occur many times in a row.

51. Amortized Analysis

53. ● The total time for this sequence of runs is therefore: 1 + 2 + 4 +... + (n-1)

54. Amortized Analysis ● Assuming there are t terms.

55. Amortized Analysis ● The amortized time per operation is:

56. Amortized Analysis ● What if, instead of doubling the capacity of the ArrayList, it increases the capacity by 1.

57. Amortized Analysis

58. ● Dividing this by n operations, we find that the amortized time per operation is now linear. ● Amortized analysis tells us that we should prefer the version of stretch() that doubles the capacity of the ArrayList.

59. Summary ● Analysis of algorithms gives us mathematical tools for comparing the efficiency of different algorithms. ● We are interested in asymptotic behavior: – What happens to the running times as the size of the data becomes arbitrarily large.

60. Summary ● The running time of a method can be expressed as a function, and functions can be classified into orders. – A function's order is unchanged if it is multiplied by a constant factor or if a lower-order function is added. – Things like faster hardware do not affect the order of an algorithm's running-time function.

61. Summary ● To find the running time of an algorithm, we determine the running time for each line. – Number of simple (constant-time) steps the line accomplishes, multiplied by the number of times the line is run.

62. Summary ● Best case ≤ average case ≤ amortized ≤ worst case – Worst-case analysis is the most common. – Average-case analysis is also useful, but requires some assumption about what “average” data looks like. – Amortized analysis is appropriate when the worst case (such as stretching an array) cannot happen on every run.