1. Algorithm Analysis Chapter 5

2. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. –recipes –directions for putting together a bicycle –programs

3. Algorithm Analysis The process by which we determine the amount of resources used by the algorithm Resources usually measured –time –space We will concentrate on run time of algorithms

4. Common functions encountered Some commonly encountered functions are: –linear –N log N –quadratic –cubic The next slide shows the graphs of these functions

5. Growth Rates

6. Example - Downloading Suppose we wish to download a file –2 seconds for setting up the connection –downloading proceeds at 2.8 Kb/sec If the downloaded file is N kilobytes, the time to download is T(N) = N/2.8 + 2 –80K file takes about 31 seconds –160K file takes about 59 seconds This is a linear algorithm

7. Big-O notation We use big-O to specify growth rates of algorithms –linear functions are specified as O(N) –quadratic functions as O(N 2 )

8. Functions in Increasing Order

9. Examples Minimum element in an array –Given an array of N elements, find the smallest. Closest points in the plane –Given N points in the plane, find the pair of points that are closest together. Collinear points in the plane –Given N points in the plane, determine if any three are on the same line.

10. Minimal Element in an Array Algorithm –Initialize min as the first element –Scan thru the other N-1 elements, updating min as appropriate There is a fixed amount of work for each element of the array The running time is O(N)

11. Closest Point in the Plane Algorithm –Calculate the distance between each pair of points [there are N(N-1)/2 pairs of points] –find the minimum of these distances Finding the minimum of N(N-1)/2 distances by this algorithm is O(N 2 )

12. Collinear Points in the Plane Algorithm –Enumerate all triples of the N points. There are N(N-1)(N-2)/6 of these triples. –check to see if the three points are on the same line This is a O(N 3 ) algorithm

13. Maximum Contiguous Subsequence Sum Problem Given (possibly negative) integers A 1, A 2,…, A N, find (and identify the subsequence corresponding to) the maximum value A i + A i+1 +…+ A k. The maximum contiguous subsequence sum is zero if all the integers are negative

14. Examples With input of [-2, 11, -4, 13, -5, 2], then the maximum sum is 20 using items at positions 2 through 4. With input of [ 1, -3, 4, -2, -1, 6], then the maximum sum is 7 using the last four items.

15. Obvious O(N 3 ) Algorithm maxSum = 0; for (i=0; i < N; i++) for (j=i; j < N; j++) { thisSum = 0; for (k=i;k maxSum) { maxSum = thisSum; seqStart = i; seqStop = j; } } return maxSum;

16. Improving the Algorithm If we can eliminate one of the loops, we will be able to reduce this algorithm to a O(N 2 ) algorithm. Observe that A i + A i+1 +…+ A k = (A i + A i+1 +…+ A k-1 ) + A k. The inner loop is used to calculate A i + A i+1 +…+ A k-1, then this result is thrown away and then A i + A i+1 +…+ A k is calculated (duplicating work)

17. Improved O(N 2 ) Algorithm maxSum = 0; for (i=0; i < N; i++) { thisSum = 0; for (j=i; j < N; j++) { thisSum += a[j]; if (thisSum > maxSum) { maxSum = thisSum; seqStart = i; seqStop = j; } } } return maxSum;

18. Observations 1.No maximum contiguous subsequence begins with a subsequence which has a negative sum. 2. All subsequences which border on the maximum contiguous subsequence must have negative or zero sums

19. Observations 3.Let Ai,j be the first sequence with Si,j < 0. Then, for any i <= p <= j and p <=q, Ap,q is not a maximum contiguous subsequence or is equal to (in sum) an already seen maximum contiguous subsequence. (Theorem 5.3 on page 159)

20. Improving again Using these observations, any time the current subsequence sum (thisSum) becomes negative, we can begin a new subsequence sum at the next position. Thus we can rewrite the algorithm using only a single loop.

21. Linear Algorithm maxSum = thisSum = 0; for (i=0, j=0; j < N; j++) { thisSum += a[j]; if (thisSum > maxSum) { maxSum = thisSum; seqStart = i; seqStop = j; } else if (thisSum < 0) { i = j+1; thisSum = 0; } } return maxSum;

22. Big-O Definition: T(N) is O(F(N)) if there are positive constants c and N 0 such that T(N)  cF(N) whenever N  N 0. This means that T is eventually bounded above by some constant times F, so F could be used an a crude upper-bound estimate of the function T. This means that the growth rate of T is no worse than the growth rate of F.

23. Big-  (omega) Definition: T(N) is  (F(N)) if there are positive constants c and N 0 such that T(N)  cF(N) whenever N  N 0. This means that T is eventually bounded below by some multiple of F so, F could be used as a crude lower-bound estimate for T. This means that the growth rate of T is no better than the growth rate of F.

24. Big-  (theta) Definition: T(N) is  (F(N)) if and only if T(N) is O(F(N)) and T(N) is  (F(N)). That means eventually T is bounded below by some constant times F and bounded above by some constant time F. This means that the growth rate of T is equal to the growth rate of F.

25. Little-o Definition: T(N) is o(F(N)) if and only if T(N) is O(F(N)) and T(N) is not  (F(N)). This means that the growth rate of T is strictly better than the growth rate of F.

26. Summary

27. The Logarithm Definition: For any B, N > 0, log B N = K if and only if B K = N In Computer Science, base 2 logarithms are used almost exclusively. The use of the symbol log indicates log 2 unless otherwise indicated

28. Use of logarithms Bits in a binary number –The number of bits needed to represent N consecutive integers is  log N  Repeated Doubling –Starting with X=1, the number of times needed to double X to reach at least N is  log N 

29. Searching Sequential Search –the average number of comparisons made to find an item in an unordered list of N items is N/2 = O(N) –An unsuccessful search requires testing every item in the list, so is also O(N)

30. Binary Search If searching a sorted list of N items, we can do better low = 1; high = N; while (low < high) { mid = (low+high)/2; if (A[mid]< x) low = mid +1; else high = mid; } if (X == A[low]) return low; return -1;

31. Homework Pages 176-181: # 3, 4, 5(a,b), 6, 8, 11, 12, 14, 15, 17, and 28.