Proofs, Recursion, and Analysis of Algorithms Mathematical Structures for Computer Science Chapter 2 Copyright © 2006 W.H. Freeman & Co.MSCS SlidesProofs, Recursion, and Analysis of Algorithms

Section 2.6Analysis of Algorithms1 We need some other basis of comparison to decide which algorithm to use in a given situation. Several criteria could be used to judge which is the “best:” Easiest to understand Which makes the most efficient use of machine memory Which runs most efficiently (time efficiency) We evaluate the time efficiency of an algorithm by estimating the number of operations it must perform. We count only the operations that are basic to the task at hand. The study of the efficiency of algorithms, that is, the number of operations they perform, is called analysis of algorithms.

Section 2.6Analysis of Algorithms2 Suppose we need to search a sorted list of n words or numbers for a particular target value x. ALGORITHM: SequentialSearch SequentialSearch(list L; integer n; itemtype x) //searches a list L of n items for item x Local variable: integer i //marks position in the list i = 1 while L[i]  x and i < n do i = i + 1 end while if L[i] = x then write(“Found”) else write(“Not found”) end if end function SequentialSearch

Section 2.6Analysis of Algorithms3 The sequential search algorithm does the maximum amount of work (number of comparisons) when x is the last item in the list or does not appear at all. In either case, all elements are compared to x, so n comparisons are done. This is the “worst case.” The minimum amount of work is done when x is the very first item on the list. Only one comparison is made. This is the “best case.” If x falls in the exact middle of the list, the search would require roughly n/2 comparisons. For most algorithms, the average behavior is very difficult to determine. We will content ourselves with the worst-case count of the number of operations required.

Section 2.6Analysis of Algorithms4 Analysis Using Recurrence Relations We can recast the sequential search algorithm from an iterative version to a recursive version. SequentialSearchRecursive(list L; integer i, n; itemtype x) //searches list L from L[i] to L[n] for item x if i > n then write(“Not found”) else if L[i] = x then write(“Found”) else SequentialSearch(L, i + 1, n, x) end if

Section 2.6Analysis of Algorithms5 Analysis Using Recurrence Relations Let C(n) represent the maximum number of comparisons required for an n-element list. C(1) = 1 (one comparison to search a 1-element list). C(n) = 1 + C(n  1) for n  2 (one comparison against the first element, then however many comparisons are required for the rest of the list). This is a first-order, linear recurrence relation with constant coefficients. This agrees with our previous analysis for the worst case.

Section 2.6Analysis of Algorithms6 Analysis Using Recurrence Relations Let C(n) stand for the maximum number of comparisons required to do a binary search on an n- element list. Then, C(n/2) stands for the maximum number of comparisons required to search a list half that size. The recurrence relation for C(n) is: C(1) = 1 C(n) = 1 + C(n/2) for n  2, n = 2 m The recurrence relation is solved to C(n) = 1 + log n.

Section 2.6Analysis of Algorithms7 Upper Bound (Euclidean Algorithm) The Euclidean algorithm uses a while loop to do successive divisions to find gcd(a, b) for positive integers a and b, a  b. A recursive version of the Euclidean algorithm can also be written. The key to the recursive version is to recognize that gcd(a, b) involves finding gcd(b, r), where r is the remainder on dividing a by b. We won’t find a recurrence relation, we will find an upper bound. An upper bound is a ceiling on the amount of work an algorithm does; the algorithm can require no more steps than the upper bound, but it may not require that many.

Section 2.6Analysis of Algorithms8 Analysis Using Recurrence Relations To find this upper bound, we will show that if i > j and i is divided by j with remainder r, then r < i/2. There are two cases: 1. If j  i/2, then r < i/2 since r < j. 2. If j > i/2, then i = 1 * j + (i  j); in other words, the quotient is 1 and r is i  j, which is less than i/2. In the Euclidean algorithm, the remainder r at any step becomes the dividend (the number being divided) two steps later. So the successive dividends are at least halved every two divisions. The remainder r at any step becomes the dividend (the number being divided) two steps later. So the successive dividends are at least halved every two divisions. The Value n can be halved log n times; therefore, at most, 2 log n divisions are done. E(n)  2 log n