CSC 2300 Data Structures & Algorithms January 30, 2007 Chapter 2. Algorithm Analysis

Today Binary Search Euclid’s Algorithm Efficient Exponentiation Recursion and recurrences

Binary Search Given an integer X and integers A 0, A 1, …, A N-1, which are presorted and already in memory, find i such that A i = X, or return i=-1 if X is not in the input. What is an obvious solution? Scan through the list from left to right. Time? O(N). Better strategy? Check if X is the middle element. If yes, we get the answer. If X is smaller, we apply the same strategy to the sorted array to the left of the middle element. If X is larger, we look to the right half.

Binary Search What is the running time? O(logN).

Analyzing Binary Search

Example We have 16 integers: 1, 3, 5, 7, 9, 11, …, 29, 31, and X=9. lowhigh#ptsmida[mid]return

Example We have 16 integers: 1, 3, 5, 7, 9, 11, …, 29, 31, and X=8. lowhigh#ptsmida[mid]return

Example We have 16 integers: 1, 3, 5, 7, 9, 11, …, 29, 31, and X=1. lowhigh#ptsmida[mid]return

Example We have 16 integers: 1, 3, 5, 7, 9, 11, …, 29, 31, and X=31. lowhigh#ptsmida[mid]return

Example We have 16 integers: 1, 3, 5, 7, 9, 11, …, 29, 31, and X=33. lowhigh#ptsmida[mid]return

Euclid’s Algorithm This algorithm computes gcd(m,n), assuming m≥n. The algorithm works by computing remainders until 0 is reached. The last nonzero remainder is the answer. Example, M=1989 and N=1590. The sequence of remainders is 399, 393, 6, 3, 0. Therefore, gcd(1989,1590)=3.

Running Time Analysis The sequence of remainders is 399, 393, 6, 3, 0. The remainder does not decrease by a constant factor in one iteration. What to do? We can prove that after two iterations, the remainder is at most half of its original value. This will show that the number of iterations is at most 2 logN = O(logN). Theorem 2.1. If M>N, then M mod N < M/2. Worst case is 2 logN? No, 1.44 logN, when M and N are consecutive Fibonacci numbers.

Efficient Exponentiation The obvious way to compute X N uses N-1 multiplications. A recursive algorithm can do better. What are the base cases? N=0 and N=1. Otherwise, if N is even, we compute X N = X N/2 X N/2, and if N is odd, X N = X N/2 X N/2 X. How many multiplications are needed? At most 2 logN. Why 2? If N is odd.

Example Compute X 62. The algorithm performs these calculations: X 3 = (X 2 )X, X 7 = (X 3 ) 2 X, X 15 = (X 7 ) 2 X, X 31 = (X 15 ) 2 X, X 62 = (X 31 ) 2, How many multiplications? Nine. In general, at most 2 logN.

Number of Multiplications Does it always require more work to compute X i than X j if i > j? No. Consider X 64 (versus X 62 ). The algorithm performs these calculations: X 2 = (X) 2, X 4 = (X 2 ) 2, X 8 = (X 4 ) 2, X 16 = (X 8 ) 2, X 32 = (X 16 ) 2, X 64 = (X 32 ) 2, How many multiplications? Six (versus nine for X 62 ).

Precise Number of Multiplies Can we find the precise number of multiplies used by the algorithm? Consider X 65. The algorithm performs these calculations: X 2 = (X) 2, X 4 = (X 2 ) 2, X 8 = (X 4 ) 2, X 16 = (X 8 ) 2, X 32 = (X 16 ) 2, X 65 = (X 32 ) 2 X, How many multiplications? Seven (versus six for X 64 ).

Another Example Consider X 63. The algorithm performs these calculations: X 3 = (X) 2 X, X 7 = (X 3 ) 2 X, X 15 = (X 7 ) 2 X, X 31 = (X 15 ) 2 X, X 63 = (X 31 ) 2 X, How many multiplications? Ten (versus six for X 64 ). What does the number depend on?

Precise Number of Multiplies Compute X N. What is the number of multiplies when N=0 or N=1? Zero. What is the important feature on the value of N? Its binary representation. Let b(N) represent the number of ones in the binary representation of N. What is the formula for #mult? [ log 2 N ] + b(N) - 1 N #mult

Fast Exponentiation Is algorithm optimal? No. Compute X 62. Fast exponentiation requires nine multiplications. Can you calculate X 62 using only 8 multiplies? How? Compute X 2, X 4, X 8, …, X 62. You are asked to solve this problem in Homework 2.

Recursion and Recurrences

Methods for Recurrences Two straightforward methods for solving recurrences are 1. Repeated expansion accompanied by summation 2. Forming telescopic sums

Method 1 Repeated expansion Recurrence: T(n) = T(n-1) + 1

Method 2 Telescoping sum Recurrence: T(n) = T(n-1) + 1

Telescoping Sum Recall Algorithm 3 to find Maximum Subsequence Sum. Recurrence: T(1) = 1, T(N) = 2 T(N/2) + N. Divide recurrence by N: T(N)/N = T(N/2)/(N/2) + 1. The recurrence is valid for any N that is a power of 2: T(N)/N = T(N/2)/(N/2) + 1 T(N/2)/(N/2) = T(N/4)/(N/4) + 1 T(N/4)/(N/4) = T(N/8)/(N/8) + 1 … T(4)/4 = T(2)/2 + 1 T(2)/2 = T(1)/1 + 1 Telescoping the sum, we get T(N)/N = T(1)/(1) + logN. So, T(N) = N logN + N = O(N logN).