1. BINARY SEARCH CS16: Introduction to Data Structures & Algorithms Thursday February 12, 2015 1

2. Outline 1) Binary Search 2) Binary Search Pseudocode 3) Analysis of Binary Search 4) In-Place Binary Search 5) Iterative Binary Search Thursday February 12, 2015 2

3. The Problem Determine whether an item, x, is in a sorted array Obvious solution: iterate through the entire array and check each element to see if it’s the one we’re searching for This solution is O(n). Can we do better? Let’s use the fact that the array is sorted! If we’re looking for the item x, we can stop searching as soon as we find an item y > x, because we know x can’t come after y in the array But what if we’re looking for 25 in the example above? Worst case still O(n). Boooo. Thursday February 12, 2015 3 11347810 1218192123 24 e.g. is 5 in this array?

4. Binary Search What if we compared x to the middle element of the array, mid? If mid == x, then we found x! If mid < x, then we know x must be in the second half of the array, if it’s there at all If mid > x, then we know x must be in the first half of the array, if it’s there at all Important observation: No matter what, we can eliminate half of the array We then end up with the same problem, but half the size! Thursday February 12, 2015 4 11347810 1218192123 24 mid WOAH WE SHOULD DO IT AGAIN

5. Binary Search Simulation Thursday February 12, 2015 5 11347810 1218192123 24 Goal: Find 5 because 5 < 10 1134781078 because 5 > 4 7 because 5 < 8 Since 7 ≠ 5, we can conclude that 5 is NOT in the array. And it only took us four comparisons!

6. Binary Search: First Analysis For an array of size n, how many comparisons do we need to make to determine if x is in the array? (worst case) After each comparison, the array size is cut in half So how many times must we divide n by 2, before we get an array of size 1? log 2 n! So binary search should be O(log n), right??? Let’s try out some pseudocode and see… Thursday February 12, 2015 6

7. Binary Search Pseudocode Thursday February 12, 2015 7 function binarySearch(A, x): // Input: A, a sorted array // x, the item to find // Output: true if x is in A, else false if A.size == 0: return false if A.size == 1: return A[0] == x mid = A.size / 2 if A[mid] == x: return true if A[mid] < x: return binarySearch(A[mid + 1...end], x) if A[mid] > x: return binarySearch(A[0...mid – 1], x)

8. Binary Search Analyzed Since each recursive call cuts the problem size in half, the recurrence relation for binary search looks like: T(1) = c T(n) = T(n/2) + f(n) Where f(n) is the amount of work done at each level of recursion Thursday February 12, 2015 8

9. Binary Search Analyzed (2) Thursday February 12, 2015 9 Haaang on. What is this sketchy business here?? In order to pass a smaller array to the recursive call, we’re making a new array and copying over half the contents! This step’s O(n), kid… function binarySearch(A, x): if A.size == 0: O(1) return false O(1) if A.size == 1: O(1) return A[0] == x O(1) mid = A.size / 2 O(1) if A[mid] == x: O(1) return true O(1) if A[mid] < x: O(1) return binarySearch(A[mid + 1...end], x) if A[mid] > x: O(1) return binarySearch(A[0...mid – 1], x) (base case 1) (base case 2)

10. Binary Search Analyzed (3) Now that we know f(n) is O(n), we can solve our recurrence relation using plug ‘n’ chug Therefore, T(n) is O(n + log n), which is O(n). That’s just as bad as iterating through the whole array! Thursday February 12, 2015 10 this sum converges to 2n, as n increases f(n), a linear function of n

11. What went wrong? In our initial simulation of binary search, we found that it took only O(log n) comparisons to solve the problem But when it came to implementing the algorithm, copying half the array ended up costing us too much at each step! The runtime went back up to O(n) This is a very common pitfall when trying to implement efficient algorithms. Sometimes taking the most straightforward approach is not enough to achieve the fast runtime you hope for In the case of binary search, this means we need to implement the algorithm in-place. In other words, we can only use the array that was given to us as input. No copying allowed! Thursday February 12, 2015 11

12. In-Place Binary Search Thursday February 12, 2015 12 function binarySearch(A, lo, hi, x): // Input: A – a sorted array // lo, hi – two valid indices of the array // x – the item to find // Output: true if x is in the array between lo and hi, inclusive if lo >= hi: return A[lo] == x mid = (lo + hi) / 2 if A[mid] == x: return true if A[mid] < x: return binarySearch(A, mid + 1, hi, x) if A[mid] > x: return binarySearch(A, lo, mid – 1, x)

13. In-Place Binary Search (2) Now it’s clear that our binary search only performs a constant number of operations at each iteration The recurrence relation becomes: T(n) = T(n/2) + c 1 Plugging in, we get: T(1) = c 0 T(2) = T(1) + c 1 = c 0 + c 1 T(4) = T(2) + c 1 = c 0 + 2c 1 T(8) = T(4) + c 1 = c 0 + 3c 1 T(2 k ) = c 0 + kc 1 If we let n = 2 k, then: T(n) = c 0 + (log 2 n)c 1 So our in-place algorithm is O(log n)! Yay! Thursday February 12, 2015 13

14. In-Place Binary Search: Iterative Thursday February 12, 2015 14 function binarySearch(A, x): // Input: A – a sorted array // x – the item to find // Output: true if x is in the array lo = 0 hi = A.size - 1 while lo < hi: mid = (lo + hi) / 2 if A[mid] == x: return true if A[mid] < x: lo = mid + 1 if A[mid] > x: hi = mid – 1 return A[lo] == x Remember: Any recursive algorithm can be implemented iteratively!