1. CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms

2. The divide-and-conquer strategy solves a problem by: Breaking it into subproblems that are themselves smaller instances of the same type of problem. Recursively solving these subproblems. Appropriately combining their answers.

3. CSC 331: Algorithm Analysis Recurrence Relations

4. Divide-and-conquer algorithms often follow a generic pattern: They tackle a problem of size n by recursively solving a subproblems of size n/b and then combining these answers in O(n d ) time, for some a, b, d > 0. Their running time can therefore be captured by the equation T(n) = aT(n/b) + O(n d ).

5. If T(n) = aT(n/b) + O(n d ) for some constants a > 0, b > 1, and d ≥ 0, then The Master Theorem tells us the running times of most of the divide-and-conquer procedures we are likely to use.

6. CSC 331: Algorithm Analysis Binary Search

7. Given a sorted array of numbers, return the index of the number k or -1 if it’s not there. We could go through each number and check to see if that number is k. If so, return the index. What’s the running time? Instead, we want to USE the fact that the array is sorted. What is a better way? (Think high/low game).

8. 235671319100 Input: Look for the 13 in: 1319100 13 Look at middle number, 7. 13>7, so recurse on second half: Look at middle number, 19. 13<19, so recurse on first half: Look at middle number, 13. l found it, with only 3 recursions.

9. Search in part of Array A for element k, and return its index. Assumes A is sorted. function binarySearch(A[left..right], k) // Base case: 0 elements left, I didn’t find it. if right< left return Not found mid<- (right-left/2) + left if k = A[mid] return mid if k < A[mid] return A[left..mid-1] if k > A[mid] return A[mid+1..right]

10. What is the recurrence relation? What is the running time? function binarySearch(A[left..right], k) // Base case: 0 elements left, I didn’t find it. if right< left return Not found mid<- (right-left)/2 + left if k = A[mid] return mid if k < A[mid] return A[left..mid-1] if k > A[mid] return A[mid+1..right]

11. CSC 331: Algorithm Analysis Mergesort

12. The problem of sorting a list of numbers lends itself immediately to a divide-and-conquer strategy. Split the list into two halves. Recursively sort each half. Merge the two sorted sublists.

13. 1025371316 Input: 1025371316 1025371316 1025371316

14. 10 25371316 1235671013 2351016713 2103571316

15. function mergesort(a[1...n]) if n > 1: return merge(mergesort(a[1...n/2]), mergesort(a[n/2+ 1...n])) else: return a

16. function merge(x[1...k], y[1...m]) if k = 0: return y[1...m] if m = 0: return x[1...k] if x[1] ≤ y[1]: return x[1] ✪ merge(x[2...k], y[1...m]) else: return y[1] ✪ merge(x[1...k], y[2...m]) where ✪ is concatenation.

17. function merge(x[1...k], y[1...m]) if k = 0: return y[1...m] if m = 0: return x[1...k] if x[1] ≤ y[1]: return x[1] ✪ merge(x[2...k], y[1...m]) else: return y[1] ✪ merge(x[1...k], y[2...m]) This merge procedure does a constant amount of work per recursive call (provided the required array space is allocated in advance), for a total running time of O(k + m).

18. Thus merge’s are linear, and the overall time take by mergesort is T(n) = 2T(n/2) + O(n). a = 2, b = 2, d = 1 ➠ O(n log n)

19. CSC 331: Algorithm Analysis Medians

20. What is the median of [45, 1, 10, 30, 25]? How do you compute the median of n numbers? Sorting takes O(n log n), can we do better? 25

21. Sorting is doing far more work than we really need - - we just want the middle element and don’t care about the relative ordering of the rest of them. When looking for a recursive solution, it is paradoxically often easier to work with a more general version of the problem.

22. Selection Input: A list of numbers S; an integer k Output: The k th smallest element of S For instance, if k = 1, the minimum of S is sought. For median, k = S/ 2.

23. For any number v, split list S into three categories: ➡ elements smaller than v (S L ) ➡ elements equal to v (S V ) ➡ elements greater than v (S R ) The three sublists can be computed in linear time O(n).

24. For instance, if the array 2365218131120541 S:S: is split on v = 13, the three subarrays generated are: 25811541 SL:SL: 13 Sv:Sv: 362120 SR:SR:

25. Now we can instantly narrow down the search to one of these sublists. If we wanted, say, the ninth-smallest element of S, we know it must be the first-smallest element of S R since | S L | + | S V | = 8. 25811541 SL:SL: 13 SV:SV: 362120 SR:SR: selection(S,9) = selection(S R,1)

26. if k ≤ | S L | selection(S,k) = selection(S L, k) if | S L | < k ≤ | S L | + | S V | selection(S,k) = v if k > | S L | + | S V | selection(S,k) = selection(S R, k - | S L | - | S V |)

27. If we could always pick v so that S L, S R ≈ 1/2(S) T(n) = T(n/2) + O(n). a = 1, b = 2, d = 1 ➠ O(n) But this requires picking v to be the median, which is our ultimate goal!

28. We will pick v randomly from S. n + (n - 1) + (n - 2) +... + n/2 = Θ(n 2 ) Naturally, the running time of our algorithm depends on the random choice of v. The worst-case scenario would be if we kept picking v to be the largest (or smallest) element of the array.

29. Θ(n) The best-case scenario would be if we picked v to be the median element of the array. Where does the average running time lie?

30. v is good if it lies within the 25th to 75th percentile of the array. A good v ensures that the sublists S L and S R have size at most three-fourths that of S. Half of the elements of any list must fall between the 25th and 75th percentile.

31. Lemma: On average a fair coin needs to be tossed two times before a “heads” is seen. After two split operations on average, the array will shrink to at most three-fourths of its size.

32. Letting T(n) be the expected running time on an array of size n, we get T(n) ≤ T(3n/4) + O(n). a = 1, b = 4/3, d = 1 ➠ O(n)

33. So, the overall ideas: Often the obvious way to solve a problem is not the most efficient. Try to find a way to divide the problem into pieces. IF you know the answer (recursively) to the piece, does that help you find the overall answer? Once you write a recursive function, see if its solution is quicker than your iterative way.

34. Practice: (Book, problem 2.23) An array A[1..n] has a majority element if one element occurs in more than half of the entries. Design an O(nlgn) algorithm to decide if the array has a majority element and if so, what the element is. You cannot sort the array, or ask if A[i]>A[j], since comparisons are not supported on the array type. (For example, if the array were of GIF files.) You can, however, ask if A[i] = A[j].