1. Young CS 331 D&A of Algo. Topic: Divide and Conquer1 Divide-and-Conquer General idea: Divide a problem into subprograms of the same kind; solve subprograms using the same approach, and combine partial solution (if necessary).

2. Young CS 331 D&A of Algo. Topic: Divide and Conquer2 The straightforward algorithm: max ← min ← A (1); for i ← 2 to n do if A (i) > max, max ← A (i); if A (i) < min, min ← A (i); Key comparisons: 2(n – 1) 1.Find the maximum and minimum The problem: Given a list of unordered n elements, find max and min

3. Young CS 331 D&A of Algo. Topic: Divide and Conquer3 List List 1 List 2 n elementsn/2 elements n/2 elements min, max min1, max1min2, max2 min = MIN ( min1, min2 ) max = MAX ( max1, max2)

4. Young CS 331 D&A of Algo. Topic: Divide and Conquer4 The Divide-and-Conquer algorithm: procedure Rmaxmin (i, j, fmax, fmin); // i, j are index #, fmax, begin // fmin are output parameters case: i = j:fmax ← fmin ← A (i); i = j –1:if A (i) < A (j) then fmax ← A (j); fmin ← A (i); elsefmax ← A (i); fmin ← A (j); else:mid ← ; call Rmaxmin (i, mid, gmax, gmin); call Rmaxmin (mid+1, j, hmax, hmin); fmax ← MAX (gmax, hmax); fmin ← MIN (gmin, hmin); end end;

5. Young CS 331 D&A of Algo. Topic: Divide and Conquer5 Example: find max and min in the array: 22, 13, -5, -8, 15, 60, 17, 31, 47 ( n = 9 ) Index:12 3456789 Array:2213-5-815601731 47 Rmaxmin(1, 9, 60, -8) 1, 5, 22, -8 6, 9, 60, 17 1, 3, 22, -5 4, 5, 15, -8 6,7, 60, 17 8, 9, 47, 31 1,2, 22, 13 3, 3,-5, -5 (1) (2) (3)(4) (5) (6) (7) (8)

6. Young CS 331 D&A of Algo. Topic: Divide and Conquer6 Analysis: For algorithm containing recursive calls, we can use recurrence relation to find its complexity T(n) - # of comparisons needed for Rmaxmin Recurrence relation:

7. Young CS 331 D&A of Algo. Topic: Divide and Conquer7 Assume n = 2 k for some integer k

8. Young CS 331 D&A of Algo. Topic: Divide and Conquer8 Theorem: Claim: where are a, b, c constants They don’t have to be the same constant. We make them the same here for simplicity. It will not affect the overall result.

9. Young CS 331 D&A of Algo. Topic: Divide and Conquer9 Proof: Assume n = c k

10. Young CS 331 D&A of Algo. Topic: Divide and Conquer10 If a < c, is a constant T(n) = bn · a constant  T(n) = O(n) if a = c,

11. Young CS 331 D&A of Algo. Topic: Divide and Conquer11 If a > c,

12. Young CS 331 D&A of Algo. Topic: Divide and Conquer12 2.Matrix multiplication The problem: Multiply two matrices A and B, each of size

13. Young CS 331 D&A of Algo. Topic: Divide and Conquer13 The traditional way: use three for-loop

14. Young CS 331 D&A of Algo. Topic: Divide and Conquer14 The Divide-and-Conquer way: transform the problem of multiplying A and B, each of size [n×n] into 8 subproblems, each of size

15. Young CS 331 D&A of Algo. Topic: Divide and Conquer15 which an 2 is for addition so, it is no improvement compared with the traditional way

16. Young CS 331 D&A of Algo. Topic: Divide and Conquer16 Example: use Divide-and-Conquer way to solve it as following:

17. Young CS 331 D&A of Algo. Topic: Divide and Conquer17 Strassen’s matrix multiplication:  Discover a way to compute the C ij ’s using 7 multiplications and 18 additions or subtractions

18. Young CS 331 D&A of Algo. Topic: Divide and Conquer18 Algorithm : procedure Strassen (n, A, B, C) // n is size, A,B the input matrices, C output matrix begin if n = 2, else (cont.)

19. Young CS 331 D&A of Algo. Topic: Divide and Conquer19 else Partition A into 4 submatrices: ; Partition B into 4 submatrices: ; call Strassen ( ; call Strassen ( call Strassen ( ;

20. Young CS 331 D&A of Algo. Topic: Divide and Conquer20 call Strassen ( ; end;

21. Young CS 331 D&A of Algo. Topic: Divide and Conquer21 Analysis:

22. Young CS 331 D&A of Algo. Topic: Divide and Conquer22 Assume n = 2 k for some integer k

23. Young CS 331 D&A of Algo. Topic: Divide and Conquer23 3.Integer multiplication The problem: Multiply two large integers (n digits) The traditional way: Use two loops, it takes O(n 2 ) operations

24. Young CS 331 D&A of Algo. Topic: Divide and Conquer24 The Divide-and-Conquer way: Suppose x and y are large integers, divide x into two parts (a and b), and divide y into c and d. x: y: ab cd So, transform the problem of multiplying two n-digit integers into four subproblems of multiplying two -digit integers.

25. Young CS 331 D&A of Algo. Topic: Divide and Conquer25 Worst-Case is: However, it is same as the traditional way. Instead, we write: Worst-Case is:

26. Young CS 331 D&A of Algo. Topic: Divide and Conquer26 The algorithm by Divide-and-Conquer: function multiplication (x,y) begin n = MAX ( # of digits in x, # of digits in y); if (x = 0) or (y = 0), return 0; else if (n = 1), return x*y in the usual way; else m = ; a = x divide 10 m ; b = x rem 10 m ; c = y divide 10 m ; d = y rem 10 m ; p1= MULTIPLICATION (a, c); p2= MULTIPLICATION (b, d); p3 = MULTIPLICATION (a+b, c+d); return ; end;

27. Young CS 331 D&A of Algo. Topic: Divide and Conquer27 4.Merge Sort The problem: Given a list of n numbers, sort them in non-decreasing order idea: Split each part into 2 equal parts and sort each part using Mergesort, then merge the two sorted sub-lists into one sorted list. The algorithm: procedure MergeSort (low, high) begin if then call MergeSort (low, mid); call MergeSort (mid+1, high); call Merge (low, mid, high); end;

28. Young CS 331 D&A of Algo. Topic: Divide and Conquer28 procedure Merge (low, mid, high) begin while (i  mid and j  high) if A(i) < A(j), then U(k)  A(i); i++; else U(k)  A(j); j++; k++; if (i > mid) move A(j) through A(high) to U(k) through U(high); else move A(i) through A(mid) to U(k) through U(high); for p  low to high A(p)  U(p); end;

29. Young CS 331 D&A of Algo. Topic: Divide and Conquer29 Analysis: Worst-Case for MergeSort is: Best-Case & Average-Case? Merge sort pays no attention to the original order of the list, it will keep divide the list into half until sub-lists of length 1, then start merging. Therefore Best-Case & Average-Case are the same as the Worst-Case. O(nLogn)

30. Young CS 331 D&A of Algo. Topic: Divide and Conquer30 5.Quick Sort The problem: Same as Merge Sort Compared with Merge Sort: Quick Sort also use Divide-and-Conquer strategy Quick Sort eliminates merging operations

31. Young CS 331 D&A of Algo. Topic: Divide and Conquer31 How it works? Use partition Eg: Sort array 65, 70, 75, 80, 85, 60, 55, 50, 45 65 70 75 80 85 60 55 50 45 60 55 50 4570 75 80 85 55 50 45 50 45 45 75 80 85 80 85 85 Pivot item 65 60 65 70 55 60 65 70 75 55 60 65 70 75 80

32. Young CS 331 D&A of Algo. Topic: Divide and Conquer32 The algorithm : procedure QuickSort (p, q) begin if p < q, thencall Partition (p, q, pivotposition); call QuickSort (p, pivotposition –1); call QuickSort (pivotposition+1, q); end; procedure Partition (low, high, pivotposition) begin v  A(low); j  low; for i  (low + 1) to high if A(i) < v, j++; A(i)  A(j); pivotposition  j; A(low)  A(pivotposition); end;

33. Young CS 331 D&A of Algo. Topic: Divide and Conquer33 Analysis: Worst-Case: Call Partition O(n) times, each time takes O(n) steps. So O(n 2 ), and it is worse than Merge Sort in the Worst-Case. Best-Case: Split the list evenly each time.

34. Young CS 331 D&A of Algo. Topic: Divide and Conquer34 Average-Case: Assume pivot is selected randomly, so the probability of pivot being the k th element is equal,  k. C(n) = # of key comparison for n items average it over all k, (C A (n) is average performance) multiply both side by n

35. Young CS 331 D&A of Algo. Topic: Divide and Conquer35 replace n by n-1 subtract (2) from (1)

36. Young CS 331 D&A of Algo. Topic: Divide and Conquer36 divide n(n + 1) for both sides  Best-Case = Average-Case

37. Young CS 331 D&A of Algo. Topic: Divide and Conquer37 6. Selection The problem: Given a list of n elements find the k th smallest one Note: special cases: when k = 1, min when k = n, max The solving strategy: If use sorting strategy like MergeSort, Both Best-Case and Worst-Case take O(n Log n) If use Quick Sort, Best-Case is O(n) (when pivot is the k th smallest one) Worst-Case will call partition O(n) times, each time takes O(n), so Worst-Case is O(n 2 ).

38. Young CS 331 D&A of Algo. Topic: Divide and Conquer38 Select1 – the first algorithm idea: use “partition” (from Quick Sort) repeatedly until we find the k th element. For each iteration: If pivot position = k  Done! If pivot position < k  to select (k-i) th element in the 2 nd sub-list. If pivot position > k  to select k th element in the 1 st sub-list. pivot i = pivot position 1 st sub-list2 nd sub-list

39. Young CS 331 D&A of Algo. Topic: Divide and Conquer39 procedure Select1 (A, n, k)// A is array, n is # of array, k is key begin m  1; j  n; loop call Partition (m, j, pivotposition); case: k = pivotposition: return A(k); k < pivotposition: j  pivotposition – 1; else: m  pivotposition + 1; end loop; end;

40. Young CS 331 D&A of Algo. Topic: Divide and Conquer40 By choosing pivot more carefully, we can obtain a selection algorithm with Worst-Case complexity O(n) How: Make sure pivot v is chosen s.t. at least some fraction of the elements will be smaller than v and at least some other fraction of elements will be greater than v. mm (median of medians) Non- decreasing Order (all columns) The middle row is in non-decreasing order Elements  mm Elements  mm Example: 35 (n) elements divide into 7 (n/r) subsets of size 5 (r)

41. Young CS 331 D&A of Algo. Topic: Divide and Conquer41 By choosing pivot more carefully, we can obtain a selection algorithm with Worst-Case complexity O(n) Use mm (median of medians) rule to choose pivot procedure Select2 (A, n, k) begin if n  r, then sort A and return the k th element; divide A into subset of size r each, ignore excess elements, and let be the set of medians of the subsets;

42. Young CS 331 D&A of Algo. Topic: Divide and Conquer42 Select2 ; use “Partition” to partition A using v as the pivot; // Assume v is at pivotposition case: k = pivotposition: return (v); k < pivotposition: let S be the set of elements A (1,…, pivotposition –1), return Select2 (S, pivotposition –1, k); else: let R be the set of element A (pivotposition+1,…, n), return Select2 (R, n- pivotposition, k-pivotposition); end case; end;

43. Young CS 331 D&A of Algo. Topic: Divide and Conquer43 Analysis: How many M i ’s  (or  ) mm? At least How many elements  (or  ) mm? At least How many elements > mm (or < mm)? At most Assume r = 5

44. Young CS 331 D&A of Algo. Topic: Divide and Conquer44 Therefore, procedure Select2 the Worst-Case complexity is: where c is chosen sufficiently large so that T(n)  cn for n < 24. Proof: Use induction to show T(n)  20  cn (n  24) IB (induction base):

45. Young CS 331 D&A of Algo. Topic: Divide and Conquer45 IH (induction hypothesis): Suppose IS (induction solution): When n = m;  T(n) = O(n) complexity of Select2 of n elements