Topic: Divide and Conquer General idea: Divide a problem into subprograms of the same kind; solve subprograms using the same approach, and combine partial solution (if necessary). Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer List List 1 List 2 n elements n/2 elements n/2 elements min, max min1, max1 min2, max2 min = MIN ( min1, min2 ) max = MAX ( max1, max2) Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer The Divide-and-Conquer algorithm: procedure Rmaxmin (i, j, fmax, fmin); // i, j are index #, fmax, begin // fmin are output parameters case: i = j: fmax ← fmin ← A (i); i = j –1: if A (i) < A (j) then fmax ← A (j); fmin ← A (i); else fmax ← A (i); fmin ← A (j); else: mid ← ; call Rmaxmin (i, mid, gmax, gmin); call Rmaxmin (mid+1, j, hmax, hmin); fmax ← MAX (gmax, hmax); fmin ← MIN (gmin, hmin); end end; Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer Example: find max and min in the array: 22, 13, -5, -8, 15, 60, 17, 31, 47 ( n = 9 ) Index: 1 2 3 4 5 6 7 8 9 Array: 22 13 -5 -8 15 60 17 31 47 Rmaxmin(1, 9, 60, -8) 1, 5, 22, -8 6, 9, 60, 17 1, 3, 22, -5 4, 5, 15, -8 6,7, 60, 17 8, 9, 47, 31 1,2, 22, 13 3, 3,-5, -5 (1) (6) (7) (2) (5) (8) (3) (4) Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer Find the maximum and minimum The problem: Given a list of unordered n elements, find max and min The straightforward algorithm:   max ← min ← A (1); for i ← 2 to n do if A (i) > max, max ← A (i); if A (i) < min, min ← A (i); Key comparisons: 2(n – 1) Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer Analysis: For algorithm containing recursive calls, we can use recurrence relation to find its complexity T(n) - # of comparisons needed for Rmaxmin Recurrence relation: Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer Assume n = 2k for some integer k Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer Theorem: Claim: They don’t have to be the same constant. We make them the same here for simplicity. It will not affect the overall result. where are a, b, c constants Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer Proof: Assume n = ck Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer If a < c, is a constant T(n) = bn · a constant  T(n) = O(n) if a = c, Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer If a > c, Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer 2. Matrix multiplication The problem:   Multiply two matrices A and B, each of size Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer The traditional way:   use three for-loop Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer The Divide-and-Conquer way: transform the problem of multiplying A and B, each of size [n×n] into 8 subproblems, each of size Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer which an2 is for addition so, it is no improvement compared with the traditional way Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer Example: use Divide-and-Conquer way to solve it as following: Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer Strassen’s matrix multiplication: · Discover a way to compute the Cij’s using 7 multiplications and 18 additions or subtractions Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer Algorithm : procedure Strassen (n, A, B, C) // n is size, A,B the input matrices, C output matrix begin if n = 2, else (cont.) Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer else Partition A into 4 submatrices: ; Partition B into 4 submatrices: ; call Strassen ( ; call Strassen ( call Strassen ( ; Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer call Strassen ( ; end; Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer Analysis: Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer Assume n = 2k for some integer k Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer 3. Integer multiplication The problem:   Multiply two large integers (n digits) The traditional way: Use two loops, it takes O(n2) operations Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer The Divide-and-Conquer way:   Suppose x and y are large integers, divide x into two parts (a and b), and divide y into c and d. x: y: a b c d So, transform the problem of multiplying two n-digit integers into four subproblems of multiplying two -digit integers. Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer Worst-Case is:   However, it is same as the traditional way. Instead, we write: Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer The algorithm by Divide-and-Conquer: function multiplication (x,y) begin n = MAX ( # of digits in x, # of digits in y); if (x = 0) or (y = 0), return 0; else if (n = 1), return x*y in the usual way; else m = ; a = x divide 10m; b = x rem 10m; c = y divide 10m; d = y rem 10m; p1= MULTIPLICATION (a, c); p2= MULTIPLICATION (b, d); p3 = MULTIPLICATION (a+b, c+d); return ; end; Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer 4. Merge Sort The problem:  Given a list of n numbers, sort them in non-decreasing order idea: Split each part into 2 equal parts and sort each part using Mergesort, then merge the two sorted sub-lists into one sorted list.   The algorithm: procedure MergeSort (low, high) begin if then call MergeSort (low, mid); call MergeSort (mid+1, high); call Merge (low, mid, high); end; Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer procedure Merge (low, mid, high) begin while (i  mid and j  high) if A(i) < A(j), then U(k) A(i); i++; else U(k) A(j); j++; k++; if (i > mid) move A(j) through A(high) to U(k) through U(high); else move A(i) through A(mid) to U(k) through U(high); for p  low to high A(p)  U(p); end; Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer Analysis: Worst-Case for MergeSort is: Best-Case & Average-Case? Merge sort pays no attention to the original order of the list, it will keep divide the list into half until sub-lists of length 1, then start merging. Therefore Best-Case & Average-Case are the same as the Worst-Case. O(nLogn) Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer 5. Quick Sort The problem:   Same as Merge Sort Compared with Merge Sort: Quick Sort also use Divide-and-Conquer strategy Quick Sort eliminates merging operations Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer How it works?   Use partition Eg: Sort array 65, 70, 75, 80, 85, 60, 55, 50, 45 65 70 75 80 85 60 55 50 45 60 55 50 45 70 75 80 85 55 50 45 50 45 45 75 80 85 80 85 85 Pivot item 65 60 65 70 55 60 65 70 75 55 60 65 70 75 80 Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer The algorithm: procedure QuickSort (p, q) begin if p < q, then call Partition (p, q, pivotposition); call QuickSort (p, pivotposition –1); call QuickSort (pivotposition+1, q); end; procedure Partition (low, high, pivotposition) v  A(low); j  low; for i  (low + 1) to high if A(i) < v, j++; A(i)  A(j); pivotposition  j; A(low)  A(pivotposition); Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer Analysis: Worst-Case: Call Partition O(n) times, each time takes O(n) steps. So O(n2), and it is worse than Merge Sort in the Worst-Case.   Best-Case: Split the list evenly each time. Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer Average-Case: Assume pivot is selected randomly, so the probability of pivot being the kth element is equal, k. C(n) = # of key comparison for n items average it over all k, (CA(n) is average performance) multiply both side by n Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer replace n by n-1 subtract (2) from (1) Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer divide n(n + 1) for both sides  Best-Case = Average-Case Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer 6. Selection (Project #2) The problem: Given a list of n elements find the kth smallest one Note: special cases: when k = 1, min when k = n, max The solving strategy: Algorithm #1 Use sorting strategy like MergeSort, sort the whole list and then return the kth element of the sorted list Both Best-Case and Worst-Case take O(n Log n) Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer Algorithm #2 idea: use “partition” (from Quick Sort) procedure repeatedly until we find the kth smallest element.   For each iteration: If pivot position i = k  Done! If pivot position i < k  to select (k-i)th element in the 2nd sub-list. If pivot position i > k  to select kth element in the 1st sub-list. Best-Case is O(n) (when pivot is the kth smallest one) Worst-Case will call partition O(n) times, each time takes O(n), so Worst-Case is O(n2). i = pivot position pivot 1st sub-list 2nd sub-list Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer procedure Algorththm#2 (A, n, k) // A is array, n is # of array, k is key begin m  1; j  n; loop call Partition (m, j, pivotposition); case: k = pivotposition: return A(k); k < pivotposition: j  pivotposition – 1; else: m  pivotposition + 1; k  k - pivotposition; end loop; end; Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer Algorithm #3 idea: Same as Algorithm#2 except it always find a special element of the list called MM (Median of Medians), and use MM as the pivot.   In class KLA activity to explain MM. We can show that both Best-Case and Worst-Case take O(n) Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer By choosing pivot more carefully, we can obtain a selection algorithm with Worst-Case complexity O(n) How: Make sure pivot v is chosen s.t. at least some fraction of the elements will be smaller than v and at least some other fraction of elements will be greater than v. Example: 35 (n) elements divide into 7 (n/r) subsets of size 5 (r) Elements  mm Non- decreasing Order (all columns) mm (median of medians) The middle row is in non-decreasing order Elements  mm Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer By choosing pivot more carefully, we can obtain a selection algorithm with Worst-Case complexity O(n) Use mm (median of medians) rule to choose pivot procedure Select2 (A, n, k) begin if n  r, then sort A and return the kth element; divide A into subset of size r each, ignore excess elements, and let be the set of medians of the subsets; Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer Select2 ; use “Partition” to partition A using v as the pivot; // Assume v is at pivotposition case: k = pivotposition: return (v); k < pivotposition: let S be the set of elements A (1,…, pivotposition –1), return Select2 (S, pivotposition –1, k);   else: let R be the set of element A (pivotposition+1,…, n), return Select2 (R, n- pivotposition, k-pivotposition);   end case; end; Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer Analysis: How many Mi’s  (or ) mm? At least How many elements  (or ) mm? How many elements > mm (or < mm)? At most Assume r = 5 Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer Therefore, procedure Select2 the Worst-Case complexity is: where c is chosen sufficiently large so that T(n)  cn for n < 24. Proof:   Use induction to show T(n)  20  cn (n  24) IB (induction base): Young CS 331 D&A of Algo. Topic: Divide and Conquer

Topic: Divide and Conquer IH (induction hypothesis):   Suppose IS (induction solution): When n = m;  T(n) = O(n) complexity of Select2 of n elements Young CS 331 D&A of Algo. Topic: Divide and Conquer