Divide-and-Conquer

Outline Introduction Merge Sort Quick Sort Closest pair of points Large integer multiplication

Steps Divide the problem into a number of subproblems. Conquer the subproblems by solving them recursively. If the subproblem sizes are small enough, however, just solve the subproblems in a straightforward manner. Combine the solutions to the subproblems into the solution for the original problem.

Merge Sort

Steps in Merge Sort Divide Divide the sequence to be sorted into two halves. Conquer Sort the two subsequences recursively using merge sort. Combine Merge the two sorted subsequences to produce the sorted answer.

Merge Sort: Example divide 3686 conquer 3686 combine divide 4512 conquer combine conquer combine

Merge Algorithm MERGE(A, p, q, r) ► initialize left and right arrays n 1 = q − p + 1 n 2 = r − q create array L[1.. n 1 + 1] create array R[1.. n 2 + 1] for i = 1 to n 1 do L[i ] = A[p + i − 1] for j = 1 to n 2 do R[ j ] = A[q + j ] L[n 1 + 1]=∞ R[n 2 + 1]=∞ i = 1 j = 1 ► merge two arrays for k = p to r do if L[i ] ≤ R[ j ] thenA[k] = L[i ] i = i + 1 elseA[k] = R[ j ] j = j + 1

Merge: Complexity MERGE(A, p, q, r) ► initialize left and right arrays n 1 = q − p + 1 n 2 = r − q create array L[1.. n 1 + 1] create array R[1.. n 2 + 1] for i = 1 to n 1 do L[i ] = A[p + i − 1] for j = 1 to n 2 do R[ j ] = A[q + j ] L[n 1 + 1]=∞ R[n 2 + 1]=∞ i = 1 j = 1 ► merge two arrays for k = p to r do if L[i ] ≤ R[ j ] thenA[k] = L[i ] i = i + 1 elseA[k] = R[ j ] j = j + 1  (n) (n)  (1) (n)(n) (n)(n)

Merge Sort Algorithm MERGE-SORT(A, p, r) if p < r then q =  (p + r)/2  MERGE-SORT(A, p, q) MERGE-SORT(A, q + 1, r) MERGE(A, p, q, r)

Merge Sort: Complexity MERGE-SORT(A, p, r) if p < r then q =  (p + r)/2  MERGE-SORT(A, p, q) MERGE-SORT(A, q + 1, r) MERGE(A, p, q, r) If n  1, T(n) = 2 T(n/2) +  (n) If n=1, T(n) =  (1) T(n) =  (n lg n) T(n/2) (n)(n)  (1)

Master Theorem Let a ≥ 1 and b > 1 be constants, T(n) = aT(n/b) + f(n) be defined on the nonnegative integers, where f(n) =  (n d ) Then T (n) can be bounded asymptotically as follows. 1. If a < b d, then T(n) =  (n d ). 2. If a = b d, then T(n) =  (n d lg n). 3. If a > b d, then T(n) =  (n log b a ).

Solving Recurrence Equation T(n) = 2 T(n/2) +  (n) when n > 1 T(n) =  (1) when n = 1 That is, T(n) =  (n lg n) … cn … 2c2c2c2c2c2c2c2c2c2c … 4c4c4c4c cn/2 cn lg n

Quicksort

Steps in Quicksort Divide Partition (rearrange) the array A[p.. r] into two subarrays A[p.. q −1] and A[q +1.. r] such that each element of A[p.. q −1]  A[q], A[q]  each element of A[q r] Compute the index q. Conquer Sort A[p.. q−1] and A[q +1.. r] by recursive calls. Combine Since the subarrays are sorted in place, nothing else need to be done.

Quicksort: Example

Quicksort Algorithm QUICKSORT(A, p, r) if p < r thenq = PARTITION(A, p, r) QUICKSORT(A, p, q − 1) QUICKSORT(A, q + 1, r) Initial call: QUICKSORT(A, 1, length[A]).

Partition: Example

Partition Algorithm PARTITION(A, p, r) x = A[r] i = p − 1 for j = p to r − 1 doif A[ j ] ≤ x then i = i + 1 exchange A[i] and A[ j] exchange A[i + 1] and A[r] return i + 1

PARTITION(A, p, r) x = A[r] i = p − 1 for j = p to r − 1 doif A[ j ] ≤ x then i = i + 1 exchange A[i] and A[ j] exchange A[i + 1] and A[r] return i + 1 Partition: Complexity O(1) O(n)O(n)

Quicksort: Complexity QUICKSORT(A, p, r) if p < r thenq = PARTITION(A, p, r) QUICKSORT(A, p, q − 1) QUICKSORT(A, q + 1, r) O(n)O(n) T(n/?)

Quicksort: Analysis Worst case: T(n) = T(n-1) + T(0) +  (n) = T(n-1) +  (n) T(n)  (n2)T(n)  (n2) Best case: T(n)= 2 T(n/2) +  (n) T(n)   (n lg n)

Average-case Analysis T(n)   (n lg n)

Closest Pairs of Points

Problem Given a set P of points in 2- dimensional space, find a closest pair of points. Distance between two points are measured by Euclidean distance --  (  x 2 +  y 2 ) Brute force method takes O (n 2 ).

Closest Pair of Points: Example divide again conquer combine

Steps in Finding Closest Pair of Points Divide Vertically divide P into PL and PR. PL is to the left and PR is to the right of the dividing line. Conquer Recursively find closest pair of points among PL and PR. dL and dR are the closest-pair distances returned for PL and PR, and d = min(dL, dR). Combine The closest pair is either the pair with distance d found by one of the recursive calls, or it is a pair of points with one point in PL and the other in PR. Find a pair whose distance < d if there exists.

P Closest point on the other side PRPL S dd Only 8 points need to be considered

P Why 8 Points PRPL S d/2 dd

Combine Let PRY be the array of points in PR sorted by the value of Y. For each point s in the d-width strip of PL, find a point t in PR such that dist(s, t) < d. To find t, look in the array PRY for 8 such points. If there is such a point, let d be the dist(s, t).

Closest Pair of Points: Algorithm Let px and py be the points in P, sorted by the value of x and y, respectively. ClosestPair(px, py) if (|px|<=3) thenreturn(BruteForceCP(px)) Using px, find the line L which vertically divides P into 2 halves. Using the dividing line L, split px into pxL and pxR. Using the dividing line L, split py into pyL and pyR. dL = ClosestPair(pxL, pyL) dR = ClosestPair(pxR, pyR) d = min(dL, dR) return ( combineCP(pyL, pyR, d) )

Closest Pair of Points: Complexity Let px and py be the points in P, sorted by the value of x and y, respectively. ClosestPair(px, py) if (|px|<=3) thenreturn(BruteForceCP(px)) Using px, find the line L which vertically divides P into 2 halves. Using the dividing line L, split px into pxL and pxR. Using the dividing line L, split py into pyL and pyR. dL = ClosestPair(pxL, pyL) dR = ClosestPair(pxR, pyR) d = min(dL, dR) return ( combineCP(pyL, pyR, d) ) O(1) O(n)O(n) 2T(n/2) O(1) O(n)O(n)

Solving the recurrence equation T(n) = 2 T(n/2) + O(n) T(n)  Ο(n log n)

Large Integer Multiplication

Dividing Multiplication x = (x 1 *2 n/2 + x 0 ) y = (y 1 *2 n/2 + y 0 ) x y = (x 1 *2 n/2 + x 0 ) (y 1 *2 n/2 + y 0 ) = x 1 y 1 *2 n + (x 0 y 1 + x 1 y 0 )*2 n/2 + x 0 y 0 (x 1 + x 0 ) ( y 1 + y 0 ) = x 1 y 1 + x 0 y 1 + x 1 y 0 + x 0 y 0 = (x 0 y 1 +x 1 y 0 )+(x 1 y 1 + x 0 y 0 ) (x 0 y 1 +x 1 y 0 ) = (x 1 + x 0 )( y 1 + y 0 ) - (x 1 y 1 + x 0 y 0 )

Algorithm recMul(x, y) n = maxbit(x, y) if (n<k) then return (x*y) x1 = x div 2 n/2 x0 = x mod 2 n/2 y1 = y div 2 n/2 y0 = y mod 2 n/2 p = recMul(x1+x0, y1+y0) q = recMul(x1, y1) r = recMul(x0, y0) return(q*2 n +(p-q-r)*2 n/2 +r)

Complexity recMul(x, y) n = maxbit(x, y) if (n<k) then return (x*y) x1 = x div 2 n/2 x0 = x mod 2 n/2 y1 = y div 2 n/2 y0 = y mod 2 n/2 p = recMul(x1+x0, y1+y0) q = recMul(x1, y1) r = recMul(x0, y0) return(q*2 n +(p-q-r)*2 n/2 +r) O(1) 3T(n/2) O(1)

Solving the recurrence equation T(n) = 3 T(n/2) + O(n) T(n)  Ο(n log 2 3 ) T(n)  Ο(n 1.59 )