Chapter 2. Divide-and-conquer Algorithms Computer Algorithms Chapter 2. Divide-and-conquer Algorithms Some of contents in this lecture slides are from textbook Sanjoy Dasgupta, Christos Papdimitriou and Umesh Vazirani, Algorithms, McGraw-Hill, 2008.

Table of Contents Basic Strategy of Divide-and-conquer Multiplication Binary Search Merge Sort Finding Median Matrix Multiplication

Divide-and-Conquer Strategy A divide-and-conquer strategy solves a problem by Break a problem into sub problems that are themselves smaller instances of the same type of problem Solve sub problems recursively. Appropriately combine their answers

Multiplication Multiplying the two n-digit numbers x, y using divide-and-conquer algorithm First step: split each digit into two parts, which is n/2 bits long.

then, x⋅y can be written, Second Step: get values of sub equations xLyL, xLyR, xRyL, xRyR recursively. Third step: compute x⋅y through the above expression using xLyL, xLyR, xRyL, xRyR obtained from the second step.

Multiplication Analysis Time Complexity The algorithm will run in T(n) = 4T(n/2)+O(n) 4 multiplications and each multiplication is done from half sized bit-number. Every recursive step requires O(n) addition. So O(n2)

Improvement Since We can calculate only three multiplications (xL+xR)(yL+yR), xLyL, xRyR rather than using four multiplications xLyR, xRyL, xLyL, xRyR. So, we have three multiplications at each recursive step, and each step takes O(n) additions. T(n) = 3T(n/2) + O(n) ⇒ O(n1.59)

Multiplication Algorithm function multiply (x, y) Input: Positive binary integers x and y Output: Their product n = max(size of x, size of y) if n = 1, return xy x_l = leftmost ⌜n/2⌝ bits of x x_r = rightmost ⌜n/2⌝ bits of x y_l = leftmost ⌜n/2⌝ bits of y y_r = rightmost ⌜n/2⌝ bits of y P1 = multiply(x_l, y_l) P2 = multiply(x_r, y_r) P3 = multiply(x_l + x_r, y_l + y_r) return P1 x 2n + (P3 - P1 - P2) x 2n/2 + P2

Divide-and-Conquer Integer Multiplication

Recurrence Relations Master Theorem If T(n) = a(T(⎡n/b⎤) + O(nd) for some constants a > 0, b > 1, and d ≥ 0, then

Binary Search The most popular divide-and-conquer algorithm. Finding a key k from the sorted list e.g. A. comparing k with the middle located value v in A. If k < v, then recursively search the first half, if k = v, done. Otherwise, search the second half recursively. Time Complexity: T(n) = T(⌜n/2⌝) + O(1). So O(log n).

Merge Sort Sorting using divide-and-conquer approach. Split a list into two halves, recursively sort each half, and then merge the two sorted sublists. function mergesort(a[1... n]) Input: An array of numbers a[1 ... n] Output: A sorted version of this array If n > 1: return merge(mergesort(a[1 ... ⌞n/2⌟]), mergesort(a[⌞ n/2 ⌟ + 1 ... n])) else return a

Time Complexity merge() takes linear time O(k + l). function merge(x[1..k],y[1..l) Input: two arrays of x[1..k] and y[1..l] Output: a sorted array if k = 0, return y[1..l] if l = 0, return x[1..k] if x[1] <= y[1] return concatenate(x[1], merge(x[2..k], y[1..l])) else return concatenate(y[1], merge(x[1..k], y[2..l])) Time Complexity merge() takes linear time O(k + l). So total merge sort takes T(n) = 2T(n/2) + O(n). So O(n log n).

Finding Medians Median of a list of numbers is its 50th percentile: half the numbers are bigger than it, and half are smaller. One method First sort the list and find the middle located value. Sorting takes O(n log n). We have to design faster algorithm. If we can’t, the method above is best.

Divide-and-Conquer Approach to Find Median Finding median problem is reduced to finding k-th smallest element problem. Pick any element v from a list S. Split a list S into three parts: elements smaller than v, those equal to v, and those greater than v.  partition algorithm Narrow search into one of sub lists based on k.

Let v = 5 and S be Then S is split into three parts. |S| = 11, |SL| = 3, |Sv| = 2. So the median will be located in SR, and search the lowest value of SR. The algorithm is defined as:  

Time Complexity Assume we are so lucky and we always split the list into half-size, then But surely we are not lucky. In worst case, total execution time will be By reasonable probability, we can split the list into three quarter size average. So total execution time will be T(n) = O(n).

Matrix Multiplication The product Z of two n × n matrices X and Y, with (i,j)th entry is Time Complexity: O(n3) ⇐ O(n) operation for each n2 number of entries.

Divide-and-Conquer Matrix Multiplication Divide X and Y such that Then XY will be computed recursively Execution Time is

Improvement Strassen’s method New running time is

Finding Closest Pair  

Example y 8 11 13 6 3 5 14 2 7 9 12 1 10 4 x

Divide and Conquer Approach Split x-y space into two partitions get the closest pair and its distance from each partition Take the smaller one from two. So algorithm would be Closest_Pair(P) Split P into P_L and P_R such that each partition has almost equal number of points. d_L = Closest_Pair(P_L) d_R = Closest_Pair(P_R) pick and return smaller points pair between d_L and d_R

Example y 8 11 13 6 3 5 14 2 7 9 12 1 10 4 x

More Things to solve Need to consider distances between two points where each points are located in different partition? Calculate the closest pair in the space near boundary between two partition.

Example y 8 ? 11 13 6 3 5 14 2 ? 7 9 12 1 10 4 x

Example y 8 11 13 6 3 5 14 2 7 9 12 1 10 4 x    

Locate closest pair from the strip Sort points in the stip. Do not need to sort for every recursive call. It is enough to sort points at the main. Find minimum distance from point pairs in the strip O(n) is enough to calculate

Closest_Pair(P) merge_sort(P) return RecursiveClosestSort(P) RecursiveClosestSort(P) if(|P| <= 3) compare three points to get minimum distanced pair. return the minimum distance among all pairs Split P into P_L and P_R with boundary l such that each partition has almost equal number of points. d_L = Closest_Pair(P_L) d_R = Closest_Pair(P_R) delta = min{distance(d_L), distance(d_R)} d = points pair whose distance is delta among for each point p1 where l-delta < p1.x < l+delta for each point p2 in ascending order where p2.y > p1.y if(distance(p1,p2) > delta) break; delta = min(delta, distance(p1,p2)) return delta