1. Algorithm Design Techniques: Divide and Conquer

2. Introduction Divide and conquer – algorithm makes at least two recursive calls Subproblems should not be overlapping –Example: Fibonacci numbers Examples –Recursive maximum subsequence sum –Linear time tree traversal –Mergesort, quicksort

3. Running Time Mergesort –T(N) = 2T(N/2) + O(N) Theorem: The solution to the equation T(N) = aT(N/b) + θ(N k ) where a>=1 and b>1, is T(N) = O(N log b a ) a > b k = O(N k logN) a = b k = O(N k ) a < b k

4. Running Time: Proof Let – N=b m N/b=b m-1 N k =(b m ) k =b mk =b km =(b k ) m T(1) = 1 T(b m ) = aT(b m-1 )+(b k ) m - divide by a m and use telescoping m T(b m )/a m = Σ(b k /a) i i=0 multiply by a m

5. Running Time: Proof m T(b m ) = a m Σ(b k /a) i i=0 a > b k – series will converge to a constant –T(N) = O(a m ) = O(a log b N ) = O(N log b a ) a = b k – each term in sum is 1, sum contains 1+log b N terms, log b a = k –T(N) = O(a m log b N) = O(N log b a log b N) = O(N k log b N) = O(N k logN) a < b k – each term will be greater than 1 –T(N) = a m (((b k /a) m+1 -1)/((b k /a)-1)) = O(a m (b k /a) m ) = O((b k ) m ) = O(N k )

6. Running Time Theorem: If        then the solution to the equation T(N) = T(  1 N) + T(  2 N) +... + T(  k N) +O(N) is T(N) = O(N) Idea: problem is made smaller at every step

7. Closest-Points Problem p 1 = (x 1, y 1 ) p 2 = (x 2, y 2 ) Euclidean distance between p 1 and p 2 is [(x 1 -x 2 ) 2 + (y 1 -y 2 ) 2 ] 1/2 Goal: Given N points, find closest pair Brute force algorithm?

8. Closest-Points Example

9. Closest-Points Problem Brute force algorithm? –O(N 2 ) Divide and Conquer –similar to maximum subsequence sum sort by X coordinate find closest two in left half (d L ) find closest two in right half (d R ) find closest two where 1 is in left, 1 in right (d C ) closest is minimum

10. Closest-Points Example dLdL dRdR dCdC

11. Closest-Points Problem find closest two where 1 is in left, 1 in right (d C ) To ensure O(NlogN) solution, this step must be done in O(N) time –  min(d L, d R ) –Only need to look at points  from center –Still cannot do brute force on these points – what would be running time? –Observe: for each point, only need to consider points that are less than  away in y coordinate –There are a maximum of 7 points to consider Points must be separated by at least  Don't need to look at points that have already been processed

12. Example: Worst Case

13. Algorithm: findClosest T(N) = NlogN + NlogN + 2T(N/2) + 7N (inner loop runs max 7 times) preprocessing step 1: sort by x coordinate preprocessing step 1: sort by y coordinate findClosest(left) findClosest(right) for(i=0; i<numPointsInStrip; i++) for(j=i+1; j<numPointsInStrip;j++) if(y coordinates of p i and p j differ by more than  break; else if(dist(p i, p j ) <   = dist(p i, p j )

14. Selection Problem Find kth smallest element in a collection of N elements Obvious algorithm? –Sort, find kth – running time? Can be solved in O(N)

15. Selection Problem Quickselect –partition (as in quicksort) into S 1 and S 2 if k < size of S 1 – quickselect on S 1 else if k == size of S 1 + 1 – found else quickselect on S 2 Running time could still be N 2 unless pivot is guaranteed to be chosen wisely

16. Selection Problem Choosing the pivot –Arrange N elements into floor(N/5) groups of 5 elements –Find median of each group. This gives a list M of floor(N/5) medians. Sort each set (8 operations) –Find median of M. Choose as pivot Call quickselect(M/2) on M elements

17. Example L H H L T T v T T SL T T SL T T S T T S Medians S – small L – large H – huge T – tiny H H H H H H H H

18. Pivot Selection 10k+5 = N 2k+1 medians 1 pivot, k L's, and K S's 2k+2 H's, and 2k+2 T's 3k+2 larger than pivot 3k+2 smaller than pivot (10k+5)-(3k+2)-1(pivot) = max 7k+2 in partition <.7N

19. Running Time: Quickselect partition into N/5 arrays of size 5 and sort find median of medians (quickselect) – use as pivot partition if pivot is kth element – return else quickselect on appropriate partition T(N) = 8(N/5) + T(.2N) + N + T(.7N) = O(N)

20. Multiplying Integers x = 61,438,521 y = 94,736,407 Typical algorithm is N 2 Use divide and conquer! X L =6,143 X R =8,521 Y L =9,473 Y R =6,407 X = X L 10 4 +X R Y = Y L 10 4 +Y R XY = X L Y L 10 8 + (X L Y R + X R Y L )10 4 + X R Y R T(N) = 4T(N/2) + O(N) = O(N 2 )

21. Multiplying Integers XY = X L Y L 10 8 + (X L Y R + X R Y L )10 4 + X R Y R X L Y R + X R Y L = (X L – X R )(Y R -Y L )+X L Y L +X R Y R Use 1 multiplication plus 2 results already computed T(N) = 3T(N/2) + O(N) T(N) = O(N log 2 3 ) = O(N 1.59 )

22. Matrix Multiplication A = 3 4 1 6 B = 5 6 9 3 1 2 5 7 4 5 3 1 5 1 2 9 1 1 8 4 4 3 5 6 3 1 4 1 C i, j = dot product of ith row of A and jth col of B C 0, 0 = (3*5)+(4*4)+(1*1)+(6*3) Brute force algorithm = O(N 3 )

23. Matrix Multiplication A 1, 1 A 1, 2 B 1, 1 B 1, 2 =C 1, 1 C 1, 2 A 2, 1 A 2, 2 B 2, 1 B 2, 2 C 2, 1 C 2, 2 C 1, 1 = A 1, 1 B 1, 1 + A 1, 2 B 2, 1 C 1, 2 = A 1, 1 B 1, 2 + A 1, 2 B 2, 2 C 2, 1 = A 2, 1 B 1, 1 + A 2, 2 B 2, 1 C 2, 2 = A 2, 1 B 1, 2 + A 2, 2 B 2, 2

24. Matrix Multiplication T(N) = 8T(N/2) + O(N 2 ) –additions take N 2 Still O(N 3 ) M 1 = (A 1, 2 - A 2, 2 ) (B 2, 1 + B 2, 2 ) M 2 = (A 1, 1 +A 2, 2 ) (B 1, 1 + B 2, 2 ) M 3 = (A 1, 1 - A 2, 1 ) (B 1, 1 + B 1, 2 ) M 4 = (A 1, 1 + A 1, 2 ) (B 2,2 ) M 5 = (A 1, 1 ) (B 1,2 -B 2,2 ) M 6 = (A 2, 2 ) (B 2,1 -B 1,1 ) M 7 = (A 2, 1 + A 2, 2 ) (B 1,1 )

25. Matrix Multiplication C 1, 1 = M 1 + M 2 – M 4 + M 6 C 1, 2 = M 4 + M 5 C 2, 1 = M 6 + M 7 C 2, 2 = M 2 - M 3 + M 5 - M 7 T(N) = 7T(N/2) + O(N 2 ) = O(N log 2 7 ) = O(N 2.81 )