Divide-and-Conquer Recursive in structure –Divide the problem into several smaller sub-problems that are similar to the original but smaller in size –Conquer the sub-problems by solving them recursively. If they are small enough, just solve them in a straightforward manner. –Combine the solutions to create a solution to the original problem

An Example: Merge Sort Divide: Divide the n -element sequence to be sorted into two subsequences of n/2 elements each Conquer: Sort the two subsequences recursively using merge sort. Combine: Merge the two sorted subsequences to produce the sorted answer.

Merge-Sort (A, p, r) INPUT: a sequence of n numbers stored in array A OUTPUT: an ordered sequence of n numbers 1. if p < r 2. then q  [(p+r)/2] 3. Merge-Sort (A, p, q) 4. Merge-Sort (A, q+1, r) 5. Merge (A, p, q, r)

Analysis of Merge Sort Divide: computing the middle takes  (1) Conquer: solving 2 sub-problem takes 2T(n/2) Combine: merging n -element takes  (n) Total: T(n) =  (1) if n = 1 T(n) = 2T(n/2) +  (n) if n > 1  T(n) =  (n lg n) (CLRS/Chapter 4)

Recurrence Relations Recurrences (Chapter 4) –Substitution Method –Iteration Method –Master Method Arising from Divide and Conquer (e.g. MERGE-SORT) T(n) =  (1) if n  c T(n) = a T(n/b) + D(n) + C(n) otherwise

Substitution Method Guessing the form of the solutions, then using mathematical induction to find the constants and show the solution works. It works well when it is easy to guess. But, there is no general way to guess the correct solution.

An Example Solve: T(n) = 3T(  n/3  ) + n T(n)  3c  n/3  lg  n/3  + n  c n lg (n/3) + n = c n lg n - c n lg3 + n = c n lg n - n (c lg 3 - 1)  c n lg n * The last step is true for c  1 / lg3.

Making a Good Guess Guessing a similar solution to the one that you have seen before –T(n) = 3T(  n/3  + 5 ) + n similar to T(n) = 3T(  n/3  ) + n when n is large, the difference between n/3 and ( n/3 + 5 ) is insignificant Another way is to prove loose upper and lower bounds on recurrence and then reduce the range of uncertainty. –Start with T(n) =  (n) & T(n) = O(n 2 )  T(n) =  (n log n)

Subtleties When the math doesn’t quite work out in the induction, try to adjust your guess with a lower-order term. For example: –We guess T(n)  O(n) for T(n) = 3T(  n/3  )+ 4, but we have T(n)  3c  n/3  + 4 = c n + 4 –New guess is T(n)  c n - b, where b  0 T(n)  3(c  n/3  - b)+4 = c n - 3b + 4 = c n - b - (2b-4) Therefore, T(n)  c n - b, if 2b - 4  0 or if b  2

Changing Variables Use algebraic manipulation to turn an unknown recurrence similar to what you have seen before. –Consider T(n) = 2T(  n 1/2  ) + lg n –Rename m = lg n and we have T(2 m ) = 2T(2 m/2 ) + m –Set S(m) = T(2 m ) and we have S(m) = 2S(m/2) + m  S(m) = O(m lg m) –Changing back from S(m) to T(n), we have T(n) = T(2 m ) = S(m) = O(m lg m) = O(lg n lg lg n)

Avoiding Pitfalls Be careful not to misuse asymptotic notation. For example: –We can falsely prove T(n) = O(n) by guessing T(n)  c n for T(n) = 2T(  n/2  ) + n T(n)  2c  n/2  + n  c n + n = O(n)  Wrong! –The err is that we haven’t proved T(n)  c n

Exercises Solution of T(n) = T(  n/2  ) + 1 is O(lg n) Solution of T(n) = 2T(  n/2  + 17) + n is O(n lg n) Solve T(n) = 2T(n 1/2 ) + 1 by making a change of variables. Don’t worry whether values are integral.

Iteration Method Expand (iterate) the recurrence and express it as a summation of terms dependent only on n and the initial conditions The key is to focus on 2 parameters –the number of times the recurrence needs to be iterated to reach the boundary condition –the sum of terms arising from each level of the iteration process Techniques for evaluating summations can then be used to provide bounds on solution.

An Example Solve: T(n) = 3T(n/4) + n T(n) = n + 3T(n/4) = n + 3[ n/4 + 3T(n/16) ] = n + 3[n/4] + 9T(n/16) = n + 3[n/4] + 9 [n/16] + 27T(n/64) T(n)  n + 3n/4 + 9n/ n/64 + … + 3 log 4 n  (1)  n  (3/4) i +  (n log 4 3 ) = 4n+ o(n) = O(n)

Recursion Trees Keep track of the time spent on the subproblems of a divide and conquer algorithm A convenient way to visualize what happens when a recursion is iterated Help organize the algebraic bookkeeping necessary to solve the recurrence

Merge Sort n n/2 n/4 n/2 n/4 Running times to merge two sublists Running time to sort the left sublist

Running Time n n/2 n/4 n/2 n/4 n=n 2 ¢ (n/2) = n 4 ¢ (n/4) = n lg n Total: n lg n

Recursion Trees and Recurrences Useful even when a specific algorithm is not specified –For T(n) = 2T(n/2) + n 2, we have

Recursion Trees T(n) =  (n 2 )

Recursion Trees For T(n) = T(n/3) + T(2n/3) + n T(n) = O(n lg n)

Master Method Provides a “cookbook” method for solving recurrences of the form T(n) = a T(n/b) + f(n) Assumptions: – a  1 and b  1 are constants – f(n) is an asymptotically positive function – T(n) is defined for nonnegative integers –We interpret n/b to mean either  n/b  or  n/b 

The Master Theorem With the recurrence T(n) = a T(n/b) + f(n) as in the previous slide, T(n) can be bounded asymptotically as follows: 1. If f(n)=O(n log b a-  ) for some constant  > 0, then T(n)=  (n log b a ). 2. If f(n) =  (n log b a ), then T(n) =  (n log b a lg n). 3. If f(n) =  ( n log b a+  ) for some constant  > 0, and if a f(n/b)  c f(n) for some constant c < 1 and all sufficiently large n, then T(n)=  (f(n)).

Simplified Master Theorem Let a  1 and b > 1 be constants and let T(n) be the recurrence T(n) = a T(n/b) + c n k defined for n  If a > b k, then T(n) =  ( n log b a ). 2. If a = b k, then T(n) =  ( n k lg n ). 3. If a < b k, then T(n) =  ( n k ).

Examples T(n) = 16T(n/4) + n – a = 16, b = 4, thus n log b a = n log 4 16 =  (n 2 ) – f(n) = n = O(n log  ) where  = 1  case 1. –Therefore, T(n) =  (n log b a ) =  (n 2 ) T(n) = T(3n/7) + 1 – a = 1, b=7/3, and n log b a = n log 7/3 1 = n 0 = 1 – f(n) = 1 =  (n log b a )  case 2. –Therefore, T(n) =  (n log b a lg n) =  (lg n)

Examples (Cont.) T(n) = 3T(n/4) + n lg n – a = 3, b=4, thus n log b a = n log 4 3 = O(n ) – f(n) = n lg n =  (n log  ) where   0.2  case 3. –Therefore, T(n) =  (f(n)) =  (n lg n) T(n) = 2T(n/2) + n lg n – a = 2, b=2, f(n) = n lg n, and n log b a = n log 2 2 = n – f(n) is asymptotically larger than n log b a, but not polynomially larger. The ratio lg n is asymptotically less than n  for any positive . Thus, the Master Theorem doesn’t apply here.

Exercises Use the Master Method to solve the following:  T(n) = 4T(n/2) + n  T(n) = 4T(n/2) + n 2  T(n) = 4T(n/2) + n 3