1. Foundations of Algorithms, Fourth Edition
Richard Neapolitan, Kumarss Naimipour
Chapter 2
Divide-and-Conquer

2. Divide and Conquer
In this approach a problem is divided into sub-problems and the same algorithm is applied to every subproblem ( often this is done recursively)
Examples
Binary Search (review algorithm in book)
Mergesort (review algorithm in book)
Quicksort

3. Figure 2.1 : The steps down by a human when searching with Binary Search. (Note: x = 18)

4. Complexity of Binary Search
Since this and many other divide and conquer algorithms are recursive you will recall that we can determine their complexity using recurrence relations. For Binary Search we have T(n) = T(n/2) + 1 =[T(n/4)+1]+1 = T(n/22) + 2 =[T(n/8+1]+ 2 = T(n/23)+ 3 … =T(n/2k)+k

5. What is T(n/2k)+k
We if we let k get larger until n=2k then we see that k = log2n. Why? Consequently the relation becomes T(n) = T(1) + log2n T(n) = log2n Since n/2k is 1 if they are equal and T(1) =1

6. MergeSort
Recall in this algorithm we divide the array into two equal parts and sort each half prior to merging. The recurrence relation is clearly T(n) = 2T(n/2) + n Recall that Merging is O(n) right?

7. Figure 2.2: The steps done by a human when sorting with Mergesort.
T(n/4)
T(n/2)
O(n)
Figure 2.2: The steps done by a human when sorting with Mergesort.

8. T(n) = 2T(n/2) + n
T(n) = 2T(n/2) + n = 2[ 2T(n/22) + n/2] + n = 22T(n/22) + 2n = 2[2T(n/23) + n/22] +2n =23T(n/23) + 3n … =2kT(n/2k) + kn If n=2k then we have T(n) = nT(1) + (log2n)n = n+ nlog2n = O(nlog2n)

9. QuickSort Works in situ!
Void quicksort(int low, int high) { int pivot; if (high > low){ partition(low, high, pivot); quicksort(low, pivot-1); quicksort(pivot+1,high); }

10. Figure 2. 3: The steps done by a human when sorting with Quicksort
Figure 2.3: The steps done by a human when sorting with Quicksort. The subarrays are enclosed in rectangles whereas the pivot points are free.

11. Partition Study this carefully
void partition (int low, int high, int&pivot) { int I,j, pivotitem; pivotitem = S[low]; // select left item (hmmm) j=low; for (i=low+1; i<=high; i++) There are many ways if (S[i] < pivotitem){ to write this function! j++; All have a complexity swap S[i] and S[j]; of O(n). } pivot= j; swap S[low] and S[pivot];

12. Complexity of Quicksort
The complexity of this algorithm depends on how good the pivot value selection is . If the value is always in the middle of array then the best case complexity is T(n) = n + 2T(n/2) Which we already have determined is T(n) = n log2 n

13. Worst case for Quicksort
This clearly will occur if each pivot value is less than (or greater) all the elements of the array. IE the array is split into 1 and n-1 size pieces. This gives a recurrence relation of T(n) = T(1) + T(n-1) + n-1 Time to sort left array right array partition

14. Worst Case analysis
T(n) = T(1) + T(n-1) + n-1 = T(n-1) + n Assume the answer is n(n-1)/2 check it out ! n(n-1)/2 = 0 + (n-1)(n-2)/2 + n-1 = (n-1)(n-2)/2 + 2(n-1)/2 =((n-1)(n-2)+ 2(n-1))/2 = (n-1)(n-2+2)/2 = n(n-1)/2 ☺

15. Quick Sort Analysis Quicksort’s worst case is θ(n2)
Does this mean that quick sort is just as bad as say selection sort, insertion sort and/or bubble sort. No!
Its all about average case performance. The average case performance for these three is θ(n2) as well.
What is the average case complexity for QS?

16. Average Case Analysis assume prob. pivotpoint is p
𝐴 𝑛 = 𝑝=1 𝑛 1 𝑛 𝐴 𝑝−1 +𝐴(𝑛−𝑝) +𝑛−1 𝐴 𝑛 = 1 𝑛 𝑝=1 𝑛 𝐴 𝑝−1 +𝐴(𝑛−𝑝) +𝑛−1 𝐴 𝑛 = 2 𝑛 𝑝=1 𝑛 𝐴 𝑝−1 +𝑛−1 See HW 22 p 86 for above conversion

17. Average case continued
Multiplying by n 𝑛𝐴 𝑛 =2 𝑝=1 𝑛 𝐴 𝑝−1 +𝑛 𝑛−1 (𝑛−1)𝐴 𝑛−1 =2 𝑝=1 𝑛−1 𝐴 𝑝−1 +(𝑛−1)(𝑛−2) Subtracting these equations we have 𝑛𝐴 𝑛 − 𝑛−1 𝐴 𝑛−1 =2𝐴 𝑛−1 +2(𝑛−1) 𝐴 𝑛 𝑛+1 = 𝐴 𝑛−1 𝑛 + 2 𝑛−1 𝑛 𝑛+1

18. Average case QS continued
Assume 𝑎 𝑛 = 𝐴 𝑛 𝑛+1 we get 𝑎 𝑛 = 𝑎 𝑛−1 + 2 𝑛−1 𝑛 𝑛+1 , 𝑎 0 =0 Applying some simple math we have 𝑎 𝑛 ≈2 ln 𝑛 Which give 𝐴(𝑛)≈ 𝑛+1 lg 𝑛= 𝑛+1 2(( ln 2)( lg 𝑛 ) ≈1.38 𝑛+1 lg 𝑛 ∈𝜃(𝑛 lg 𝑛)

19. Matrix Multiplication (Strassen)
Lets look at the product of two 2 by 2’s
𝑐 11 𝑐 12 𝑐 21 𝑐 22 = 𝑎 11 𝑎 12 𝑎 21 𝑎 22 × 𝑏 11 𝑏 12 𝑏 21 𝑏 22
Clearly after you do the homework #26
m1=(a11+a22)(b11+b22) m5=(a11+a12)b22
m2=(a11+a22)b m6=(a21+a11)(b11+b12)
m3=a11(b12+b22) m7=(a12+a22)(b21+b22)
m4=a22(b21+b11) will give the following!

20. And the answer is Original method 8 mult, four add/sub
𝐶= 𝑚 1 + 𝑚 4 − 𝑚 5 + 𝑚 7 𝑚 3 + 𝑚 5 𝑚 2 + 𝑚 4 𝑚 1 + 𝑚 3 − 𝑚 2 + 𝑚 6
Original method
8 mult, four add/sub
Strassen’s method
7 mult, and 18 add/sub
Hmmmm! So what’s the big deal?

21. Big Matrices 2n by 2n
𝐶 11 𝑐 12 𝐶 21 𝑐 22 = 𝐴 11 𝐴 12 𝐴 21 𝐴 22 × 𝐵 11 𝐵 12 𝐵 21 𝐵 22 Where C11 is the upper left hand corner of the matrix of size n/2 by n/2. The others are similarly defined. Now m1=(A11+A22)(B11+B22) Is the sum and product of matrices

22. Our function is then
void Strassen(int n, A,B, C)// these are nxn mats { if (n<= threshold) computer C=AxB normally Partition A and B into eight submatrices strassen(n/2, A11+A22, B11+B22, M1); strassen(n/2, A21+A22, B11, M2) etc // making 7 recursive calls } NOT EIGHT!

23. Complexity T(n) = 7T(n/2) + cn2 T(n) = θ(nlg7) = O(n2.81) Which is
Using the general theorem.
The best know is
Coppersmith and Winograd with a time complexity of
O(n2.376)
Why am I using big O here?

24. Recalling General Theorem See page 588
Assume 𝑇 𝑛 =𝑎𝑇 𝑛 𝑏 +𝑐 𝑛 𝑘 for n>1 and n a power of b, T(1)=d 𝑇 𝑛 ∈ 𝜃 𝑛 𝑘 𝑖𝑓 𝑎< 𝑏 𝑘 𝜃( 𝑛 𝑘 lg 𝑛) 𝑖𝑓 𝑎= 𝑏 𝑘 𝜃 𝑛 𝑙𝑜𝑔 𝑏 𝑎 𝑖𝑓 𝑎> 𝑏 𝑘

25. Just a side note
Suppose we has 8 recursive calls instead of 7 in the above case. Then the recurrence relation would be T(n) = 8T(n/2) + cn2 This has a complexity of what?

26. When not to use divide and conquer
An instance of size n is divided into two or more instances each almost of size n.
An instance of size n is divided into almost n instances of size n/c, where c is a constant