Lecture 2: Divide and Conquer algorithms Phan Thị Hà Dương Algorithms: Design and Analysis Summer School 2013 at VIASM: Random Structures and Algorithms Lecture 2: Divide and Conquer algorithms Phan Thị Hà Dương

Lecture 2: Divide and Conquer algorithms 0. Introduction 1. Mergesort 2. Quicksort 3. *Strassen’s algorithm for matrix multiplication

0. Introduction Divide: Divide the problem to subproblems. Conquer: Solve recursively subproblems. Combine: Use results of subproblems and combine them to obtain result of initial problem. Determine Threshold: for which problem, the algorithm return directly result without dividing to smaller problems.

1. Merge sort algorithm Divide: Divide the n-element sequence to be sorted into two subsequences of n/2 elements each. Conquer: Sort the two subsequences recursively using merge sort. Combine: Merge the two sorted subsequences to produce the sorted answer. MERGE-SORT(A, p, r) if p < r then q← ⌊(p+r)/2⌋ MERGE-SORT(A, p, q) MERGE-SORT(A, q + 1, r) MERGE(A, p, q, r)

Example

Analyzing Merge sort algorithms Divide: D(n) = Θ(1). Conquer: solve two subproblems, each of size n/2, which contributes 2T (n/2) to the running time. Combine: the MERGE procedure on an n-element subarray takes time Θ(n), so C(n) = Θ(n). T(n) = θ(1) if n = 1 2 T(n/2) + θ(n) if n >1 And then T(n) = O (n lg n)

2. Quick Sort algorithms Quick Sort is a sorting algorithm whose worst-case running time is Θ(n2). In spite of this slow worst-case running time, Quick Sort is often the best practical choice for sorting because it is remarkably efficient on the average: its expected running time is Θ(n lg n), and the constant factors hidden in the Θ(n lg n) notation are quite small. It also has the advantage of sorting in place, and it works well even in virtual memory environments. Randomized Quick Sort has an expected running time of 0(n lg n): really efficient.

Idea of Quick Sort Algorithm Divide: Partition (rearrange) the array A[p.. r] into two (possibly empty) subarrays A[p.. q - 1] and A[q + 1..r] such that each element of A[p.. q - 1] is less than or equal to A[q], which is smaller than each element of A[q + 1.. r]. Compute the index q as part of this partitioning procedure. Conquer: Sort the two subarrays A[p.. q -1] and A[q +1.. r] by recursive calls to quicksort. Combine: Since the subarrays are sorted in place, no work is needed to combine them: the entire array A[p.. r] is now sorted.

Quick Sort QUICKSORT(A, p, r) 1 if p < r 2 then q ← PARTITION(A, p, r) 3 QUICKSORT(A, p, q - 1) 4 QUICKSORT(A, q + 1, r)

Partition algorithm PARTITION(A, p, r) 1 x←A[r] 2 i←p-1 3 For j←p to r-1 4 do if A[j] ≤ x 5 then i ← i + 1 6 exchange A[i] ↔ A[j] 7 exchange A[i + 1] ↔ A[r] 8 return i+1 The running time of PARTITION on the subarray A[p.. r] is Θ(n), where n = r - p + 1

Analyzing Quicksort: worst-case partitioning T(n) = T(n-1) + T(0) + Θ(n) = T(n-1) + Θ(n) = Θ(n^2) Question: In each case we have the worst-case ?

Best-case partitioning T(n) ≤ 2T(n/2) + Θ(n) = Θ(n lg n)

Balanced partitioning T(n) ≤ T(9n/10) + T(n/10) + cn

3. Strassen’s algorithm for matrix multiplication Presents Strassen's remarkable recursive algorithm for multiplying n × n matrices, which runs in Θ(n^lg 7) = O(n2.81) time. For sufficiently large values of n, therefore, it outperforms the naive Θ(n3) matrix-multiplication algorithm MATRIX-MULTIPLY from

Idea of Strassen’s algorithm We wish to compute the product C = AB, where each of A, B, and C are n × n matrices. Assuming that n is an exact power of 2, we divide each of A, B, and C into four n/2 × n/2 matrices, rewriting the equation C = AB as follows r = ae + bg s = af + bh t = ce + dg u = cf + dh

Analyzing of Strassen’s algorithm Naïve Algorithm T(n) = 8 T(n/2) + Θ(n^2) Strassen’s algorithm: T(n) = 8 T(n/2) + Θ(n^2) = Θ(n^lg 7) = O(n ^2,81)

Home Works Implement on C (or C++, or Pascal, or Java) the algorithms of sorting: Insertion Sort, Heap Sort, Merge Sort and Quick Sort. Run your program on inputs of arrays of 8 elements, of 16 elements, of 12 elements. Implement the Strassen’s algorithm, and run your program on: an input of two matrix of size 4 an input of two matrix of size 8