1. Lecture 2: Divide and Conquer

2. Basic Algorithm Design Techniques
Divide and conquer
Dynamic Programming
Greedy
Common Theme: To solve a large, complicated problem, break it into many smaller sub-problems.

3. Divide and Conquer
Idea: Break the problem into several unrelated sub- problems, then combine the solutions to solve the original problem.
Recipe:
1. (divide) Divide the problem into sub-problems
2. (conquer) Solve the sub-problems recursively.
3. (merge) Combine the solutions.

4. Example 1 Merge Sort
Goal: Sort an array of numbers in ascending order.
Input: Array a[] = {6, 2, 4, 1, 5, 3, 7, 8}
Algorithm:
MergeSort(a[])
IF Length(a) < 2 THEN RETURN a (Base Case)
Partition a[] evenly into two arrays b[], c[] (Divide)
b[] = MergeSort(b[])
c[] = MergeSort(c[]) (Conquer)
RETURN Merge(b[], c[]) (Merge)

5. Merge Sort: Example: {6, 2, 4, 1, 5, 3, 7, 8} {1, 2, 3, 4, 5, 6, 7, 8}
{6, 2, 4, 1}
{1, 2, 4, 6}
{5, 3, 7, 8}
{3, 5, 7, 8}
{6, 2} {2, 6}
{4, 1}
{1, 4}
{5, 3}
{3, 5}
{7, 8}

6. Key Design Step: How to Merge
{1, 2, 4, 6} {3, 5, 7, 8}
Idea: Smallest element must be between the first elements of the two arrays.
Determine the smallest, remove it and continue.
b[] = {1, 2, 4, 6} c[] = {3, 5, 7, 8}
{1, 2, 3, 4, 5, 6, 7, 8}

7. Merge Step: Pseudocode
b[] = {1, 2, 4, 6} c[] = {3, 5, 7, 8}
{1, 2, 3, 4, 5, 6, 7, 8}
Merge(b[], c[])
a[] = empty
i = 1, j = 1
WHILE i <= Length(b[]) OR j <= Length(c[])
IF b[i] < c[j] THEN
a.append(b[i]); i = i+1
ELSE
a.append(c[j]); j = j+1
RETURN a[]
Careful when you actually write a program (i,j can be out of bound)

8. Example 2 Counting Inversions
Goal: Given array a[], count the number of pairs (i,j) such that i < j but a[i] > a[j].
Input: Array a[] = {6, 2, 4, 1, 5, 3, 7, 8}
Answer = 9

9. Designing the algorithm
Recall Recipe:
Recipe:
1. (divide) Divide the problem into sub-problems
2. (conquer) Solve the sub-problems recursively.
3. (merge) Combine the solutions.

10. Failed Attempt: Partition a[] evenly to two arrays b[], c[].
Count Inversions recursively.
Combine results
a[] = {6, 2, 4, 1, 5, 3, 7, 8}, 9 inversions b[] = {6, 2, 4, 1}, 5 inversions c[] = {5, 3, 7, 8}, 1 inversion
#inversion = #inversion in b + #inversion in c #inversion between b[], c[]
Can take O(n2) time!

11. How to count inversions more efficiently?
b[] = {6, 2, 4, 1}, 5 inversions c[] = {5, 3, 7, 8}, 1 inversion
For each number in b, want to know how many numbers in c are smaller. 6: 2, 2: 0, 4: 1, 1: 0, result = = 3.
Much easier if c[] is sorted!
Idea: Can sort the arrays when we are counting inversions.

12. Sort&Count Sort&Count(a[])
IF Length(a) < 2 THEN RETURN (a[], 0) (Base Case)
Partition a[] evenly into two arrays b[], c[] (Divide)
(b[], count_b) = Sort&Count(b[])
(c[], count_c) = Sort&Count(c[]) (Conquer)
(a[], count_bc) = Merge&Count(b[], c[]) (Merge)
RETURN a[], count_b+count_c+count_bc.

13. Merge&Count b[] = {1, 2, 4, 6} c[] = {3, 5, 7, 8}
{1, 2, 3, 4, 5, 6, 7, 8}
When 4 goes into the array, pointer in c[] is pointing at 5 (2nd element), so 4 is larger than 1 element in c[].
When 6 goes into the array, pointer in c[] is pointing at 7 (3rd element), so 6 is larger than 2 elements in c[].

14. Merge&Count Merge&Count(b[], c[]) a[] = empty i = 1, j = 1 count = 0
WHILE i <= Length(b[]) OR j <= Length(c[])
IF b[i] < c[j] THEN
a.append(b[i]); i = i+1
count = count + (j - 1)
ELSE
a.append(c[j]); j = j+1
RETURN a[], count
The i-th element of a[] is smaller than c[j], but larger than c[1,2,…, j-1]