Lecture 2: Divide and Conquer

Basic Algorithm Design Techniques Divide and conquer Dynamic Programming Greedy Common Theme: To solve a large, complicated problem, break it into many smaller sub-problems.

Divide and Conquer Idea: Break the problem into several unrelated sub- problems, then combine the solutions to solve the original problem. Recipe: 1. (divide) Divide the problem into sub-problems 2. (conquer) Solve the sub-problems recursively. 3. (merge) Combine the solutions.

Example 1 Merge Sort Goal: Sort an array of numbers in ascending order. Input: Array a[] = {6, 2, 4, 1, 5, 3, 7, 8} Algorithm: MergeSort(a[]) IF Length(a) < 2 THEN RETURN a. (Base Case) Partition a[] evenly into two arrays b[], c[]. (Divide) b[] = MergeSort(b[]) c[] = MergeSort(c[]) (Conquer) RETURN Merge(b[], c[]). (Merge)

Merge Sort: Example: {6, 2, 4, 1, 5, 3, 7, 8} {1, 2, 3, 4, 5, 6, 7, 8} {6, 2, 4, 1} {1, 2, 4, 6} {5, 3, 7, 8} {3, 5, 7, 8} {6, 2} {2, 6} {4, 1} {1, 4} {5, 3} {3, 5} {7, 8}

Key Design Step: How to Merge {1, 2, 4, 6} {3, 5, 7, 8} Idea: Smallest element must be between the first elements of the two arrays. Determine the smallest, remove it and continue. b[] = {1, 2, 4, 6} c[] = {3, 5, 7, 8} {1, 2, 3, 4, 5, 6, 7, 8}

Merge Step: Pseudocode b[] = {1, 2, 4, 6} c[] = {3, 5, 7, 8} {1, 2, 3, 4, 5, 6, 7, 8} Merge(b[], c[]) a[] = empty i = 1, j = 1 WHILE i <= Length(b[]) OR j <= Length(c[]) IF b[i] < c[j] THEN a.append(b[i]); i = i+1 ELSE a.append(c[j]); j = j+1 RETURN a[] Careful when you actually write a program (i,j can be out of bound)

Example 2 Counting Inversions Goal: Given array a[], count the number of pairs (i,j) such that i < j but a[i] > a[j]. Input: Array a[] = {6, 2, 4, 1, 5, 3, 7, 8} Answer = 9

Designing the algorithm Recall Recipe: Recipe: 1. (divide) Divide the problem into sub-problems 2. (conquer) Solve the sub-problems recursively. 3. (merge) Combine the solutions.

Failed Attempt: Partition a[] evenly to two arrays b[], c[]. Count Inversions recursively. Combine results a[] = {6, 2, 4, 1, 5, 3, 7, 8}, 9 inversions b[] = {6, 2, 4, 1}, 5 inversions c[] = {5, 3, 7, 8}, 1 inversion #inversion = #inversion in b + #inversion in c + #inversion between b[], c[] Can take O(n2) time!

How to count inversions more efficiently? b[] = {6, 2, 4, 1}, 5 inversions c[] = {5, 3, 7, 8}, 1 inversion For each number in b, want to know how many numbers in c are smaller. 6: 2, 2: 0, 4: 1, 1: 0, result = 2+0+1+0 = 3. Much easier if c[] is sorted! Idea: Can sort the arrays when we are counting inversions.

Sort&Count Sort&Count(a[]) IF Length(a) < 2 THEN RETURN (a[], 0). (Base Case) Partition a[] evenly into two arrays b[], c[]. (Divide) (b[], count_b) = Sort&Count(b[]) (c[], count_c) = Sort&Count(c[]) (Conquer) (a[], count_bc) = Merge&Count(b[], c[]). (Merge) RETURN a[], count_b+count_c+count_bc.

Merge&Count b[] = {1, 2, 4, 6} c[] = {3, 5, 7, 8} {1, 2, 3, 4, 5, 6, 7, 8} When 4 goes into the array, pointer in c[] is pointing at 5 (2nd element), so 4 is larger than 1 element in c[]. When 6 goes into the array, pointer in c[] is pointing at 7 (3rd element), so 6 is larger than 2 elements in c[].

Merge&Count Merge&Count(b[], c[]) a[] = empty i = 1, j = 1 count = 0 WHILE i <= Length(b[]) OR j <= Length(c[]) IF b[i] < c[j] THEN a.append(b[i]); i = i+1 count = count + (j - 1) ELSE a.append(c[j]); j = j+1 RETURN a[], count The i-th element of a[] is smaller than c[j], but larger than c[1,2,…, j-1]