Divide and Conquer Applications Sanghyun Park Fall 2002 CSE, POSTECH

Merge Sort Sort the first half of the array using merge sort. Sort the second half of the array using merge sort. Merge the first half of the array with the second half.

Merge Algorithm Merge is an operation that combines two sorted arrays. Assume the result is to be placed in a separate array called result (already allocated). The two given arrays are called front and back. front and back are in increasing order. For the complexity analysis, the size of the input, n, is the sum n front + n back.

Merge Algorithm For each array keep track of the current position. REPEAT until all the elements of one of the given arrays have been copied into result: – Compare the current elements of front and back. – Copy the smaller into the current position of result (break the ties however you like). – Increment the current position of result and the array that was copied from. Copy all the remaining elements of the other given array into result.

Merge Algorithm - Complexity Every element in front and back is copied exactly once. Each copy is two accesses, so the total number of accessing due to copying is 2n. The number of comparisons could be as small as min(n front, n back ) or as large as n-1. Each comparison is two accesses.

Merge Algorithm - Complexity In the worst case the total number of accesses is 2n +2(n-1) = O(n). In the best case the total number of accesses is 2n + 2min(n front,n back ) = O(n). The average case is between the worst and best case and is therefore also O(n).

Merge Sort Algorithm Split anArray into two non-empty parts anyway you like. For example, front = the first n/2 elements in anArray back = the remaining elements in anArray Sort front and back by recursively calling MergeSort. Now you have two sorted arrays containing all the elements from the original array. Use merge to combine them, put the result in anArray.

MergeSort Call Graph (n=7) Each box represents one invocation of MergeSort. How many levels are there in general if the array is divided in half each time? 0~6 0~23~6 3~4 3~34~4 5~6 5~56~6 1~2 1~12~2 0~0

MergeSort Call Graph (general) Suppose n = 2 k. How many levels? How many boxes on level j? What values is in each box at level j? n n/2 n/

MergeSort: Complexity Analysis Each invocation of mergesort on p array positions does the following: – Copies all p positions once (#accesses = O(p)) – Calls merge (#accesses = O(p)) Observe that p is the same for all invocations at the same level, therefore total number of accesses at a given level j is O((#invocations at level j) * p j ).

MergeSort: Complexity Analysis The total number of accesses at level j is O((#invocations at level j) * p j ) = O(2 j-1 * n/2 j-1 ) = O(n) In other words, the total number of accesses at each level is O(n). The total number of accesses for the entire mergesort is the sum of the accesses for all the levels. (#levels) * O(n) = O(log n) * O(n) = O(n log n)

MergeSort: Complexity Analysis Best case:O(n log n) Worst case:O(n log n) Average case:O(n log n)

Quick Sort Quicksort can be seen as a variation of mergesort in which front and back are defined in a different way.

Quicksort Algorithm Partition anArray into two non-empty parts. – Pick any value in the array, pivot. – small = the elements in anArray < pivot – large = the elements in anArray > pivot – Place pivot in either part, so as to make sure neither part is empty. Sort small and large by recursively calling QuickSort. You could use merge to combine them, but because the elements in small are smaller than elements in large, simply concatenate small and large, and put the result into anArray.

Quicksort: Complexity Analysis Like mergesort, a single invocation of quicksort on an array of size p has complexity O(p): – p comparisons = 2*p accesses – 2*p moves (copying) = 4*p accesses Best case: every pivot chosen by quicksort partitions the array into equal-sized parts. In this case quicksort is the same big-O complexity as mergesort – O(n log n)

Quicksort: Complexity Analysis Worst case: the pivot chosen is the largest or smallest value in the array. Partition creates one part of size 1 (containing only the pivot), the other of size p-1. n 1n-1 n

Quicksort: Complexity Analysis Worst case: There are n-1 invocations of quicksort (not counting base cases) with arrays of size: p = n, n-1, n-2, …, 2 Since each of these does O(p), the total number of accesses is O(n) + O(n-1) + … + O(1) = O(n 2 ) Ironically the worst case occurs when the list is sorted (or near sorted)!

Quicksort: Complexity Analysis The average case must be between the best case O(n log n) and the worst case is O(n 2 ). Analysis yields a complex recurrence relation. The average case number of comparisons turns out to be approximately 1.386*n*log n – 2.846*n. Therefore the average case time complexity is O(n log n).

Quicksort: Complexity Analysis Best case O(n log n) Worst case O(n2) Average case O(n log n) Note that the quick sort is inferior to insertion sort and merge sort if the list is sorted, nearly sorted, or reverse sorted.

Closest Pair Of Points Given n points in 2D, find the pair that are closest.

Applications We plan to drill holes in a metal sheet. If the holes are too close, the sheet will tear during drilling. Verify that two holes are closer than a threshold distance (e.g., holes are at least 1 inch apart.)

Air Traffic Control 3D --- Locations of airplanes flying in the neighborhood of a busy airport are known. We want to be sure that no two planes get closer than a given threshold distance.

Simple Solution For each of the n(n-1)/2 pairs of points, determine the distance between the points in the pair. Determine the pair with the minimum distance. O(n 2 ) time

Divide and Conquer Solution When n is small, use simple solution. When n is large, – Divide the point set into two roughly equal parts A and B. – Determine the closest pair of points in A. – Determine the closest pair of points in B. – Determine the closest pair of points such that one point is in A and the other in B. – From the three closest pairs computed, select the one with least distance.

Example Divide so that points in A have x coordinate <= that of points in B.

Example Find the closest pair in A. Let d 1 be the distance between the points in this pair.

Example Find the closest pair in B. Let d 2 be the distance between the points in this pair.

Example Let d = min{d 1,d 2 }. Is there a pair with one point in A and the other in B, and their distance < d?

Example Candidates lie within d of the dividing line. Call these regions R A and R B, respectively.

Example Let q be a point in R A. q need to be paired only with those points in R B that are within d of q.y.

Example Points that are to be paired with q are in d x 2d rectangle of R B (comparing region of q). Points in this rectangle are at least d apart. (What is the maximum number of points in this rectangle?)

Example So the comparing region of q has at most 6 points. So number of pairs to check is <= 6 * |R A | = O(n).

Time Complexity Create a sorted list of points (by x-coordinate): O(n log n) time Create a sorted list of points (by y-coordinate): O(n log n) time Using these two lists, the required pairs of points from R A and R B can be constructed in O(n) time. Let n < 4 define a small instance.

Time Complexity Let t(n) be the time to find the closest pair excluding the time to create the two sorted lists. t(n) = c, n < 4, where c is a constant. when n >= 4, t(n) = t(ceil(n/2)) + t(floor(n/2)) + dn, where d is a constant. To solve the recurrence, assume n is a power of 2 and use repeated substitution. t(n) = O(n log n)

Closest Pair Of Points Given n points in 2D, find the pair that are closest. C++ routines are given on pp of the text. The complete program is also available at