1. Lecture 4 Divide and Conquer for Nearest Neighbor Problem
Shang-Hua Teng

2. Merge-Sort(A,p,r) A procedure sorts the elements in the sub-array A[p
Merge-Sort(A,p,r) A procedure sorts the elements in the sub-array A[p..r] using divide and conquer
Merge-Sort(A,p,r)
if p >= r, do nothing
if p< r then
Merge-Sort(A,p,q)
Merge-Sort(A,q+1,r)
Merge(A,p,q,r)
Starting by calling Merge-Sort(A,1,n)

3. A = MergeArray(L,R) Assume L[1:s] and R[1:t] are two sorted arrays of elements: Merge-Array(L,R) forms a single sorted array A[1:s+t] of all elements in L and R.
A = MergeArray(L,R)
for k to s + t
do if
then
else

4. Complexity of MergeArray
At each iteration, we perform 1 comparison, 1 assignment (copy one element to A) and 2 increments (to k and i or j )
So number of operations per iteration is 4.
Thus, Merge-Array takes at most 4(s+t) time.
Linear in the size of the input.

5. Merge (A,p,q,r) Assume A[p. q] and A[q+1
Merge (A,p,q,r) Assume A[p..q] and A[q+1..r] are two sorted Merge(A,p,q,r) forms a single sorted array A[p..r].
Merge (A,p,q,r)

6. Merge-Sort(A,p,r) A procedure sorts the elements in the sub-array A[p
Merge-Sort(A,p,r) A procedure sorts the elements in the sub-array A[p..r] using divide and conquer
Merge-Sort(A,p,r)
if p >= r, do nothing
if p< r then
Merge-Sort(A,p,q)
Merge-Sort(A,q+1,r)
Merge(A,p,q,r)

7. Divide and Conquer
Divide the problem into a number of sub-problems (similar to the original problem but smaller);
Conquer the sub-problems by solving them recursively (if a sub-problem is small enough, just solve it in a straightforward manner.
Combine the solutions to the sub-problems into the solution for the original problem

8. Merge Sort
Divide the n-element sequence to be sorted into two subsequences of n/2 element each
Conquer: Sort the two subsequences recursively using merge sort
Combine: merge the two sorted subsequences to produce the sorted answer
Note: during the recursion, if the subsequence has only one element, then do nothing.

9. Algorithm Design Paradigm I
Solve smaller problems, and use solutions to the smaller problems to solve larger ones
Divide and Conquer
Correctness: mathematical induction

10. Running Time of Merge-Sort
Running time as a function of the input size, that is the number of elements in the array A.
The Divide-and-Conquer scheme yields a clean recurrences.
Assume T(n) be the running time of merge-sort for sorting an array of n elements.
For simplicity assume n is a power of 2, that is, there exists k such that n = 2k .

11. Recurrence of T(n)
T(1) = 1
for n > 1, we have
if n = 1
if n > 1

12. Solution of Recurrence of T(n)
T(n) = 4 nlog n + n = O(nlog n)
Picture Proof by Recursion Tree

13. Two Dimensional Divide and Conquer
Can we extend the divide and conquer idea to 2 dimensions?
We will consider a slightly simpler problem
(handout #33, Chapter 33.4)

14. Closest Pair Problems Input: Output:
A set of points P = {p1,…, pn} in two dimensions
Output:
The pair of points pi, pj that minimize the Euclidean distance between them.

15. Closest Pair Problem

16. Closest Pair Problem

17. Divide and Conquer O(n2) time algorithm is easy Assumptions:
No two points have the same x-coordinates
No two points have the same y-coordinates
How do we solve this problem in 1 dimensions?
Sort the number and walk from left to right to find minimum gap

18. Divide and Conquer
Divide and conquer has a chance to do better than O(n2).
Assume that we can find the median in O(n) time!!!
We can first sort the point by their x-coordinates

19. Closest Pair Problem

20. Divide and Conquer for the Closest Pair Problem
Divide by x-median

21. Divide L R
Divide by x-median

22. Conquer L R
Conquer: Recursively solve L and R

23. Combination I L R Takes the smaller one of d1 , d2 : d = min(d1 , d2 )

24. Combination II Is there a point in L and a point in R whose distance is smaller than d ?
Takes the smaller one of d1 , d2 : d = min(d1 , d2 )

25. Combination II If the answer is “no” then we are done!!!
If the answer is “yes” then the closest such pair forms the closest pair for the entire set
Why????
How do we determine this?

26. Combination II Is there a point in L and a point in R whose distance is smaller than d ?
Takes the smaller one of d1 , d2 : d = min(d1 , d2 )

27. Combination II Is there a point in L and a point in R whose distance is smaller than d ?
Need only to consider the narrow band
O(n) time

28. Combination II Is there a point in L and a point in R whose distance is smaller than d ?
Denote this set by S, assume Sy is sorted list of S by y-coordinate.

29. Combination II
There exists a point in L and a point in R whose distance is less than d if and only if there exist two points in S whose distance is less than d.
If S is the whole thing, did we gain any thing?
If s and t in S has the property that ||s-t|| < d, then s and t are within 30 position of each other in the sorted list Sy.

30. Combination II Is there a point in L and a point in R whose distance is smaller than d ?
There are at most one point in each box

31. Closest-Pair Closest-pair(P) Preprocessing: Divide Conquer Combination
Construct Px and Py as sorted-list by x- and y-coordinates
Divide
Construct L, Lx , Ly and R, Rx , Ry
Conquer
Let d1= Closest-Pair(L, Lx , Ly )
Let d2= Closest-Pair(R, Rx , Ry )
Combination
Let d = min(d1 , d2 )
Construct S and Sy
For each point in Sy, check each of its next 30 points down the list
If the distance is less than d , update the d as this smaller distance

32. Complexity Analysis Preprocessing takes O(n lg n) time
Divide takes O(n) time
Conquer takes 2 T(n/2) time
Combination takes O(n) time
So totally takes O(n lg n) time