1. Instructor Neelima Gupta ngupta@cs.du.ac.in

2. Table of Contents Divide and Conquer

3. Divide the problem into smaller sub-problems. Solve the sub-problems recursively Combine the solution of the smaller sub-problems to obtain the solution of the bigger problem. Size of subproblems must reduce There must be some initial conditions The two together ensures that the algorithm terminates

4. Familiar Egs of divide and conquer Quick sort Merge sort (John von Neumann, 1945)

5. Time If T(n) is the time to sort ‘n’ numbers :- T(n)=T(n/2)+T(n/2)+ Ө (n) =2T(n/2) + cn = Ө (nlog n) Where Ө (n )= time required to merge two sorted lists, each of size at most n/2

6. Multiplying 2 large integers Suppose for simplicity, numbers are of equal length Eg- suppose the 2 number are 2345 and 3854 O(n 2 ) time by the usual successive add algorithm. We can reduce this time using divide and conquer strategy.

7. Multiplying 2 large integers split the numbers into roughly 2 halves Here x 0 =45; x 1 =23; y 0 =54; y 1 =38;

8. Mathematically x= x 1.10 n/2 + x 0 (1) y= y 1.10 n/2 + y 0 (2) xy=x 1 y 1.10 n +(x 1 y 0 +y 1 x 0 ).10 n/2 +x 0 y 0 (3) In our eg n=4 x=23*10 2 + 45 y=38*10 2 + 54

9. x 1 y 1 =874;x 1 y 0 =1242;x 0 y 1 =1710;x 0 y 0 =2430; Using (1),(2),(3) answer came out to be = 874*10 4 +(2952)*10 2 +2430 adding these numbers take linear time.

10. Time Analysis Computing x 1 y 1,(x 1 y 0 +y 1 x 0 ),x 0 y 0 T(n)=4T(n/2)+cn = Ө (n 2 ) So, no improvement. We’ll use a smarter way to compute the middle term.

11. (x 0 +x 1 )(y 0 +y 1 )=x 1 y 1 +(x 1 y 0 +y 1 x 0 )+x 0 y 0 Thus, x 1 y 0 +y 1 x 0 = (x 0 +x 1 )(y 0 +y 1 ) - x 0 y 0 - x 1 y 1 Thus, only 3 multiplications suffice

12. So T(n)is reduced to T(n)=3T(n/2)+cn = Ө (n log 2 3 )< Ө (n 2 ) where 3T(n/2) is the time required to multiply x 1 y 1, (x 0 +x 1 )(y 0 +y 1 ), x 0 y 0

13. Complex Roots of Unity Consider x n = 1 It has n distinct complex roots as follows: ω j,n = e (2 π ij)/n, j = 1 to n – 1 where i = √-1 These numbers are called n th roots of unity

14. n th roots of unity For example, for n = 2 x 2 = 1 => x = +1, -1 e (2 π ij)/n, j = 0,1 j = 0 e (2 π ij)/n = e 0 = cos 0 + i sin 0 = 1 j = 1 e (2 π i)/2 = e π i = cos π + i sin π = -1 Thanks to : Megha and Mitul

15. These can be pictured as a set of equally spaced points lying on a unit circle as shown in figure for n = 8. Figure by Mitul If ω j,2n ( j = 0, 1 … 2n-1) are (2n) th roots of unity then clearly ω 2 j,2n ( j = 0, 1 … n-1) are n th roots of unity. For j = n … 2n -1, roots repeat. ω j,2n = e (2 π ij)/2n = e (π ij)/n ω j,2n 2 = (e (π ij)/n ) 2 = e (2 π ij)/n It is also visible from the figure below Figure by Mitul

16. Discrete Fourier Transform DFT of a polynomial with coefficient vector is the vector y = where Θ(n 2 ) time to compute DFT is trivial, each y i can be computed in Θ(n) time. FFT is a method to compute DFT in Θ(n log n) time, It makes use of special properties of complex roots of unity.

17. Fast Fourier Transform Instead of n, we will deal with 2n Break the polynomial in 2 parts : A even (x) = a 0 + a 2 x + … + a n-2 x (n-2)/2 A odd (x) = a 1 + a 3 x + a 5 x 2 + ….+ a n-1 x (n-2)/2 A(x) = A even (x 2 ) + x.A odd (x 2 ) Let p(x) = A even (x 2 ) = a 0 +a 2 x 2 + a 4 x 4 +….+ a n-2 x n-2 and q(x) = A odd (x 2 ) + a 1 + a 3 x 2 + a 5 x 4 +….+a n-1 x n-2 x.q(x) = a 1 x + a 3 x 3 + a 5 x 5 +…+a n-1 x n-1 Thanks to : Megha and Mitul

18. Fast Fourier Transform contd… Thus, y j = A(ω j,2n ) = A even (ω 2 j,2n ) + ω j,2n.A odd (ω 2 j,2n ) (2n) th root of unityn th root of unity A even and A odd are polynomials with n terms, thus they can be computed at n th roots of unity, recursively.

19. FFT contd.. Thus we arrive at the following recurrence to compute the DFT T(n) = 2T(n/2) + O(n) = n log n Thus given a polynomial, its DFT can be computed in O(n log n) time.

20. Vandermonde Matrix Computing DFT is equivalent to n is only a control variable and can be replaced by 2n Since DFT can be computed in O(n log n) time, this matrix-vector product can be computed in O(n log n) time. Vandermonde matrix

21. Computing DFT -1 Theorem:For j,k = 0 …2n -1, (j,k) entry of V -1 2n is ω -k j,2n /2n Thus, given y i ‘s, a i ‘s and hence the polynomial can be computed as follows : As before n can be replaced by 2n. This matrix-vector multiplication is similar to the previous one and hence can be computed in O(n log n) time.

22. Convolution Let A = and B = be 2 vectors C = A o B = ( Length = 2n -1) C k = ∑ i+j = k a i b j AIM : to obtain convolution in (nlog(n)) time Thanks to : Megha and Mitul

23. Applications: polynomial multiplication Suppose we have two polynomials A(x) = a 0 + a 1 x + ………+ a n-1 x n-1 B(x) = b 0 + b 1 x + ………+ b n-1 x n-1 Then, C(x) = A(x).B(x)

24. Algortihm for AoB Let A(x) and B(x) be two polynomials with coefficients from the vectors A and B respectively. 1. Compute A and B at (2n)th roots of unity i.e. compute A(ω j,2n ) and B(ω j,2n ), j = 0,1 …2n-1 using FFT. O(n log n) time. 2. Compute C(ω j,2n ) = A(ω j,2n ). B(ω j,2n ) …. O(n) time. 3. Reconstruct C(x) using FFT -1 ….O(n log n) time.

25. Submitted By: Jewel Pruthi(18) Juhi Jain(19)

26. Problem : Given a set of points p 1,p 2,--------p n, our aim is to find out the closest pair of points. Finding Closest Pair in one-dimension Complexity – O(nlogn) How? Thanks to Jewel and Juhi

27. Sort the points.(takes O(nlogn) time) Find distance between every pair of consecutive points. In n comparisons,we will find the distance between every pair of consecutive points. In another n comparisons,we will find minimum of the distances found. Thanks to Jewel and Juhi

28. Finding Closest Pair in two- dimension Proposed algorithm:Sort the points p 1,p 2,------p n say on increasing order of x-coordinates. where p i =(x i,y i ) Distance between 2 consecutive points d=sqrt((y 2 -y 1 ) 2 + (x 2 -x 1 ) 2 ) st x 1 ≤x 2 ≤x 3 ------≤x n Ques: Will this algorithm work? Thanks to Jewel and Juhi

29. No Consider p 1,p 2,p 3 st d(p 1 p 2 ) > d(p 2 p 3 ) after sorting p 1,p 2,p 3 on x-coordinates. Algorithm returns p 2 p 3 but we can see that p 1 p 3 are closer. Thanks to Jewel and Juhi p1p1 p2 p2 p3p3

30. Note : For minimum distance,the two points need not be consecutive on x/y-axis. Brute Force Approach Calculate distance between every possible pair of points. Time Complexity:- O(n 2 ) Can we improve on the time complexity? Thanks to Jewel and Juhi

31. Divide and Conquer Approach Arrange the points in increasing order of x-coordinates say P x & increasing order of y-coordinates say P y. Divide the set of n points into 2 halves Q and R (breaking on middle of P x ). Compute the closest pair in Q and in R recursively. Thanks to Jewel and Juhi Solve recursively Q R

32. Let (q 1,q 2 ) and (r 1,r 2 ) be the closest pairs obtained in Q and R respectively. Let δ=min { d(q 1,q 2 ), d(r 1,r 2 ) } Thanks to Jewel and Juhi Solve recursively Q R (q 1,q 2 )(r 1,r 2 )

33. Ques: Does Ǝ a pair of points say (s 1,s 2 ) such that d(s 1,s 2 ) < δ ? Soln: Let p* be the point with maximum x-coordinate in Q and let x* be its x-coordinate. Draw a line through p* described by the equation L : x=x* Thanks to Jewel and Juhi

34. Thanks to Jewel and Juhi p* δ δ L x* q(q x ) r(r x ) Q R This distance is x*-q x Now in the highlighted triangle we can see that the length of horizontal line (x*-q x ) < hypotenuse according to Pythagoras theorem.  x*-q x <hypotenuse<d(q,r)< δ Similarly we can prove that r x -x* < d(q,r) < δ

35. Consider square boxes each of side δ/2 in this vertical strip. Thanks to Jewel and Juhi δ/2 S qyqy ryry L Square of side δ/2

36. Claim : No box contains more than 1 point. Proof: If the 2 farthest points in the box are the 2 diagonal points of the square, then 1. Distance between the 2 points = √( (δ/2) 2 + (δ/2) 2 ) = (δ/√2) < δ 2. Both the points are within Q or both are within R This implies that we have two points within Q (/R) with distance < δ which is a contradiction to the definition of δ. Thanks to Jewel and Juhi

37. Claim : Between any pair of 2 points q and r in S with d(q,r) < δ, there can be no more than 12 points. Thanks to Jewel and Juhi No of points ≤ 12 ṕ 1 ṕ 2 ṕ k ṕ r

38. Proof: Let P y1 … P yt be the points of S in the increasing order of y co-ordinates. If there are more than 12 points between q and r in the above list then there will be at least 3 rows between the y co-ordinate of q and y co-ordinate of r. Then, the vertical distance between q and r is > 3 × (δ/2) > δ Thus, the actual distance which is > vertical distance (show through fig.) > δ ---Contradiction to the definition of points q and r. Thanks to Jewel and Juhi

39. For i = 1 to t For j = i+1 to i + 12 Compute d(P yi, P yj ) Compute the minimum of the 12t pairs above --- O(n) time.

40. THANKYOU