1. The Divide-and-Conquer Strategy
Chapter 4
The Divide-and-Conquer Strategy

2. 4-0 The Divide-and-Conquer Strategy
Recall: Binary searching: (Top-Down splitting)
E.g. find the maximum of a set S of n numbers (Button-Up merging) => T(n)= 2T(n/2)+1

3. 4-0 The Divide-and-Conquer Strategy
Time complexity:
Assume n = 2k,
T(n) = 2T(n/2)+1
= 2(2T(n/4)+1)+1
= 4T(n/4)+2+1 :
=2k-1T(2)+2k-2+…+4+2+1
=2k-1+2k-2+…+4+2+1
=2k-1 = n-1

4. 4-0 A general divide-and-conquer algorithm
Method :
Divide : Partition the problem into “independent” subproblems recursively until the subproblem’s size is small enough.
Conquer : solve the independent “small” subproblems.
Combine : merge the subsolutions into a solution.

5. 4-0 A general divide-and-conquer algorithm
General Structure : (Recursive structure)
S(P) : solution of P.
small( P ) : decide P small or not .
Algorithm D_and_C( P )
if small(P) then return S(P) (Be careful !!!)
else
divide P into small subproblem P1, P2, …, Pk , k≧1.
D_ and_C( Pi ), for k ≧ i ≧ 1.
return Combine ( S( P1 ) , S( P2 ), …, S( Pk ) )
End D_and_C.

6. 4-0 A general divide-and-conquer algorithm
Time Complexity T(n) :
g(n) n  c
T(n) =
T(n1)+T(n2)+…+T(nk)+f(n) , otherwise
f(n) : time for partitioning and combining.
Usually:
T(1) n = 1
aT(n/b) +f(n) n > 1
Small problem : 1 element
Divide into “a” subproblems with size “n/b”.

7. 4-0 A general divide-and-conquer algorithm
Time complexity:
where S(n) : time for splitting/partitioning
M(n) : time for merging/Combining
b : a constant
c : a constant
E.g. quick sort
E.g. merge sort

8. 4-0 A general divide-and-conquer algorithm
Substitution Method to solve T(n) :
Example:
T(1) n = 1
T(n) =
2T(n/2) +n n > 1

9. 4-0 Techniques for Solving Recurrence (Conti.)
Theorem (General substitution method)
Assume that a,b,c  0 and k is a constant.
b for n=1
T(n) =
a T( n/c ) + b * nk for n >1
O (nk ) if a < ck
Then, T (n) = O ( nk log n ) if a = ck
O ( n logc a ) if a > ck

10. General substitution method

11. General substitution method (cont.)

12. General substitution method (cont.)
Note: Checking logca is > = < k
O (nk ) if a < ck
T (n) = O ( nk logc n ) if a = ck (see equation (1))
O ( n logc a ) if a > ck

13. 4-0 Techniques for Solving Recurrence (Conti.)
Example:
T1(n) = 4 T1(n/2)+3n and T1(1) = c
T1 = O( n2)
T2(n) = 2 T2(n/4) + 3n and T2 (1) = c
T2 = O( n)
Hanoi Tower problem
T(n) = 2 T(n-1) + 1 and T2 (1) = 1
T(n) = 2n-1=O( 2n)

14. 4-0 Techniques for Solving Recurrence (Conti.)
Transformation :
n =1
Let

15. 4-0 Techniques for Solving Recurrence (Conti.)
Transformation :
n =2

16. 4-0 Techniques for Solving Recurrence (Conti.)
Transformation :

17. 4-0 Techniques for Solving Recurrence (Conti.)
Exercise 4.6 find T(n)

18. 4-0 Techniques for Solving Recurrence (Conti.)

19. 4-0 Techniques for Solving Recurrence (Conti.)
Recursion Tree :
Example:
T(n)= 1 n=1
2T(n/2)+ n2 n>1
Cost n2 to divide T(n) into 2 subproblems with size n/2
T(n)
T(n/2) T(n/2)
T(n/4) T(n/4) T(n/4) T(n/4)

20. 4-0 Techniques for Solving Recurrence (Conti.)
Recursion Tree (Conti.): n2
(n/2) (n/2)2
(n/4)2 (n/4)2 (n/4)2 (n/4)2 n2
1/2 n2
1/4 n2
…………………………………….
…………

21. 4-0 Techniques for Solving Recurrence (Conti.)
Recursion Tree (Conti.):

22. 4-1 2-D maxima finding problem
Def : A point (x1, y1) dominates (x2, y2) if x1 > x2 and y1 > y2. A point is called a maxima if no other point dominates it.
Straightforward method : Compare every pair of points
Time complexity: O(n2)
Applications:
Thinking (Image….)

23. 4-1 2-D maxima finding problem
Divide-and-conquer method:
The maximal points of SL and SR

24. 4-1 Divide-and-conquer approach
Input : A set of n planar points.
Output : The maximal points of S.
Step 1 If S contains only one point, return it as the maxima. Otherwise, find a line L perpendicular to the X-axis which separates the set of points into two subsets SL and SR , each of which consisting of n/2 points.

25. 4-1 Divide-and-conquer approach
Step 2 Recursively find the maximal points of SL and SR .
Step 3 Project the maximal points of SL and SR onto L and sort these points according to their y-values. Conduct a linear scan on the projections and discard each of the maximal points of SL if its y-value is less than the y-value of some maximal point of SR .
This step (O(nlogn)) can be modified

26. 4-1 Divide-and-conquer approach
Time complexity:
Assume n = 2k (Try it )
T(n) = O(n log n) + O(n log2n)
= O(n log2n)
(Note: log(n/2)=logn-log2)

27. 4-1 Divide-and-conquer approach
Improvement:
(1) Step 3: Find the largest y-value of SR.
Time complexity:
T(n) = O(n log n)

28. 4-1 Divide-and-conquer approach
(2) The sorting of y-values need be done only once (only one presorting).
No sorting is needed in Step3.
Time complexity:
O(n log n) + T(n) = O(n log n)
where

29. 4-2 The closest pair problem
Given a set S of n points, find a pair of points which are closest together.
1-D version :
Stupid method: compute distances of all pairs, and select the smallest pair.
Time complexity : O(n2 )
Solved by sorting, then compute the distance between two sequential points.
Time complexity : O(n log n)
2-D version: This section presents this version.
3-D version: Thinking

30. 4-2 The closest pair problem
Be careful !
Why?: at most 6 points in A:
2-D version:

31. 4-2 Divide-and-conquer approach
Input : A set S of n points in the plane.
Output : The distance between two closest points.
Step 1 Sort points in S according to their y-values and x-values.
Step 2 If S contains only two points, return their distance.

32. 4-2 Divide-and-conquer approach
Step 3 Find a median line L perpendicular to the X-axis to divide S into two subsets, with equal sizes, SL and SR. Every point in SL lies to the left of L and every point in SR lies to the right of L.
Step 4 Recursively apply Step 2 and Step 3 to solve the closest pair problems of SL and SR. Let dL(dR) denote the distance between the closest pair in SL (SR). Let d = min(dL, dR).

33. 4-2 Divide-and-conquer approach
Step 5 Project all points within the slab bounded by L-d and L+d onto the line L. For a point P in the half-slab bounded by L-d and L. Let its y-value by denoted as yP . For each such P, find all points in the half-slab bounded by L and L+d whose y-value fall within yP +d and yP -d. If the distance d between P and a point in the other half-slab is less than d, let d=d . The final value of d is the answer. Therefore, the merge step requires only O(n) time.

34. 4-2 Divide-and-conquer approach
Time complexity: O(n log n)
Step 1: O(n log n): Sorting n points
Steps 2~5:
T(n) = O(n log n)

35. 4-3 The convex hull problem
concave polygon:
convex polygon:
The convex hull of a set of planar points is the smallest convex polygon containing all of the points.

36. 4-3 The convex hull problem
A polygon is defined to be convex if for any two points p1 and p2 inside the polygon, the directed line segment from p1 to p2 (<p1, p2>) is fully contained in the polygon.

37. 4-3 The convex hull problem
Stupid method-1: Maybe stupid or excellent
有個很簡單的演算法，就是對於每個點都去檢查是否在某三個點所形成的三角形裡面，若是的話，它就不會是 convex hull 上的點，不然就是 convex hull 上的點。
只是每個點都需要檢查 O(n3) 次，共有 n 個點，所 complexity 是 O(n4)。
Stupid method-2: O(n2)。
1. Find the lowest point (the base-point).
2. Compute all angles between two lines which includes
One line: this base-point and the previous point.
One of many lines: this base-point and other points.
3.Determine the next point (find the smallest value)
4. Let this point be the base-point and goto Step2.

38. 4-3 The convex hull problem
The divide-and-conquer strategy to solve the problem:

39. 4-3 The convex hull problem
Select an interior point p.
There are 3 sequences of points which have increasing polar angles with respect to p.
(1) g, h, i, j, k (points in SL , first finding lowest point)
(2) a, b, c, d (points in SR , finding a and b points)
(3) f, e (points in SR , first finding lowest point)
Merge these 3 sequences into 1 sequence: (Merge sort)
g, h, a, b, f, c, e, d, i, j, k. (Time complexity O(n))
Note: Step 2 and 3 may be replaced with the following:
Compute polar angles of all points in SL and SR
and sort them. => (Time complexity O(nlogn))

40. 4-3 The convex hull problem
Polar angles

41. 4-3 The convex hull problem
Apply Graham scan to examine the points one by one and eliminate the points which cause reflexive angles.
For example: points b and f need to be deleted.

42. 4-3 The convex hull problem
Final result:

43. 4-3 Divide-and-Conquer approach
Input : A set S of planar points
Output : A convex hull for S
Step 1 If S contains no more than five points, use exhaustive searching to find the convex hull and return.
Step 2 Find a median line perpendicular to the X-axis which divides S into SL and SR ; SL lies to the left of SR . (O(n))

44. 4-3 Divide-and-Conquer approach
Step 3 Recursively construct convex hulls for SL and SR. Denote these convex hulls by Hull(SL) and Hull(SR) respectively.
Step 4 Find an interior point P of SL. Find the vertices v1 and v2 of Hull(SR) which divide the vertices of Hull(SR) into two sequences of vertices which have increasing polar angles with respect to P. Without loss of generality, let us assume that v1 has greater y-value than v2 . Then form three sequences as follows:

45. 4-3 Divide-and-Conquer approach
Sequence 1: all of the convex hull vertices of Hull(SL) in counterclockwise direction.
Sequence 2: the convex hull vertices of Hull(SR) from v2 to v1 in counter-clockwise direction.
Sequence 3: the convex hull vertices of Hull(SR) from v2 to v1 in clockwise direction.
Note: v1 has greater y-value than v2

46. 4-3 Divide-and-Conquer approach
Merge these three sequences and conduct the Graham scan. Eliminate the points which are reflexive and the remaining points from the convex hull.
Time complexity:
T(n) = 2T(n/2) + O(n)
= O(n log n)

47. 4-4 The Voronoi diagram problem (Option)
Def : Given two points Pi, Pj  S, let H(Pi,Pj) denote the half plane containing Pi. The Voronoi polygon associated with Pi is defined as Pi Pj
H(Pj ,Pi )
H(Pi , Pj)

48. 4-4 The Voronoi diagram problem
E.g. The Voronoi diagram for three points
V(P1)=H(p1,p2) ∩ H(p1,p3)
V(P2)=H(p2,p1) ∩ H(p2,p3)
V(P3)=H(p3,p1) ∩ H(p3,p2)
Each Lij is perpendicular bisector of the line

49. 4-4 The Voronoi diagram problem
E.g. A Voronoi diagram of 6 points:
The vertices of the Voronoi diagram are called Voronoi points and its segments are called Voronoi edges.
Applications: (Known Voronoi edges in advance)
Easy to know “Nearest neighbors” of points on a plane

50. 4-4 The Voronoi diagram problem
E.g. A Delaunay triangulation:
Each Lij is perpendicular bisector of the line
Remark:
Three perpendicular bisectors of a triangle must intersect on a point.

51. 4-4 Divide-and-Conquer approach
Input : A set S of n planar points.
Output : The Voronoi diagram of S.
Step 1 If S contains less than 4 points, solve directly and return.
Step 2 Find a median line L perpendicular to the X-axis which divides S into SL and SR such that SL (SR) lies to the left(right) of L and the sizes of SL and SR are equal.

52. 4-4 Divide-and-Conquer approach
Step 3 Construct Voronoi diagrams of SL and SR recursively. Denote these Voronoi diagrams by VD(SL) and VD(SR).
Step 4 Construct a dividing piece-wise linear hyperplane HP which is the locus of points simultaneously closest to a point in SL and a point in SR. Discard all segments of VD(SL) which lie to the right of HP and all segments of VD(SR) that lie to the left of HP. The resulting graph is the Voronoi diagram of S. ()
This merging step can be seen in Slide- 54

53. 4-4 Voronoi diagram after Step 2

54. 4-4 Piecewise linear hyperplane (Finding convex hulls of VD(SL) and VD(SR) ) ( Generating 5 edges)

55. 4-4 The Final Voronoi diagram

56. 4-4 How to merge two Voronoi diagrams ?

57. 4.4 How to merge two Voronoi diagrams ? (Good descriptions for steps)
Merging:

58. 4.4 Merges Two Voronoi Diagrams into One Voronoi Diagram
Input : (a) SL and SR where SL and SR are divided by a perpendicular line L.
(b) VD(SL ) and VD(SR ).
Output : VD(S) where S = SL ∩SR
Step 1 Find the convex hulls of SL and SR . Let them be denoted as Hull(SL) and Hull(SR), respectively. (A special algorithm for finding a convex hull in this case will by given later. O(n) )
Note that: a convex hull can be found by finding the infinite rays)

59. 4.4 Merges Two Voronoi Diagrams into One Voronoi Diagram
Step 2 Find segments and which join HULL(SL ) and HULL(SR ) into a convex hull (Pa and Pc belong to SL and Pb and Pd belong to SR) Assume that lies above Let x = a, y = b, SG= and HP =  .
Step 3 Find the perpendicular bisector of SG. Denote it by BS. Let HP = HP∪{BS}. If SG = , go to Step 5; otherwise, go to Step 4.

60. 4.4 Merges Two Voronoi Diagrams into One Voronoi Diagram
Step 4 The ray from VD(SL ) and VD(SR) which BS first intersects with must be a perpendicular bisector of either or for some z. If this ray is the perpendicular bisector of , then let SG = ; otherwise, let SG = Go to Step 3.
Step 5 Discard the edges of VD(SL) which extend to the right of HP and discard the edges of VD(SR) which extend to the left of HP. The resulting graph is the Voronoi diagram of S = SL∪SR.

61. 4-4 Lower bound
The lower bound of the Voronoi diagram problem is (n log n).
∵sorting  Voronoi diagram problem
The Voronoi Diagram for a Set of Points on a Straight Line

62. 4-4 Time complexity
Time complexity for merging 2 Voronoi diagrams: O(n)
Total  T(n) = 2T(n/2) + O(n) => O(n log n)

63. 4-5 Fast Fourier Transform (FFT) (Option)
Why is DFT so important ?
Any waveform can be decomposed into the linear sum of cosine functions (of various frequencies), square functions, triangular functions, and etc
Applications
Speech Processing
Image Processing

64. Simple example for Fourier Transform
Figure 1 presents a cosine function with freq. 3

65. Simple example for Fourier Transform
Figure 2 presents a signal, but it is not cosine or sine function.

66. Simple example for Fourier Transform
Figure 3: Frequency components (頻譜)
By Fourier Transform, Figure 2 is transformed into Figure 3
That is, Figure 2 is composed by two cosine functions as

67. Extra: Fast Fourier Transform (FFT) (Option)
wn = 1 (complex n-th root of unity) using eiu to interpreter it
eiu=cos(u)+i•sin(u) (Definition)
e2 ik/n , k=0,1,2,…n-1 , thus there is 8 roots, because of (e2 ik/n )n=1
i, π/2
1, 2π(0)
-1, π
-i , 3π/2

68. 4-5 Fast Fourier Transform (FFT) (Option)
Fourier transform => To compute A(f)
A(f) = a(t)e2iftdt
Discrete Fourier transform (DFT)
Given a0, a1, …, an-1 => To compute Aj
Aj = ake2ijk/n ,0  j  n-1
Note that: Given a signal f(t), get n samples as
a0 =f(0), a1=f(1), …, an-1 =f(n-1)

69. Discrete Fourier transform (DFT)
Note that: Given a signal f(t), get n samples as
a0 =f(0), a1=f(1), …, an-1 =f(n-1).
DFT problem:
That is, Given a0, a1, …, an-1
=> To compute Aj ,0  j  n-1

70. Discrete Fourier transform (DFT)
, for t = 0,1,2,…,n-1
f(t) is composed by n/2 cosine functions
i denotes frequency, i=0,1,2,…,n/2-1
Ai may be a complex number such as 2+3j
If |Ai| is large, then the cosine with frequency i
determine f(t) mainly.
If |Ai| is small, then the cosine with frequency i
determine f(t) trivially. , 。

71. Discrete Fourier transform (DFT)
1-sec music signal

72. Discrete Fourier transform (DFT)
Transformed Figure by DFT, Frequency components (頻譜)
真正重要的頻率集中在3000Hertz之前 , 我們幾乎可以說, 3000Hertz以後的訊號是不太重要的。
離散傅葉爾轉換有一個對稱性 , 只需要看8192hertz以前的頻率就能夠了 , 大於8192Hertz的頻率是多餘的。

73. 4-5 Fast Fourier Transform (FFT) (Option)
A straightforward method to calculate DFT requires O(n2) time
DFT can be solved by the divide-and-conquer strategy (FFT) => O(n log n)

74. 4-5 Fast Fourier Transform (FFT) (Option)
E.g. n =4
A0 = a0 + a1 + a2 + a3
A1 = a0 + a1e2i/4 + a2e2i(2)/4 + a3e2i(3)/4
= a0 + a1ei/2 + a2ei + a3e3i/2
A2 = a0 + a1e2i(2)/4 + a2e2i(2)(2)/4 + a3e2i(2)(3)/4
= a0 + a1ei + a2e2i + a3e3i
A3 = a0 + a1e2i(3)/4 + a2e2i(3)(2)/4 + a3e2i(3)(3)/4
= a0 + a1e3i /2 + a2e3i + a3e9i/2

75. 4-5 Fast Fourier Transform (FFT) (Option)
Rewrite as:
A0 = a0 + a1 + a2 + a3
A1 = a0 + a1ei/2 + a2ei + a3e3i/2
A2 = a0 + a1ei + a2e2i + a3e3i
A3 = a0 + a1e3i /2 + a2e3i + a3e9i/2
Another from: eiu=cos(u)+i•sin(u), i=
A0 = a0 + a2 + (a1 + a3)
A2 = a0 + a2e2i + (a1ei + a3e3i)
= a0 + a2 - (a1 + a3)
i, π/2
1, 2π(0)
-1, π
-i , 3π/2

76. 4-5 Fast Fourier Transform (FFT) (Option)
When we calculate A0 , we shall calculate (a0 + a2) and (a1 + a3). Later, A2 can be easily found.
Similarly,
A1 = (a0 + a2ei) + a1ei/2 + a3e3i/2
A3 = (a0 + a2e3i) + (a1e3i /2 + a3e9i/2)
= (a0 + a2ei) - (a1ei /2 + a3e3i/2)
E.g. n = 8, let e2i/8= w8 = ei/4
(w8 = 1, w4 = -1, w2 = i, w6 = -i)

77. 4-5 Fast Fourier Transform (FFT) (Option)
Rewrite as:
A0 = B0 + C0 A4 = B0 － C0
A1 = B1 + C1 A5 = B1 － C1
A2 = B2 + C2 A6 = B2 － C2
A3 = B3 + C3 A7 = B3 － C3
We can apply the same method to calculate Bi’s and Ci’s.
B0 = a0+a2+a4+a6
B2 = a0+a2w4+a4w8+a6w12
B0 = (a0+a4)+(a2+a6)
B2 = (a0+a4w8)+(a2w4+a6w12)
= (a0+a4)-(a2+a6)

78. 4-5 Fast Fourier Transform (FFT) (Option)
Similarly,
B1 = a0+a2w2+a4w4+a6w6
= (a0+a4w4) + (a2w2+a6w6)
B3 = a0+a2w6+a4w12+a6w18
= (a0+a4w12) + (a2w6+a6w18)
= (a0+a4w4) - (a2w2+a6w6)

79. 4-5 Fast Fourier Transform (FFT) (Option)
In general, let w = e2i/n (assume n is even)
(wn = 1, wn/2 = -1)
Aj = a0+a1wj+a2w2j+…+an-1w(n-1)j
={a0+a2w2j+a4w4j+…+an-2w(n-2)j} +
{a1wj+a3w3j+…+an-1w(n-1)j }
= Bj + Cj
Aj+n/2 = a0+a1wj+n/2+a2w2j+n+a3w3j+3n/2+…+
an-1w(n-1)j+(n(n-1)/2)
= a0-a1wj+a2w2j-a3w3j+…+an-2w(n-2)j-an-1w(n-1)j
= Bj - Cj

80. 4-5 Divide-and-Conquer approach (Option)
Input : a0 , a1 , …, an-1, n = 2k
Output : Aj, j=0, 1, 2, …, n-1, where Aj = 0kn-1ake2ijk/n
Step 1 If n=2, compute
A0 = a0 + a1,
A1 = a0 - a1,
and return.
Step 2 Divide each aj, 0  j  n/2 - 1 into two sequences:
Oj and Ej, where Oj(Ej), consists of odd-numbered
(even-numbered) terms of Aj.

81. 4-5 Divide-and-Conquer approach (Option)
Step 3 Recursively calculate the sums of terms in Oj and Ej . Denote the sum of terms of Oj and Ej by Bj and Cj, respectively.
Step 4 Compute Aj by the following formual :
Aj = Bj + Cj for 0  j  n/2 - 1
Aj+n/2 = Bj - Cj for 0  j  n/2 - 1.
Time complexity :
T(n) = 2T(n/2) + O(n)
= O(n log n)

82. 4-6 Strassen’s matrix multiplicaiton
Let A, B and C be n  n matrices
C = AB
C(i, j) = A(i, k)B(k, j)
The straightforward method to perform a matrix multiplication requires O(n3) time.

83. 4-6 Divide-and-Conquer approach
C = AB
C11 = A11 B11 + A12 B21
C12 = A11B12 + A12 B22
C21 = A21 B11 + A22 B21
C22 = A21 B12 + A22 B22
Time complexity:
(# of additions : n2)
T(n) = O(n3)

84. 4-6 Divide-and-Conquer approach
Strassen’s method:
P = (A11 + A22)(B11 + B22)
Q = (A21 + A22)B11
R = A11(B12 - B22)
S = A22(B21 - B11)
T = (A11 + A12)B22
U = (A21 - A11)(B11 + B12)
V = (A12 - A22)(B21 + B22).

85. 4-6 Divide-and-Conquer approach
C11 = P + S - T + V
C12 = R + T
C21 = Q + S
C22 = P + R - Q + U
Thus, 7 multiplications and 18 additions or subtractions
Time complexity:

86. 4-6 Time complexity  T(n) = an2 + 7T(n/2)
= an2 + 7(a(n/2)2 + 7T(n/4))
= an2 + (7/4)an2 + 72T(n/4)
= … :
= an2(1 + 7/4 + (7/4)2+…+(7/4)k-1)+7kT(1)
 cn2(7/4)log2n+7log2n, c is a constant
= cnlog24+log27-log24 +nlog27
= O(nlog27) (Recall Slide 4-9)
 O(n2.81)