5 - 1 § 5 The Divide-and-Conquer Strategy e.g. find the maximum of a set S of n numbers.

Slides:



Advertisements
Similar presentations
A simple example finding the maximum of a set S of n numbers.
Advertisements

The Divide-and-Conquer Strategy
Prune-and-Search Method
4 -1 Chapter 4 The Divide-and-Conquer Strategy Outlines 4-1 The 2-Dimensional Maxima Finding Problem 4-2 The Closest Pair Problem 4-3 The Convex.
© The McGraw-Hill Companies, Inc., Chapter 2 The Complexity of Algorithms and the Lower Bounds of Problems.
Divide-and-Conquer The most-well known algorithm design strategy:
Divide and Conquer Strategy
4 -1 Chapter 4 The Divide-and-Conquer Strategy A simple example finding the maximum of a set S of n numbers.
The Divide-and-Conquer Strategy
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 5 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.
Chapter 6 Divide and Conquer  Introduction  Binary Search  Mergesort  The Divide and Conquer Paradigm  Quicksort  Multiplication of Large Integers.
1 Divide-and-Conquer The most-well known algorithm design strategy: 1. Divide instance of problem into two or more smaller instances 2. Solve smaller instances.
Divide and Conquer The most well known algorithm design strategy: 1. Divide instance of problem into two or more smaller instances 2. Solve smaller instances.
Lectures on Recursive Algorithms1 COMP 523: Advanced Algorithmic Techniques Lecturer: Dariusz Kowalski.
CS4413 Divide-and-Conquer
The Divide-and-Conquer Strategy
5 -1 Chapter 5 The Divide-and-Conquer Strategy A simple example finding the maximum of a set S of n numbers.
Chapter 4: Divide and Conquer Master Theorem, Mergesort, Quicksort, Binary Search, Binary Trees The Design and Analysis of Algorithms.
Theory of Algorithms: Divide and Conquer
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Divide and Conquer.
DIVIDE AND CONQUER APPROACH. General Method Works on the approach of dividing a given problem into smaller sub problems (ideally of same size).  Divide.
Divide and Conquer. Recall Complexity Analysis – Comparison of algorithm – Big O Simplification From source code – Recursive.
Convex Hull Problem Presented By Erion Lin. Outline Convex Hull Problem Voronoi Diagram Fermat Point.
Chapter 4 Divide-and-Conquer Copyright © 2007 Pearson Addison-Wesley. All rights reserved.
Nattee Niparnan. Recall  Complexity Analysis  Comparison of Two Algos  Big O  Simplification  From source code  Recursive.
CS38 Introduction to Algorithms Lecture 7 April 22, 2014.
Lecture 8 Jianjun Hu Department of Computer Science and Engineering University of South Carolina CSCE350 Algorithms and Data Structure.
Algorithm Design Strategy Divide and Conquer. More examples of Divide and Conquer  Review of Divide & Conquer Concept  More examples  Finding closest.
The Selection Problem. The selection problem Input: A set S of n elements Output: The k-th smallest element of S The median problem: to find the -th smallest.
Chapter 4 Divide-and-Conquer Copyright © 2007 Pearson Addison-Wesley. All rights reserved.
2 -1 Chapter 2 The Complexity of Algorithms and the Lower Bounds of Problems.
Chapter 4: Divide and Conquer The Design and Analysis of Algorithms.
Prune-and-search Strategy
2 -1 Chapter 2 The Complexity of Algorithms and the Lower Bounds of Problems.
Design and Analysis of Algorithms - Chapter 41 Divide and Conquer The most well known algorithm design strategy: 1. Divide instance of problem into two.
The Complexity of Algorithms and the Lower Bounds of Problems
Divide and Conquer The most well known algorithm design strategy: 1. Divide instance of problem into two or more smaller instances 2. Solve smaller instances.
Design and Analysis of Algorithms - Chapter 41 Divide and Conquer The most well known algorithm design strategy: 1. Divide instance of problem into two.
DIVIDE & CONQUR ALGORITHMS Often written as first as a recursive algorithm Master’s Theorem: T(n) = aT(n/b) + cn i, for some constant integer i, and constants.
2 -1 Chapter 2 The Complexity of Algorithms and the Lower Bounds of Problems.
Theory of Algorithms: Divide and Conquer James Gain and Edwin Blake {jgain | Department of Computer Science University of Cape Town.
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 Chapter.
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 5 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.
Computer Science 101 Fast Searching and Sorting. Improving Efficiency We got a better best case by tweaking the selection sort and the bubble sort We.
5 -1 Chapter 5 The Divide-and-Conquer Strategy A simple example finding the maximum of a set S of n numbers.
© The McGraw-Hill Companies, Inc., Chapter 6 Prune-and-Search Strategy.
Divide and Conquer Andreas Klappenecker [based on slides by Prof. Welch]
The Selection Problem. 2 Median and Order Statistics In this section, we will study algorithms for finding the i th smallest element in a set of n elements.
1 Prune-and-Search Method 2012/10/30. A simple example: Binary search sorted sequence : (search 9) step 1  step 2  step 3  Binary search.
Divide-and-Conquer The most-well known algorithm design strategy: 1. Divide instance of problem into two or more smaller instances 2.Solve smaller instances.
Computer Science 101 Fast Algorithms. What Is Really Fast? n O(log 2 n) O(n) O(n 2 )O(2 n )
Divide and Conquer Andreas Klappenecker [based on slides by Prof. Welch]
Divide and Conquer Strategy
Young CS 331 D&A of Algo. Topic: Divide and Conquer1 Divide-and-Conquer General idea: Divide a problem into subprograms of the same kind; solve subprograms.
Concepts of Algorithms CSC-244 Unit 15 & 16 Divide-and-conquer Algorithms ( Binary Search and Merge Sort ) Shahid Iqbal Lone Computer College Qassim University.
Convex Hull 2012/10/23. Convex vs. Concave A polygon P is convex if for every pair of points x and y in P, the line xy is also in P; otherwise, it is.
Advanced Algorithms Analysis and Design
Chapter 4 Divide-and-Conquer
CSCE 411 Design and Analysis of Algorithms
Divide and Conquer Mergesort Quicksort Binary Search Selection
Unit-2 Divide and Conquer
The Divide-and-Conquer Strategy
Topic: Divide and Conquer
The Divide-and-Conquer Strategy
Divide and Conquer Algorithms Part I
Divide-and-Conquer The most-well known algorithm design strategy:
Topic: Divide and Conquer
Divide and Conquer Merge sort and quick sort Binary search
Presentation transcript:

5 - 1 § 5 The Divide-and-Conquer Strategy e.g. find the maximum of a set S of n numbers

5 - 2 time complexity: assume n = 2 k T(n) = 2T(n/2)+1 = 2(2T(n/4)+1)+1 = 4T(n/4)+2+1 : =2 k-1 T(2)+2 k-2 +… =2 k-1 +2 k-2 +… =2 k -1 = n-1

5 - 3 A general divide-and-conquer algorithm Step 1: If the problem size is small, solve this problem directly; otherwise, split the original problem into 2 sub-problems with equal sizes. Step 2: Recursively solve these 2 sub-problems by applying this algorithm. Step 3: Merge the solutions of the 2 sub-problems into a solution of the original problem.

5 - 4 time complexity: where S(n): time for splitting M(n): time for merging b: a constant c: a constant.

5 - 5 Binary search e.g  search 7: needs 3 comparisons time: O(log n) The binary search can be used only if the elements are sorted and stored in an array.

5 - 6 Algorithm Binary-Search Input: A sorted sequence of n elements stored in an array. Output: The position of x (to be searched). Step 1: If only one element remains in the array, solve it directly. Step 2: Compare x with the middle element of the array. Step 2.1: If x = middle element, then output it and stop. Step 2.2: If x < middle element, then recursively solve the problem with x and the left half array. Step 2.3: If x > middle element, then recursively solve the problem with x and the right half array.

5 - 7 Algorithm BinSearch(a, low, high, x) // a[]: sorted sequence in nondecreasing order // low, high: the bounds for searching in a [] // x: the element to be searched // If x = a[j], for some j, then return j else return – 1 if (low > high) then return – 1 // invalid range if (low = high) then // if small P if (x == a[i]) then return i else return -1 else // divide P into two smaller subproblems mid = (low + high) / 2 if (x == a[mid]) then return mid else if (x < a[mid]) then return BinSearch(a, low, mid-1, x) else return BinSearch(a, mid+1, high, x)

5 - 8 quicksort e.g. sort into nondecreasing order [ ] [ ] [ ] [ ] 26 [ ] [ ] 26 [ ] [ 1 5] 11 [19 15] 26 [ ] [ ] [ ] [48 37] 59 [61]

5 - 9 Algorithm Quicksort Input: A set S of n elements. Output: the sorted sequence of the inputs in nondecreasing order. Step 1: If |S|  2, solve it directly Step 2: (Partition step) Use a pivot to scan all elements in S. Put the smaller elements in S 1, and the larger elements in S 2. Step 3: Recursively solve S 1 and S 2.

time in the worst case: (n-1)+(n-2) = = O(n 2 ) n(n-1) 2 time in the best case: In each partition, the problem is always divided into two subproblems with almost equal size.... log 2 n n × 2 = n n 2 × 4 = n n 4

T(n): time required for sorting n elements T(n) ≦ cn+2T(n/2), for some constant c. ≦ cn+2(c . n/2 + 2T(n/4)) ≦ 2cn + 4T(n/4) ≦ cnlog 2 n + nT(1) = O(nlogn)...

merge sort two-way merge: [ ][ ] [ ] merge time complexity: O(m+n), m and n: lengths of the two sorted lists merge sort (nondecreasing order) [25][57][48][37][12][92][86][33] [25 57][37 48][12 92][33 86] [ ][ ] [ ] pass 1 pass 2 pass 3 log 2 n passes are required. time complexity: O(nlogn)

Algorithm Merge-Sort Input: A set S of n elements. Output: the sorted sequence of the inputs in nondecreasing order. Step 1: If |S|  2, solve it directly Step 2: Recursively apply this algorithm to solve the leaf half part and right half part of S, and the results are stored in S 1 and S 2, respectively. Step 3: Perform the two-way merge scheme on S 1 and S 2.

D maxima finding problem Def: A point (x 1, y 1 ) dominates (x 2, y 2 ) if x 1 > x 2 and y 1 > y 2. A point is called a maxima if no other point dominates it.

Straightforward method: compare every pair of points time complexity: O(n 2 ). The maximal of S L and S R

Algorithm Maximal-Points Input: A set S of n planar points. Output: The maximal points of S. Step 1: If |S|=1, return it as the maxima. Otherwise, find a line L perpendicular to the X-axis which separates S into two subsets S L and S R with equal size. Step 2: Recursively find the maximal points of S L and S R. Step 3: Find the largest y-value of S R, denoted as y max. Conduct a linear scan on the maximal points of S L and discard each one if its y-value is less than y max. Time complexity: T(n) = O(n log n)

The closest pair problem Given a set S of n points, find a pair of points which are closest together. 1-D version: solved by sorting time: O(n log n) 2-D version:

at most 6 points in rectangle A:

Algorithm Closest-Pair Input: A set S of n points in the plane. Output: The distance between two closest points. Step 1. Sort points in S according to their y-values and x-values. Step 2. If S contains only one point, return  as its distance. Step 3. Find a median line L perpendicular to the X-axis to divide S into two subsets, with equal sizes, S L and S R. Every point in S L lies to the left of L and every point in S R lies to the right of L. Step 4. Recursively apply Step 2 and Step 3 to solve the closest pair problems of S L and S R. Let d L (d R ) denote the distance between the closest pair in S L (S R ). Let d = min(d L, d R ).

Step 5. Project all points within the slab bounded by L-d and L+d onto the line L. For a point P in the half-slab bounded by L-d and L, Let its y-value by denoted as y P. For each such P, find all points in the half-slab bounded by L and L+d whose y-value fall within y P +d and y P -d. If the distance d between P and a point in the other half-slab is less than d, let d=d. The final value of d is the answer. time complexity: O(n log n) Step 1: O(n log n) Steps 2~5:  T(n) = O(n log n)

The convex hull problem concave polygon: convex polygon: The convex hull of a set of planar points is the smallest convex polygon containing all of the points.

the divide-and-conquer strategy to solve the problem: Step 1: Select an interior point p. Step 2: There are 3 sequences of points which have increasing polar angles with respect to p. (1) g, h, i, j, k (2) a, b, c, d (3) f, e Step 3: Merge these 3 sequences into 1 sequence: g, h, a, b, f, c, e, d, i, j, k.

Step 4: Apply Graham scan to examine the points one by one and eliminate the points which cause reflexive angles. e.g. points b and f need to be deleted. Final result:

Algorithm Convex Hull Input: A set S of planar points Output: A convex hull for S Step 1. If S contains no more than five points, use exhaustive searching to find the convex hull and return. Step 2. Find a median line perpendicular to the X-axis which divides S into S L and S R ; S L lies to the left of S R. Step 3. Recursively construct convex hulls for S L and S R. Denote these convex hulls by Hull(S L ) and Hull(S R ) respectively. Step 4. Find an interior point P of S L. Find the vertices v 1 and v 2 of Hull(S R ) which divide the vertices of Hull(S R ) into two sequences of vertices which have increasing polar angles with respect to P. Without loss of generality, let us assume that v 1 has greater y- value than v 2.

Then form three sequences as follows: a)Sequence 1: all of the convex hull vertices of Hull(S L ) in counterclockwise direction. b)Sequence 2: the convex hull vertices of Hull(S R ) from v 2 to v 1 in counter-clockwise direction. c)Sequence 3: the convex hull vertices of Hull(S R ) from v 2 to v 1 in clockwise direction. Merge these three sequences and conduct the Graham scan. Eliminate the points which are reflexive and the remaining points from the convex hull. time complexity: T(n) = 2T(n/2) + O(n) = O(n log n)

Strassen’s matrix multiplication Let A, B and C be n  n matrices C = AB C(i, j) = A(i, k)B(k, j)  The straightforward method to perform a matrix multiplication requires O(n 3 ) time.  The divide-and-conquer strategy:

Time complexity: (# of additions: n 2 )  T(n) = O(n 3 )  Strassen’s method: P = (A 11 + A 22 )(B 11 + B 22 ) Q = (A 21 + A 22 )B 11 R = A 11 (B 12 - B 22 ) S = A 22 (B 21 - B 11 ) T = (A 11 + A 12 )B 22 U = (A 21 - A 11 )(B 11 + B 12 ) V = (A 12 - A 22 )(B 21 + B 22 ) C 11 = P + S - T + V C 12 = R + T C 21 = Q + S C 22 = P + R - Q + U

multiplications and 18 additions or subtractions. time complexity:

The Fast Fourier Transform (FFT) Fourier transform: Inverse Fourier transform: Discrete Fourier transform (DFT): given a 0, a 1, …, a n-1, compute

Inverse DFT: DFT can be computed in O(n 2 ) time by a straightforward method. DFT can be solved by the divide-and-conquer strategy (FFT, Fast Fourier Transform) in O(nlogn) time.

The FFT method e.q. n=4, w=e i2 π /4, w 4 =1, w 2 =-1 b 0 =a 0 +a 1 +a 2 +a 3 b 1 =a 0 +a 1 w+a 2 w 2 +a 3 w 3 b 2 =a 0 +a 1 w 2 +a 2 w 4 +a 3 w 6 b 3 =a 0 +a 1 w 3 +a 2 w 6 +a 3 w 9 another form: b 0 =(a 0 +a 2 )+(a 1 +a 3 ) b 2 =(a 0 +a 2 w 4 )+(a 1 w 2 +a 3 w 6 ) =(a 0 +a 2 )-(a 1 +a 3 ) When we calculate b 0, we shall calculate (a 0 +a 2 ) and (a 1 +a 3 ). Later, b 2 van be easily calculated. Similarly, b 1 =(a 0 + a 2 w 2 )+(a 1 w+a 3 w 3 ) =(a 0 -a 2 )+w(a 1 -a 3 ) b 3 =(a 0 +a 2 w 6 )+(a 1 w 3 +a 3 w 9 ) =(a 0 -a 2 )-w(a 1 -a 3 ).

e.g. n=8, w=e i2 π /8, w 8 =1, w 4 =-1 b 0 =a 0 +a 1 +a 2 +a 3 +a 4 +a 5 +a 6 +a 7 b 1 =a 0 +a 1 w+a 2 w 2 +a 3 w 3 +a 4 w 4 +a 5 w 5 +a 6 w 6 +a 7 w 7 b 2 =a 0 +a 1 w 2 +a 2 w 4 +a 3 w 6 +a 4 w 8 +a 5 w 10 +a 6 w 12 +a 7 w 14 b 3 =a 0 +a 1 w 3 +a 2 w 6 +a 3 w 9 +a 4 w 12 +a 5 w 15 +a 6 w 18 +a 7 w 21 b 4 =a 0 +a 1 w 4 +a 2 w 8 +a 3 w 12 +a 4 w 16 +a 5 w 20 +a 6 w 24 +a 7 w 28 b 5 =a 0 +a 1 w 5 +a 2 w 10 +a 3 w 15 +a 4 w 20 +a 5 w 25 +a 6 w 30 +a 7 w 35 b 6 =a 0 +a 1 w 6 +a 2 w 12 +a 3 w 18 +a 4 w 24 +a 5 w 30 +a 6 w 36 +a 7 w 42 b 7 =a 0 +a 1 w 7 +a 2 w 14 +a 3 w 21 +a 4 w 28 +a 5 w 35 +a 6 w 42 +a 7 w 49  b 0 =(a 0 +a 2 +a 4 +a 6 )+(a 1 +a 3 +a 5 +a 7 ) b 1 =(a 0 +a 2 w 2 +a 4 w 4 +a 6 w 6 )+ w(a 1 +a 3 w 2 +a 5 w 4 +a 7 w 6 ) b 2 =(a 0 +a 2 w 4 +a 4 w 8 +a 6 w 12 )+ w 2 (a 1 +a 3 w 4 +a 5 w 8 +a 7 w 12 ) b 3 =(a 0 +a 2 w 6 +a 4 w 12 +a 6 w 18 )+ w 3 (a 1 +a 3 w 6 +a 5 w 12 +a 7 w 18 ) b 4 =(a 0 +a 2 +a 4 +a 6 )-(a 1 +a 3 +a 5 +a 7 ) b 5 =(a 0 +a 2 w 2 +a 4 w 4 +a 6 w 6 )-w(a 1 +a 3 w 2 +a 5 w 4 +a 7 w 6 ) b 6 =(a 0 +a 2 w 4 +a 4 w 8 +a 6 w 12 )-w 2 (a 1 +a 3 w 4 +a 5 w 8 +a 7 w 12 ) b 7 =(a 0 +a 2 w 6 +a 4 w 12 +a 6 w 18 )-w 3 (a 1 +a 3 w 6 +a 5 w 12 +a 7 w 18 )

rewrite as: b 0 =c 0 +d 0 b 4 =c 0 -d 0 =c 0 +w 4 d 0 b 1 =c 1 +wd 1 b 5 =c 1 -wd 1 =c 1 +w 5 d 1 b 2 =c 2 +w 2 d 2 b 6 =c 2 -w 2 d 2 =c 2 +w 6 d 2 b 3 =c 3 +w 3 d 3 b 7 =c 3 -w 3 d 3 =c 3 +w 7 d 3 c 0 =a 0 +a 2 +a 4 +a 6 c 1 =a 0 +a 2 w 2 +a 4 w 4 +a 6 w 6 c 2 =a 0 +a 2 w 4 +a 4 w 8 +a 6 w 12 c 3 =a 0 +a 2 w 6 +a 4 w 12 +a 6 w 18 Let x=w 2 =e i2 π /4 c 0 =a 0 +a 2 +a 4 +a 6 c 1 =a 0 +a 2 x+a 4 x 2 +a 6 x 3 c 2 =a 0 +a 2 x 2 +a 4 x 4 +a 6 x 6 c 3 =a 0 +a 2 x 3 +a 4 x 6 +a 6 x 9 Thus, {c 0,c 1,c 2,c 3 } is FFT of {a 0,a 2,a 4,a 6 }. Similarly, {d 0,d 1,d 2,d 3 } is FFT of {a 1,a 3,a 5,a 7 }.

In general, let w=e i2 π /n (assume n is even.) w n =1, w n/2 =-1 b j =a 0 +a 1 w j +a 2 w 2j +…+a n-1 w (n-1)j, ={a 0 +a 2 w 2j +a 4 w 4j +…+a n-2 w (n-2)j }+ w j {a 1 +a 3 w 2j +a 5 w 4j +…+a n-1 w (n-2)j } =c j +w j d j b j+n/2 =a 0 +a 1 w j+n/2 +a 2 w 2j+n +a 3 w 3j+3n/2 +… +a n-1 w (n-1)j+n(n-1)/2 =a 0 -a 1 w j +a 2 w 2j -a 3 w 3j +…+a n-2 w (n-2)j -a n-1 w (n-1)j =c j -w j d j =c j +w j+n/2 d j

Algorithm Fast Fourier Transform Input: a 0, a 1, …, a n-1, n = 2 k Output: b j, j=0, 1, 2, …, n-1 where Step 1. If n=2, compute b 0 = a 0 + a 1, b 1 = a 0 - a 1, and return. Step 2. Recursively find the Fourier transform of {a 0, a 2, a 4, …, a n-2 } and {a 1, a 3, a 5, …,a n-1 }, whose results are denoted as {c 0, c 1, c 2, …, c n/2-1 } and {d 0, d 1, d 2, …, d n/2- 1 }.

Step 3. Compute b j : b j = c j + w j d j for 0  j  n/2 - 1 b j+n/2 = c j - w j d j for 0  j  n/ time complexity: T(n) = 2T(n/2) + O(n) = O(n log n)

An application of the FFT  polynomial multiplication {b 0, b 1, …, b n-1 } is the DFT of {a 0, a 1, …, a n-1 }. {a 0, a 1, …, a n-1 } is the inverse DFT of {b 0, b 1, …, b n-1 }. Polynomial multiplication: The straightforward product requires O(n 2 ) time.

A fast polynomial multiplication: Step 1: Let N be the smallest integer that N=2 q and N  2n-1. Step 2: Compute FFT of {a 0, a 1, …, a n-1, 0, 0, …, 0} N Step 3: Compute FFT of {c 0, c 1, …, c n-1, 0, 0, …, 0} N Step 4: Compute f(w j ) g(w j ), j=0, 1, …, N-1, where w=e i2  /N

Step 5: h(w j )=f(w j )g(w j ) Compute inverse DFT of {h(w 0 ), h(w 1 ), …, h(w N-1 )}. The resulting sequence of numbers are the coefficients of h(x). details in Step5: Let h(x)=b 0 + b 1 x+ b 2 x 2 + …+ b N-1 x N-1 {b 0, b 1, b 2, …, b N-1 } is the inverse DFT of {h(w 0 ), h(w 1 ), …, h(w N-1 )}. {h(w 0 ), h(w 1 ), …, h(w N-1 )} is the DFT of {b 0, b 1, b 2, …, b N-1 }, and h(w j )=f(w j )g(w j ). time: O(NlogN)=O(nlogn), N<4n.