Advanced Topics in Algorithms and Data Structures Page 1 Parallel merging through partitioning The partitioning strategy consists of: Breaking up the given.

Slides:



Advertisements
Similar presentations
Algorithms Algorithm: what is it ?. Algorithms Algorithm: what is it ? Some representative problems : - Interval Scheduling.
Advertisements

General algorithmic techniques: Balanced binary tree technique Doubling technique: List Ranking Problem Divide and concur Lecture 6.
Parallel List Ranking Advanced Algorithms & Data Structures Lecture Theme 17 Prof. Dr. Th. Ottmann Summer Semester 2006.
Back to Sorting – More efficient sorting algorithms.
PERMUTATION CIRCUITS Presented by Wooyoung Kim, 1/28/2009 CSc 8530 Parallel Algorithms, Spring 2009 Dr. Sushil K. Prasad.
Equality Join R X R.A=S.B S : : Relation R M PagesN Pages Relation S Pr records per page Ps records per page.
Linear-time Median Def: Median of elements A=a 1, a 2, …, a n is the (n/2)-th smallest element in A. How to find median? sort the elements, output the.
Optimal PRAM algorithms: Efficiency of concurrent writing “Computer science is no more about computers than astronomy is about telescopes.” Edsger Dijkstra.
Counting the bits Analysis of Algorithms Will it run on a larger problem? When will it fail?
Heaps1 Part-D2 Heaps Heaps2 Recall Priority Queue ADT (§ 7.1.3) A priority queue stores a collection of entries Each entry is a pair (key, value)
Advanced Topics in Algorithms and Data Structures Lecture 7.2, page 1 Merging two upper hulls Suppose, UH ( S 2 ) has s points given in an array according.
Lecture 3: Parallel Algorithm Design
A simple example finding the maximum of a set S of n numbers.
1 Parallel Parentheses Matching Plus Some Applications.
Partitioning and Divide-and-Conquer Strategies ITCS 4/5145 Parallel Computing, UNC-Charlotte, B. Wilkinson, Jan 23, 2013.
© The McGraw-Hill Companies, Inc., Chapter 2 The Complexity of Algorithms and the Lower Bounds of Problems.
The Divide-and-Conquer Strategy
CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms.
Divide and Conquer. Recall Complexity Analysis – Comparison of algorithm – Big O Simplification From source code – Recursive.
Advanced Topics in Algorithms and Data Structures Lecture 7.1, page 1 An overview of lecture 7 An optimal parallel algorithm for the 2D convex hull problem,
Nattee Niparnan. Recall  Complexity Analysis  Comparison of Two Algos  Big O  Simplification  From source code  Recursive.
Advanced Topics in Algorithms and Data Structures Lecture pg 1 Recursion.
Advanced Topics in Algorithms and Data Structures Classification of the PRAM model In the PRAM model, processors communicate by reading from and writing.
Advanced Topics in Algorithms and Data Structures Lecture 6.1 – pg 1 An overview of lecture 6 A parallel search algorithm A parallel merging algorithm.
1 Tuesday, November 14, 2006 “UNIX was never designed to keep people from doing stupid things, because that policy would also keep them from doing clever.
2 -1 Chapter 2 The Complexity of Algorithms and the Lower Bounds of Problems.
Divide-and-Conquer1 7 2  9 4   2  2 79  4   72  29  94  4.
Advanced Topics in Algorithms and Data Structures 1 Lecture 4 : Accelerated Cascading and Parallel List Ranking We will first discuss a technique called.
Prune-and-search Strategy
1 Lecture 11 Sorting Parallel Computing Fall 2008.
Accelerated Cascading Advanced Algorithms & Data Structures Lecture Theme 16 Prof. Dr. Th. Ottmann Summer Semester 2006.
Parallel Merging Advanced Algorithms & Data Structures Lecture Theme 15 Prof. Dr. Th. Ottmann Summer Semester 2006.
5 - 1 § 5 The Divide-and-Conquer Strategy e.g. find the maximum of a set S of n numbers.
Fall 2008Array Manipulation Algorithms1. Fall 2008Array Manipulation Algorithms2 Searching Let A = (a 1, a 2, …, a n ) be a sorted array of data such.
CS 104 Introduction to Computer Science and Graphics Problems Data Structure & Algorithms (3) Recurrence Relation 11/11 ~ 11/14/2008 Yang Song.
CHAPTER 11 Searching. 2 Introduction Searching is the process of finding a target element among a group of items (the search pool), or determining that.
Advanced Topics in Algorithms and Data Structures Page 1 An overview of lecture 3 A simple parallel algorithm for computing parallel prefix. A parallel.
The Complexity of Algorithms and the Lower Bounds of Problems
Advanced Topics in Algorithms and Data Structures 1 Two parallel list ranking algorithms An O (log n ) time and O ( n log n ) work list ranking algorithm.
Basic PRAM algorithms Problem 1. Min of n numbers Problem 2. Computing a position of the first one in the sequence of 0’s and 1’s.
Unit 1. Sorting and Divide and Conquer. Lecture 1 Introduction to Algorithm and Sorting.
Simulating a CRCW algorithm with an EREW algorithm Lecture 4 Efficient Parallel Algorithms COMP308.
Lecture 12: Parallel Sorting Shantanu Dutt ECE Dept. UIC.
Computer Science and Engineering Parallel and Distributed Processing CSE 8380 February 8, 2005 Session 8.
Reynolds 2006 Complexity1 Complexity Analysis Algorithm: –A sequence of computations that operates on some set of inputs and produces a result in a finite.
CHAPTER 09 Compiled by: Dr. Mohammad Omar Alhawarat Sorting & Searching.
© The McGraw-Hill Companies, Inc., Chapter 6 Prune-and-Search Strategy.
Complexity of algorithms Algorithms can be classified by the amount of time they need to complete compared to their input size. There is a wide variety:
1 Designing algorithms There are many ways to design an algorithm. Insertion sort uses an incremental approach: having sorted the sub-array A[1…j - 1],
File Organization and Processing Week 13 Divide and Conquer.
CSC 211 Data Structures Lecture 13
1 Prune-and-Search Method 2012/10/30. A simple example: Binary search sorted sequence : (search 9) step 1  step 2  step 3  Binary search.
By: Lokman Chan Recursive Algorithm Recursion Definition: A function that is define in terms of itself. Goal: Reduce the solution to.
1 Section 2.1 Algorithms. 2 Algorithm A finite set of precise instructions for performing a computation or for solving a problem.
Review 1 Selection Sort Selection Sort Algorithm Time Complexity Best case Average case Worst case Examples.
Chapter 15 A External Methods. © 2004 Pearson Addison-Wesley. All rights reserved 15 A-2 A Look At External Storage External storage –Exists beyond the.
1 Parallel Sorting Algorithm. 2 Bitonic Sequence A bitonic sequence is defined as a list with no more than one LOCAL MAXIMUM and no more than one LOCAL.
Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar
Introduction to Complexity Analysis. Computer Science, Silpakorn University 2 Complexity of Algorithm algorithm คือ ขั้นตอนการคำนวณ ที่ถูกนิยามไว้อย่างชัดเจนโดยจะ.
1 Ch.19 Divide and Conquer. 2 BIRD’S-EYE VIEW Divide and conquer algorithms Decompose a problem instance into several smaller independent instances May.
Young CS 331 D&A of Algo. Topic: Divide and Conquer1 Divide-and-Conquer General idea: Divide a problem into subprograms of the same kind; solve subprograms.
PREVIOUS SORTING ALGORITHMS  BUBBLE SORT –Time Complexity: O(n 2 ) For each item, make (n –1) comparisons Gives: Comparisons = (n –1) + (n – 2)
Unit-8 Sorting Algorithms Prepared By:-H.M.PATEL.
© 2006 Pearson Addison-Wesley. All rights reserved15 A-1 Chapter 15 External Methods.
Sorting – Lecture 3 More about Merge Sort, Quick Sort.
Divide and Conquer Algorithms Sathish Vadhiyar. Introduction  One of the important parallel algorithm models  The idea is to decompose the problem into.
Lecture 3: Parallel Algorithm Design
2.6 Applications Of Induction & other ideas Important Theorems
Merge Sort 4/10/ :25 AM Merge Sort 7 2   7  2   4  4 9
Divide and Conquer Merge sort and quick sort Binary search
Presentation transcript:

Advanced Topics in Algorithms and Data Structures Page 1 Parallel merging through partitioning The partitioning strategy consists of: Breaking up the given problem into many independent subproblems of equal size Solving the subproblems in parallel This is similar to the divide-and-conquer strategy in sequential computing.

Advanced Topics in Algorithms and Data Structures Page 2 Partitioning and Merging Given a set S with a relation , S is linearly ordered, if for every pair a,b  S. either a  b or b  a. The merging problem is the following:

Advanced Topics in Algorithms and Data Structures Page 3 Partitioning and Merging Input: Two sorted arrays A = ( a 1, a 2,..., a m ) and B = ( b 1, b 2,..., b n ) whose elements are drawn from a linearly ordered set. Output: A merged sorted sequence C = ( c 1, c 2,..., c m+n ).

Advanced Topics in Algorithms and Data Structures Page 4 Merging For example, if A = (2,8,11,13,17,20) and B = (3,6,10,15,16,73), the merged sequence C = (2,3,6,8,10,11,13,15,16,17,20,73).

Advanced Topics in Algorithms and Data Structures Page 5 Merging A sequential algorithm Simultaneously move two pointers along the two arrays Write the items in sorted order in another array

Advanced Topics in Algorithms and Data Structures Page 6 Partitioning and Merging The complexity of the sequential algorithm is O ( m + n ). We will use the partitioning strategy for solving this problem in parallel.

Advanced Topics in Algorithms and Data Structures Page 7 Partitioning and Merging Definitions: rank ( a i : A ) is the number of elements in A less than or equal to a i  A. rank ( b i : A ) is the number of elements in A less than or equal to b i  B.

Advanced Topics in Algorithms and Data Structures Page 8 Merging For example, consider the arrays: A = (2,8,11,13,17,20) B = (3,6,10,15,16,73) rank ( 11 : A ) = 3 and rank ( 11 : B ) = 3.

Advanced Topics in Algorithms and Data Structures Page 9 Merging The position of an element a i  A in the sorted array C is: rank ( a i : A ) + rank ( a i : B ). For example, the position of 11 in the sorted array C is: rank ( 11 : A ) + rank ( 11 : B ) = = 6.

Advanced Topics in Algorithms and Data Structures Page 10 Parallel Merging The idea is to decompose the overall merging problem into many smaller merging problems. When the problem size is sufficiently small, we will use the sequential algorithm.

Advanced Topics in Algorithms and Data Structures Page 11 Merging The main task is to generate smaller merging problems such that: Each sequence in such a smaller problem has O (log m ) or O (log n ) elements. Then we can use the sequential algorithm since the time complexity will be O (log m + log n ).

Advanced Topics in Algorithms and Data Structures Page 12 Parallel Merging Step 1. Divide the array B into blocks such that each block has log m elements. Hence there are m /log m blocks. For each block, the last elements are i log m, 1  i  m /log m

Advanced Topics in Algorithms and Data Structures Page 13 Parallel Merging Step 2. We allocate one processor for each last element in B. For a last element i log m, this processor does a binary search in the array A to determine two elements a k, a k+1 such that a k  i log m  a k+1. All the m /log m binary searches are done in parallel and take O (log m ) time each.

Advanced Topics in Algorithms and Data Structures Page 14 Parallel Merging After the binary searches are over, the array A is divided into m /log m blocks. There is a one-to-one correspondence between the blocks in A and B. We call a pair of such blocks as matching blocks.

Advanced Topics in Algorithms and Data Structures Page 15 Parallel Merging Each block in A is determined in the following way. Consider the two elements i log m and ( i + 1 ) log m. These are the elements in the ( i + 1 )-th block of B. The two elements that determine rank ( i log m : A ) and rank (( i + 1 ) log m : A ) define the matching block in A

Advanced Topics in Algorithms and Data Structures Page 16 Parallel Merging These two matching blocks determine a smaller merging problem. Every element inside a matching block has to be ranked inside the other matching block. Hence, the problem of merging a pair of matching blocks is an independent subproblem which does not affect any other block.

Advanced Topics in Algorithms and Data Structures Page 17 Parallel Merging If the size of each block in A is O (log m ), we can directly run the sequential algorithm on every pair of matching blocks from A and B. Some blocks in A may be larger than O (log m ) and hence we have to do some more work to break them into smaller blocks.

Advanced Topics in Algorithms and Data Structures Page 18 Parallel Merging If a block in A i is larger than O (log m ) and the matching block of A i is B j, we do the following We divide A i into blocks of size O (log m ). Then we apply the same algorithm to rank the boundary elements of each block in A i in B j. Now each block in A is of size O (log m ) This takes O (log log m) time.

Advanced Topics in Algorithms and Data Structures Page 19 Parallel Merging Step 3. We now take every pair of matching blocks from A and B and run the sequential merging algorithm. One processor is allocated for every matching pair and this processor merges the pair in O (log m ) time. We have to analyse the time and processor complexities of each of the steps to get the overall complexities.

Advanced Topics in Algorithms and Data Structures Page 20 Parallel Merging Complexity of Step 1 The task in Step 1 is to partition B into blocks of size log m. We allocate m /log m processors. Since B is an array, processor P i, 1  i  m /log m can find the element i  log m in O (1) time.

Advanced Topics in Algorithms and Data Structures Page 21 Parallel Merging Complexity of Step 2 In Step 2, m /log m processors do binary search in array A in O (log n ) time each. Hence the time complexity is O (log n ) and the work done is (m  log n )/ log m  (m  log( m + n )) / log m  ( m + n ) for n, m  4. Hence the total work is O ( m + n ).

Advanced Topics in Algorithms and Data Structures Page 22 Parallel Merging Complexity of Step 3 In Step 3, we use m /log m processors Each processor merges a pair A i, B i in O (log m ) time.Hence the total work done is m. Theorem Let A and B be two sorted sequences each of length n. A and B can be merged in O (log n ) time using O ( n ) operations in the CREW PRAM.