1 Lecture 11 Sorting Parallel Computing Fall 2008.

Slides:



Advertisements
Similar presentations
Section 5: More Parallel Algorithms
Advertisements

PRAM Algorithms Sathish Vadhiyar. PRAM Model - Introduction Parallel Random Access Machine Allows parallel-algorithm designers to treat processing power.
Lecture 3: Parallel Algorithm Design
Algorithms Analysis Lecture 6 Quicksort. Quick Sort Divide and Conquer.
1 Parallel Parentheses Matching Plus Some Applications.
ALGORITMOS DE ORDENACIÓN EN PARALELO
Lecture 7 : Parallel Algorithms (focus on sorting algorithms) Courtesy : SUNY-Stony Brook Prof. Chowdhury’s course note slides are used in this lecture.
QuickSort 4 February QuickSort(S) Fast divide and conquer algorithm first discovered by C. A. R. Hoare in If the number of elements in.
Advanced Topics in Algorithms and Data Structures Lecture pg 1 Recursion.
Sorting Chapter Sorting Consider list x 1, x 2, x 3, … x n We seek to arrange the elements of the list in order –Ascending or descending Some O(n.
Parallel Sorting Algorithms Comparison Sorts if (A>B) { temp=A; A=B; B=temp; } Potential Speed-up –Optimal Comparison Sort: O(N lg N) –Optimal Parallel.
Advanced Topics in Algorithms and Data Structures Lecture 6.1 – pg 1 An overview of lecture 6 A parallel search algorithm A parallel merging algorithm.
Advanced Topics in Algorithms and Data Structures Page 1 Parallel merging through partitioning The partitioning strategy consists of: Breaking up the given.
Uzi Vishkin.  Introduction  Objective  Model of Parallel Computation ▪ Work Depth Model ( ~ PRAM) ▪ Informal Work Depth Model  PRAM Model  Technique:
Slides for Parallel Programming Techniques & Applications Using Networked Workstations & Parallel Computers 2nd ed., by B. Wilkinson & M
1 Tuesday, November 14, 2006 “UNIX was never designed to keep people from doing stupid things, because that policy would also keep them from doing clever.
Chapter 10 in textbook. Sorting Algorithms
Sorting Algorithms CS 524 – High-Performance Computing.
1 Friday, November 17, 2006 “In the confrontation between the stream and the rock, the stream always wins, not through strength but by perseverance.” -H.
Algorithms and Applications
Parallel Merging Advanced Algorithms & Data Structures Lecture Theme 15 Prof. Dr. Th. Ottmann Summer Semester 2006.
CHAPTER 11 Sorting.
Sorting Algorithms: Topic Overview
Sorting Algorithms Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar To accompany the text ``Introduction to Parallel Computing'', Addison Wesley,
CS 584. Sorting n One of the most common operations n Definition: –Arrange an unordered collection of elements into a monotonically increasing or decreasing.
CSCI-455/552 Introduction to High Performance Computing Lecture 22.
1 Sorting Algorithms - Rearranging a list of numbers into increasing (strictly non-decreasing) order. ITCS4145/5145, Parallel Programming B. Wilkinson.
Lecture 12: Parallel Sorting Shantanu Dutt ECE Dept. UIC.
Chapter 6 Parallel Sorting Algorithm Sorting Parallel Sorting Bubble Sort Odd-Even (Transposition) Sort Parallel Odd-Even Transposition Sort Related Functions.
1 Parallel Sorting Algorithms. 2 Potential Speedup O(nlogn) optimal sequential sorting algorithm Best we can expect based upon a sequential sorting algorithm.
Slides for Parallel Programming Techniques & Applications Using Networked Workstations & Parallel Computers 2nd ed., by B. Wilkinson & M
Fall 2013 Instructor: Reza Entezari-Maleki Sharif University of Technology 1 Fundamentals of Programming Session 17 These.
Outline  introduction  Sorting Networks  Bubble Sort and its Variants 2.
C++ Programming: Program Design Including Data Structures, Fourth Edition Chapter 19: Searching and Sorting Algorithms.
CS453 Lecture 3.  A sequential algorithm is evaluated by its runtime (in general, asymptotic runtime as a function of input size).  The asymptotic runtime.
C++ Programming: From Problem Analysis to Program Design, Second Edition Chapter 19: Searching and Sorting.
Sorting Chapter Sorting Consider list x 1, x 2, x 3, … x n We seek to arrange the elements of the list in order –Ascending or descending Some O(n.
Chapter 18: Searching and Sorting Algorithms. Objectives In this chapter, you will: Learn the various search algorithms Implement sequential and binary.
1. 2 Sorting Algorithms - rearranging a list of numbers into increasing (strictly nondecreasing) order.
Review 1 Selection Sort Selection Sort Algorithm Time Complexity Best case Average case Worst case Examples.
1 Searching and Sorting Searching algorithms with simple arrays Sorting algorithms with simple arrays –Selection Sort –Insertion Sort –Bubble Sort –Quick.
Slides for Parallel Programming Techniques & Applications Using Networked Workstations & Parallel Computers 2nd ed., by B. Wilkinson & M
Chapter 9 Sorting 1. The efficiency of data handling can often be increased if the data are sorted according to some criteria of order. The first step.
Chapter 9 Sorting. The efficiency of data handling can often be increased if the data are sorted according to some criteria of order. The first step is.
Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar
ICS201 Lecture 21 : Sorting King Fahd University of Petroleum & Minerals College of Computer Science & Engineering Information & Computer Science Department.
1 Ch.19 Divide and Conquer. 2 BIRD’S-EYE VIEW Divide and conquer algorithms Decompose a problem instance into several smaller independent instances May.
Data Structures and Algorithms in Parallel Computing Lecture 8.
Non-Linear Algebra Problems Sathish Vadhiyar SERC IISc.
Parallel Programming - Sorting David Monismith CS599 Notes are primarily based upon Introduction to Parallel Programming, Second Edition by Grama, Gupta,
Unit-8 Sorting Algorithms Prepared By:-H.M.PATEL.
CSCI-455/552 Introduction to High Performance Computing Lecture 21.
Sorting: Parallel Compare Exchange Operation A parallel compare-exchange operation. Processes P i and P j send their elements to each other. Process P.
Data Structures and Algorithms Instructor: Tesfaye Guta [M.Sc.] Haramaya University.
Sorting – Lecture 3 More about Merge Sort, Quick Sort.
Divide and Conquer Algorithms Sathish Vadhiyar. Introduction  One of the important parallel algorithm models  The idea is to decompose the problem into.
Searching and Sorting Searching algorithms with simple arrays
Lecture 3: Parallel Algorithm Design
Auburn University COMP7330/7336 Advanced Parallel and Distributed Computing Parallel Odd-Even Sort Algorithm Dr. Xiao.
Teach A level Computing: Algorithms and Data Structures
Parallel Sorting Algorithms
Algorithm Design Methods
Quicksort and Mergesort
Unit-2 Divide and Conquer
Parallel Computing Spring 2010
Parallel Sorting Algorithms
Sub-Quadratic Sorting Algorithms
Parallel Sorting Algorithms
Sorting Algorithms - Rearranging a list of numbers into increasing (strictly non-decreasing) order. Sorting number is important in applications as it can.
Parallel Sorting Algorithms
Presentation transcript:

1 Lecture 11 Sorting Parallel Computing Fall 2008

2 Sorting Algorithm Rearranging a list of numbers into increasing (strictly nondecreasing) order.

3 Potential Speedup O(nlogn) optimal for any sequential sorting algorithm without using special properties of the numbers. Best we can expect based upon a sequential sorting algorithm using n processors is Optimal parallel time complexity = O(n logn)/n= O(logn) Has been obtained but the constant hidden in the order notation extremely large.

4 Compare-and-Exchange Sorting Algorithms: Compare and Exchange Form the basis of several, if not most, classical sequential sorting algorithms. Two numbers, say A and B, are compared. If A > B, A and B are exchanged, i.e.: if (A > B) { temp = A; A = B; B = temp; }

5 Message Passing Method For P1 to send A to P2 and P2 to send B to P1. Then both processes perform compare operations. P1 keeps the larger of A and B and P2 keeps the smaller of A and B:

6 Merging Two Sublists

7 Bubble Sort First, largest number moved to the end of list by a series ofcompares and exchanges, starting at the opposite end. Actions repeated with subsequent numbers, stopping just before the previously positioned number. In this way, the larger numbers move (“bubble”) toward one end,

8 Bubble Sort

9 Time Complexity Number of compare and exchange operations Indicates a time complexity of O(n^2) given that a single compare-and-exchange operation has a constant complexity, O(1).

10 Parallel Bubble Sort Iteration could start before previous iteration finished if doesnot overtake previous bubbling action:

11 Odd-Even (Transposition) Sort Variation of bubble sort. Operates in two alternating phases, even phase and odd phase. Even phase Even-numbered processes exchange numbers with their right neighbor. Odd phase Odd-numbered processes exchange numbers with their right neighbor.

12 Sequential Odd-Even Transposition Sort

13 Parallel Odd-Even Transposition Sort Sorting eight numbers

14 Parallel Odd-Even Transposition Sort Consider the one item per processor case. There are n iterations; in each iteration, each processor does one compare-exchange –which can all be done in parallel. The parallel run time of this formulation is Θ(n). This is cost optimal with respect to the base serial algorithm but not the optimal serial algorithm.

15 Parallel Odd-Even Transposition Sort

16 Parallel Odd-Even Transposition Sort Consider a block of n/p elements per processor. The first step is a local sort. In each subsequent step, the compare exchange operation is replaced by the compare split operation. There are p phases with each phase performing Θ(n/p) compares and Θ(n/p) communication. The parallel run time of the formulation is The parallel formulation is cost-optimal for p= O(logn).

17 Quicksort Very popular sequential sorting algorithm that performs well with average sequential time complexity of O(nlogn). First list divided into two sublists. All numbers in one sublist arranged to be smaller than all numbers in other sublist. Achieved by first selecting one number, called a pivot, against which every other number is compared. If the number is less than the pivot, it is placed in one sublist. Otherwise, it is placed in the other sublist. Pivot could be any number in the list, but often first number in list chosen. Pivot itself could be placed in one sublist, or the pivot could be separated and placed in its final position

18 Quicksort

19 Quicksort Example of the quicksort algorithm sorting a sequence of size n= 8.

20 Parallel Quicksort Lets start with recursive decomposition -the list is partitioned by a single process and then each of the subproblems is handled by a different processor. The time for this algorithm is lower-bounded by Ω(n)! –Not cost optimal as the process-time product is Ω(n^2). Can we parallelize the partitioning step -in particular, if we can use n processors to partition a list of length n around a pivot in O(1)time, we have a winner. This is difficult to do on real machines, though.

21 Parallel Quicksort Using tree allocation of processes

22 Parallel Quicksort With the pivot being withheld in processes:

23 Analysis Fundamental problem with all tree constructions –initial division done by a single processor, which will seriously limit speed. Tree in quicksort will not, in general, be perfectly balanced Pivot selection very important to make quicksort operate fast.

24 Parallelizing Quicksort: PRAM Formulation We assume a CRCW (concurrent read, concurrent write) PRAM with concurrent writes resulting in an arbitrary write succeeding. The formulation works by creating pools of processors. Every processor is assigned to the same pool initially and has one element. Each processor attempts to write its element to a common location (for the pool). Each processor tries to read back the location. If the value read back is greater than the processor's value, it assigns itself to the 'left' pool, else, it assigns itself to the 'right' pool. Each pool performs this operation recursively. Note that the algorithm generates a tree of pivots. The depth of the tree is the expected parallel runtime. The average value is O(logn).

25 Parallelizing Quicksort: PRAM Formulation A binary tree generated by the execution of the quicksortalgorithm. Each level of the tree represents a different array-partitioning iteration. If pivot selection is optimal, then the height of the tree is Θ(log n), which is also the number of iterations.

26 Parallelizing Quicksort: PRAM Formulation The execution of the PRAM algorithm on the array shown in (a).

27 End Thank you!