“Sorting networks and their applications”, AFIPS Proc. of 1968 Spring Joint Computer Conference, Vol. 32, pp 307-314.

Slides:



Advertisements
Similar presentations
PERMUTATION CIRCUITS Presented by Wooyoung Kim, 1/28/2009 CSc 8530 Parallel Algorithms, Spring 2009 Dr. Sushil K. Prasad.
Advertisements

Parallel Sorting Sathish Vadhiyar. Sorting  Sorting n keys over p processors  Sort and move the keys to the appropriate processor so that every key.
Lecture 3: Parallel Algorithm Design
1 Parallel Parentheses Matching Plus Some Applications.
Theory of Computing Lecture 3 MAS 714 Hartmut Klauck.
Advanced Topics in Algorithms and Data Structures Lecture 7.1, page 1 An overview of lecture 7 An optimal parallel algorithm for the 2D convex hull problem,
Advanced Topics in Algorithms and Data Structures Lecture pg 1 Recursion.
1 Today’s Material Divide & Conquer (Recursive) Sorting Algorithms –QuickSort External Sorting.
Algorithms: Sorting. Rand Sort. Compare-and-exchange. Merge Sort. Quick Sort. Odd-even merge sort. Bitonic merge sort.
Parallel Sorting Algorithms Comparison Sorts if (A>B) { temp=A; A=B; B=temp; } Potential Speed-up –Optimal Comparison Sort: O(N lg N) –Optimal Parallel.
Advanced Topics in Algorithms and Data Structures Lecture 6.1 – pg 1 An overview of lecture 6 A parallel search algorithm A parallel merging algorithm.
Advanced Topics in Algorithms and Data Structures Page 1 Parallel merging through partitioning The partitioning strategy consists of: Breaking up the given.
Slides for Parallel Programming Techniques & Applications Using Networked Workstations & Parallel Computers 2nd ed., by B. Wilkinson & M
1 Tuesday, November 14, 2006 “UNIX was never designed to keep people from doing stupid things, because that policy would also keep them from doing clever.
Chapter 10 in textbook. Sorting Algorithms
Sorting Algorithms CS 524 – High-Performance Computing.
Algorithms and Applications
Parallel Merging Advanced Algorithms & Data Structures Lecture Theme 15 Prof. Dr. Th. Ottmann Summer Semester 2006.
CHAPTER 11 Sorting.
Sorting Algorithms: Topic Overview
CS 584. Sorting n One of the most common operations n Definition: –Arrange an unordered collection of elements into a monotonically increasing or decreasing.
Sorting Algorithms Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar To accompany the text ``Introduction to Parallel Computing'', Addison Wesley,
CS 584. Sorting n One of the most common operations n Definition: –Arrange an unordered collection of elements into a monotonically increasing or decreasing.
Bitonic and Merging sorting networks Efficient Parallel Algorithms COMP308.
CSCI-455/552 Introduction to High Performance Computing Lecture 22.
MergeSort Source: Gibbs & Tamassia. 2 MergeSort MergeSort is a divide and conquer method of sorting.
Basic PRAM algorithms Problem 1. Min of n numbers Problem 2. Computing a position of the first one in the sequence of 0’s and 1’s.
Simulating a CRCW algorithm with an EREW algorithm Lecture 4 Efficient Parallel Algorithms COMP308.
1 Sorting Algorithms - Rearranging a list of numbers into increasing (strictly non-decreasing) order. ITCS4145/5145, Parallel Programming B. Wilkinson.
Lecture 12: Parallel Sorting Shantanu Dutt ECE Dept. UIC.
Computer Science and Engineering Parallel and Distributed Processing CSE 8380 February 8, 2005 Session 8.
1 Parallel Sorting Algorithms. 2 Potential Speedup O(nlogn) optimal sequential sorting algorithm Best we can expect based upon a sequential sorting algorithm.
Slides for Parallel Programming Techniques & Applications Using Networked Workstations & Parallel Computers 2nd ed., by B. Wilkinson & M
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 Chapter.
Outline  introduction  Sorting Networks  Bubble Sort and its Variants 2.
CS223 Advanced Data Structures and Algorithms 1 Sorting and Master Method Neil Tang 01/21/2009.
Lecture 6 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea.
1. 2 Sorting Algorithms - rearranging a list of numbers into increasing (strictly nondecreasing) order.
Sorting Data Structures and Algorithms (60-254). Sorting Sorting is one of the most well-studied problems in Computer Science The ultimate reference on.
Lecture 4 Sorting Networks. Comparator comparator.
Slides for Parallel Programming Techniques & Applications Using Networked Workstations & Parallel Computers 2nd ed., by B. Wilkinson & M
1 Parallel Sorting Algorithm. 2 Bitonic Sequence A bitonic sequence is defined as a list with no more than one LOCAL MAXIMUM and no more than one LOCAL.
CSCI-455/552 Introduction to High Performance Computing Lecture 23.
Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar
Data Structures and Algorithms in Parallel Computing Lecture 8.
Parallel Programming - Sorting David Monismith CS599 Notes are primarily based upon Introduction to Parallel Programming, Second Edition by Grama, Gupta,
Unit-8 Sorting Algorithms Prepared By:-H.M.PATEL.
Sorting: Parallel Compare Exchange Operation A parallel compare-exchange operation. Processes P i and P j send their elements to each other. Process P.
S ORTING ON P ARALLEL C OMPUTERS Dr. Sherenaz Al-Haj Baddar KASIT University of Jordan
Merge Sort.
Lecture 4 Sorting Networks
Lecture 3: Parallel Algorithm Design
Sorting Networks Characteristics The basic unit: a comparator
Auburn University COMP7330/7336 Advanced Parallel and Distributed Computing Parallel Odd-Even Sort Algorithm Dr. Xiao.
Chapter 4 Divide-and-Conquer
Parallel Sorting Algorithms
MergeSort Source: Gibbs & Tamassia.
Bitonic Sorting and Its Circuit Design
CS Two Basic Sorting Algorithms Review Exchange Sorting Merge Sorting
Data Structures Review Session
Parallel Sorting Algorithms
Bitonic and Merging sorting networks
Lecture 6 Algorithm Analysis
Parallel Sorting Algorithms
Lecture 6 Algorithm Analysis
Quick-Sort 2/25/2019 2:22 AM Quick-Sort     2
Quick-Sort 4/8/ :20 AM Quick-Sort     2 9  9
Sorting Algorithms - Rearranging a list of numbers into increasing (strictly non-decreasing) order. Sorting number is important in applications as it can.
Parallel Sorting Algorithms
DIVIDE AND CONQUER.
Presentation transcript:

“Sorting networks and their applications”, AFIPS Proc. of 1968 Spring Joint Computer Conference, Vol. 32, pp Kenneth E. Batcher Professor, Kent State University

SJTU2013SJTU2013 Background Sorting is fundamental Low bound of any sequential sorting algorithms is O(nlogn) Can we improve the time complexity further? –Parallel algorithms –Circuit/Network Design – Parallel Computing Models

SJTU2013SJTU2013 ① Bitonic Sequence 双调序列 sequence of elements {a 0, a 1, …, a n-1 } where either –(1) there exists an index, i, 0  i  n-1, such that {a 0, …, a i } is monotonically increasing, and {a i+1, …, a n-1 } is monotonically decreasing, –e.g. {1, 2, 4, 7, 6, 0} Or –(2) there exists a cyclic shift of indices so that (1) is satisfied –e.g. {8, 9, 2, 1, 0, 4}  {0, 4, 8, 9, 2, 1}

SJTU2013SJTU2013 Value of element aiai a 0 a 1 a 2 a 3 a 4 a 5 a 6 a 7 { 3, 5, 7, 9, 8, 6, 4, 2 } Value of element aiai a 0 a 1 a 2 a 3 a 4 a 5 a 6 a 7 { 8, 6, 4, 2, 3, 5, 7, 9} ① Bitonic Sequence : Examples

SJTU2013SJTU2013 Value of element aiai a 0 a 1 a 2 a 3 a 4 a 5 a 6 a 7 { 3, 5, 7, 9, 11, 13, 15, 17 } ① Bitonic Sequence : Examples Value of element aiai a 0 a 1 a 2 a 3 a 4 a 5 a 6 a 7 { 5, 3, 1, 2, 4, 6, 8, 7 }

SJTU2013SJTU2013 Bitonic Sort: basic idea Consider a bitonic sequence S of size n where –the first half ( {a 0, a 1, …, a n/2-1 } ) is increasing, and the second half ( {a n/2, a n/2+1, …, a n-1 } ) is decreasing Value of element aiai a 0 a 1... a n/2-1 a n/2 a n/2 +1 … a n-1

SJTU2013SJTU2013 ② “Bitonic Split” 双调分裂 Pair-wise min-max comparison –s 1 = {min(a 0, a n/2 ), min(a 1, a n/2+1 ), …, min(a n/2-1, a n-1 )} –s 2 = {max(a 0, a n/2 ), max(a 1, a n/2+1 ), …, max(a n/2-1, a n-1 )} a n/2 a0a0 a n/2-1 a n-1 value aiai a 0 a 1... a n/2-1 a n/2 a n/2 +1 … a n-1 Compare and exchange S2S2 S1S1 value

SJTU2013SJTU2013 There exists –an element b in S 1 such that all elements before b is increasing and all elements after b is decreasing –an element c in S 2 such that all elements before c is decreasing and all elements after c is increasing S 1 and S 2 –Both S 1 and S 2 are bitonic sequences –Any elements in S 1 < any elements in S 2 (because b < c and b is the maximum value in S 1 and c is the minimum value in S 2 ) S2S2 S1S1 value c b

SJTU2013SJTU2013 pair-wise min-max comparison e.g. { 2, 4, 6, 8, 7, 5, 3, 1} { 2, 4, 6, 8 7, 5, 3, 1 } => S1={2, 4, 3, 1} S2={7, 5, 6, 8} bitonic sequence of size 8 => 2 bitonic sequence of size 4 Compare and exchange

SJTU2013SJTU2013 ② Bitonic Split The split is applicable to any bitonic sequence. Need not to have the 1 st half to be increasing/decreasing and the 2 nd half to be decreasing/increasing: Bitonic(n) 2 Bitonic(n/2) Bitonic Split

SJTU2013SJTU2013 Sorting a bitonic sequence By using bitonic split recursively, INPUT: a bitonic sequence of size n  Phase 1: 2 bitonic sequence of size n/2  Phase 2: 4 bitonic sequence of size n/4  …  Phase (log n): n bitonic sequence of size 1  a sorted sequence can be generated by concatenating the n bitonic sequence of size 1

SJTU2013SJTU2013 ③ Bitonic Merge 双调合并 sort a bitonic sequence using bitonic splits length

SJTU2013SJTU2013 Bitonic Merge Circuit :  BM[16]

SJTU2013SJTU2013 Questions ? How can we convert an unsorted sequence to a bitonic sequence ? (then, by using bitonic split recursively, a sorted sequence can be formed).

SJTU2013SJTU length Turn an unsorted sequence into a bitonic sequence: ③ Bitonic Merge (BM) Operation At every phase, sort a bitonic sequence of size 2, 4, 8, 16 into a monotonically increasing or decreased sequence

SJTU2013SJTU2013 Turn an unsorted sequence into a bitonic sequence

SJTU2013SJTU length ④ Bitonic Sort

SJTU2013SJTU2013 Sort (any ordered of) sequence Using bitonic merge repeatedly Definition: –  BM[n]: increasing bitonic merge of size n bitonic merge : sort a bitonic sequence of size n into a monotonically increasing sequence –  BM[n]: decreasing bitonic merge of size n bitonic merge that sort a bitonic sequence of size n into a monotonically decreasing sequence

SJTU2013SJTU2013 Steps: Divide the sequence into a group of 2 –any sequence of size 2 is a bitonic sequence: either the increasing part is of size 2 and the decreasing part is of size 0, or vice versa Using  BM[2] on a group to form an increasing sequence, and  BM[2] on the adjacent group to form an decreasing sequence Concatenate the two group to form a bitonic sequence of size 4

SJTU2013SJTU2013 Repeat the above steps on other groups Repeat the above steps recursively, until a bitonic sequence of size n is formed Using bitonic merge again to turn the bitonic sequence into a sorted sequence Steps:

SJTU2013SJTU2013 –  BM[n]: increasing bitonic merge of size n –  BM[n]: decreasing bitonic merge of size n Bitonic Sorting Circuit: BS(18)

SJTU2013SJTU2013 Sort (any ordered of) sequence Hence, n unsorted numbers n/2 group of 2-number bitonic sequence n/4 group of 4-number bitonic sequence … 1 group of n-number bitonic sequence a sorted sequence

SJTU2013SJTU2013 ⑤ Complexity of Bitonic Sort Parallel bitonic sort with n processor –The last stage of an n-element bitonic sorting need to merge n-element, and has a depth of log(n) –Other stages perform a complete sort of n/2 elements –Depth, d(n) = d(n/2) + log(n) –d(n) = … + log(n) =  (log 2 n) –Complexity: T(n) =  (log 2 n)

SJTU2013SJTU2013 ⑤ Complexity of Bitonic Sort Parallel sorting with a block of elements per processor –sort the local block of elements first (using any sorting algorithm such as quicksort, bitonic sort) –sort the elements among processors using parallel bitonic sort –T(n) = T(local_sort) + T(comparisons) +T(communication) Only computation time is considered here (you need to consider all communication time also)

SJTU2013SJTU2013 ⑥ Concluding Remarks Bitonic Sorting: Common Sense Regression to Computer Science One of 10 Most Important Papers Parallel Algorithm: Ascend/Descend –Another example: Prefix sum Network Model:

SJTU2013SJTU Bitonic Sorting Network Hypercube connections! Try to Write Bitonic Sorting algorithm on hypercube.

SJTU2013SJTU Bitonic Sort on Butterfly

SJTU2013SJTU Bitonic Sort on Butterfly

SJTU2013SJTU Bitonic Sort on Butterfly

SJTU2013SJTU Bitonic Sort on Butterfly

SJTU2013SJTU Bitonic Sort on Butterfly

SJTU2013SJTU Bitonic Sort on Butterfly

SJTU2013SJTU Bitonic Sort on Butterfly

SJTU2013SJTU Bitonic Sort on Butterfly

SJTU2013SJTU Bitonic Sort on Butterfly

SJTU2013SJTU Bitonic Sort on Butterfly

SJTU2013SJTU Bitonic Sort on Butterfly

SJTU2013SJTU Bitonic Sort on Butterfly

SJTU2013SJTU Bitonic Sort on Butterfly

SJTU2013SJTU2013 PRAM Model P1P2P3Pn … Memory Access time from any processor to any memory unit is equal It is impossible in practice So it is an ideal model for parallel computing

SJTU2013SJTU2013 PRAM Model Program for Sum= a(1)+a(2)+…+a(N) for i = 1 to log N for j= 1 to n/ 2 i parallel do a(j) = a(j) + a(N/ 2 i + j) endpar endfor Finally a(1) is the sum

SJTU2013SJTU2013 Hypercube Model Suppose node N(i) holds element a(i), where i is the value of node index x 1 x 2 …x n for i = 1 to n for j= i to n parallel do N(00…0 (x j =0) x j+1 …x n )  N(00…0 (x j =1) x j+1 …x n ); a(00…0 (x j =0) x j+1 …x n ) = a(00…0 (x j =0) x j+1 …x n ) + a(00…0 (x j =1) x j+1 …x n ) endpar endfor Finally node 00…0 holds the sum

SJTU2013SJTU2013 Hypercube Model Suppose node 000 holds element a(0) and 111holds element a(7) a(4) a(1) a(3) a(7) a(2) a(0) a(5) a(6) a(1)+a(5) a(3)+a(7)a(2)+a(6) a(0)+a(4) a(1)+a(5)+a(3)+a(7) +a(2)+a(6)a(0)+a(4) +a(1)+a(5)+a(3)+a(7)+a(2)+a(6)a(0)+a(4)