1. Parallel Sorting Algorithms Comparison Sorts if (A>B) { temp=A; A=B; B=temp; } Potential Speed-up –Optimal Comparison Sort: O(N lg N) –Optimal Parallel speed-up O(lg N) if P=N –Huge Big Oh constant Source of problems –Duplicate computations to reduce message passing –Different precision at different processors Key –Processors independently working on data sections AB Max(A,B) Min(A,B) B Max(A,B) A

2. Data Partitioning P processors and n numbers 88 50 28 25 43 42 28 25 88 50 28 25 98 80 43 42 98 88 80 50 43 42 28 25 Version 1 P2 Performs Comparisons Version 2 Duplicate Computations 88 50 28 25 98 80 43 42 98 88 80 50 43 42 28 25 88 50 28 25 98 80 43 42 98 88 80 50 43 42 28 25 P1 P2 P1 P1 Gets Smaller Numbers; P2 Gets Larger Numbers Question: What is the complexity of finding the mid point?

3. Sequential Bubble Sort Pass = 0; Do more = false; for (int i=0; i<n-pass; i++) if (array[i] > array[i+1]) {t=a[i]; a[i]=a[i+1]; a[i+1]=t; more=true;} pass++; Until (swaps = false); Complexity: ∑ i=1->n i = n(n-1)/2 Example: Sort: 42785136 End of Pass 1: 24751386End of Pass 2: 24513678 End of Pass 3: 24135678 End of Pass 4: 21345678 End of Pass 5: 12345678

4. Parallel Bubble Sort Pipeline Approach –Start the next pass before the previous iteration completes –Ensure that the next pass doesn’t overtake the previous pass Phase 4Phase 3Phase 2Phase 1

5. Odd-Even Transposition Sort Bubble Sort Modification Operates in even and odd passes –Even passes: exchange with right neighbor –Odd passes: exchange with left neighbor 4 2 7 8 5 1 3 6 2 4 7 1 8 3 5 6 2 1 4 3 7 5 8 6 1 2 3 4 5 6 7 8 0123456701234567 4 7 8 1 5 3 4 1 7 3 8 5 2 3 4 5 7 6 2 2 1 1 2 3 4 5 6 7 6 6 8 8 Step P 0 P 1 P 2 P 3 P 4 P 5 P 6 P 7

6. Merge Sort An optimal comparison sort Complexity: O(n lg n) Divide and conquer process allocation mergeSort(array1, array2, low, high) { middle = (low+high)/2; mergeSort(array, low, middle) mergeSort(array, middle+1, high) merge(array, low, middle+1, high) } 42785136 42785136 42785136 42785136 24781536 24781356 12345678

7. Quick Sort Perhaps the most popular sequential sorting method Average optimal sequential complexity: O(n lg n) Parallel efficiency limitations –Partitions are unbalanced –A single processor performs the initial partitioning Work pool approach –One part given to a processor, another returned to work pool 42785136 32145786 21345786 123678

8. Bitonic Sequences Monotonically increasing sequence –A sequence of increasing numbers (E.g.: 12345678) Bitonic sequence –Two sequences, one increasing and one decreasing( E.g.: 56784321) –Note: a sequence is still considered Bitonic if the above condition occurs after performing a rotate operation –E.g.: 21567843 (Perform left rotate of two digits) Special Characteristic of a Bitonic Sequence –Lg(n) compare and exchange operations on two Bitonic sequence results in a monotonic sequence –Notes: The first half of the sequence is monotonically increasing The second half of the sequence is monotonically decreasing

9. Special Bitonic Properties Assumptions –A[0], A[1], A[n-1] is a Bitonic sequence –A[0]  A[n/2] is monotonically increasing –A[n/2+1]  A[n] is monotonically decreasing –The value of n is n even power of 2 Goal: Form a monotonic sequence of size n Solution: Requires lg(n) steps –Steps: BitonicMerges of n/2, n/4, …, 1 –Bitonic k merge compares and exchanges A[k] with A[k+i]; 0<=k < n-i. 3 5 8 9 7 4 2 1 3 4 2 1 7 5 8 9 Bitonic The original bitonic sequence Example of a Bitonic 4 Merge where n=8

10. Bitonic Merge Sort Form Bitonic pairs (k=2) of adjacent numbers alternating between increasing and decreasing orders At each step i, recursively form larger Bitonic sequences (|k i+1| = |2*k i| ) –Remember to perform lg(i) Bitonic merges at each step The sort completes in lg 2 n steps. 8 3 4 79 21 5 3 8 7 4 2 9 5 1 3 4 7 8 5 9 2 1 3 4 7 8 9 5 2 1 Step 1: k=2, lg 2 steps Step 2: k=4, lg 4 steps 3 4 2 1 9 5 7 8 2 1 3 47 5 9 8 3 4 7 8 9 5 2 1 1 2 3 4 5 7 8 9 Step 3: k=8, lg 8 steps Example

11. Larger Bitonic Sort Example 1.Original Unsorted Numbers: 20,15,7,5,3,8,12,4,16,13,12,18,9,4,3,2 2.After Sorting Pairs of Numbers: 15,20, 7,5, 3,8, 12,4, 13,16, 18,12, 4,9, 3,2 3.Groups of four merged: 0->2,1->3; 4->6,5->7; 8->10, 9->11; 12->14, 13->15 7,5,15,20, 12,8,3,4, 13,12,18,16, 4,9,3,2 4.Groups of two merged: 0->1; 2->3; 4->5; 6->7; 8->9; 10->11; 12->13; 14->15 5,7, 15,20, 12,8, 4,3, 12,13, 16,18, 9,4, 3,2 5.Groups of eight merged: 0->4,1->5,2->6,3->7, 8->12,9->13,10->14,11->15 5,7,4,3,12,8,15,20, 12,13,16,18,9,4,3,2 6.Groups of four merged: 0->2,1->3; 4->6,5->7; 8->10, 9->11; 12->14, 13->15 4,3,5,7, 12,8,15,20, 16,18,12,13, 9,4,3,2 7.Groups of two merged: 0->1; 2->3; 4->5; 6->7; 8->9; 10->11; 12->13; 14->15 3,4, 5,7, 8,12, 15,20, 18,16, 13,12, 9,4, 3,2 8.Group of sixteen merged: 0->8; 1->9; 2->10; 3->11; 4->12; 5->13; 6->14; 7->15 3,4, 5,7, 8,4,3,2,18,16,13,12,9,12,15,20 9.Groups of eight merged: 0->4,1->5,2->6,3->7, 8->12,9->13,10->14,11->15 3,4, 5,7, 8,4,3,2, 9,12,13,12,18,16,15,20 10.Groups of four merged: 0->2,1->3; 4->6,5->7; 8->10, 9->11; 12->14, 13->15 3,2, 3,4, 5,4, 8,7, 9,12, 13,12, 15,16, 18,20 11.After final merge: 0->1; 2->3; 4->5; 6->7; 8->9; 10->11; 12->13; 14->15 2,3,3,4,4,5,7,8,9,12,12,13,15,16,18,20 Notes: Red indicates ascending; Blue indicates descending

12. Batcher’s Odd-Even Merge Sort Assumptions –A[0] to A[n-1] sorted –Array elements A[n] to A[2*n-1] are sorted Goal: Form a sorted array of length 2*n Procedure –Sort the even indices –Sort the odd indices –Perform an odd-even merge Odd-even merge –Leave A[0] and A[2*n-1] unchanged –Compare & swap: A[i]&A[i+1] i=1;i<2*n-1;i+=2 a low [] 2 4 5 8 a high [] 1 3 6 7 Two sorted listsEven and odd indices sorted a low [] 1 3 2 4 a high [] 5 7 6 8 Compare and exchange a low [] 1 2 3 4 a high [] 5 6 7 8 Odd-even Merge Example: Form sorted array of 8 elements

13. Larger Odd-Even Example 1.Original Unsorted Numbers 20,15,7,5,3,8,12,4,16,13,12,18,9,4,3,2 2.Sort Pairs of Numbers 15,20, 5,7, 3,8, 4,12, 13,16, 12,18, 4,9, 2,3 3.Odd-even sort of indices 0-3, 4-7, 8-11, 12-15 5,7,15,20 3,8,4,12, 12,16,13,18 2,3,4,9 4.Odd-even merge 5,7,15,20 3,4,8,12, 12,13,16,18 2,3,4,9 5.Odd-even sort of indices 0-7, 8-15 3,4,5,7,8,12,15,20, 2,3,4,9,12,13,16,18 6.Odd-even merge 3,4,5,7,8,12,15,20 2,3,4,9,12,13,16,18 7.Odd-even sort of indices 0-15 2,3,3,4,4,7,5,9,8,12,12,13,15,18,16,20 8.Odd-even merge 2,3,3,4,4,5,7,8,9,12,12,13,15,16,18,20