1. S ORTING ON P ARALLEL C OMPUTERS Dr. Sherenaz Al-Haj Baddar KASIT University of Jordan s.baddar@ju.edu.jo

2. S ORTING IN PARALLEL ? Either parallelize sequential sorting algorithms Parallel quick sort, bubble sort, insertion sort Use parallel sorting algorithms [K] Bitonic sorting oblivious sorting! Odd/even merge sorting Cole’s parallel merge sort …etc

3. B ITONIC S ORTING well-known parallel sorting algorithm Fast and efficient n keys are sorted in O(log 2 n) time using O(n) PEs Yeah not cost optimal! Cool comparison pattern Developed by K.E. Batcher back in 1968[B] Gaining even more importance because it is cool to use with GPUs and FPGAs

4. H OW D OES B ITONIC S ORTING W ORK ? The Bitonic Sorting Strategy Max(a 1,a n+1 ) Min(a 1,a n+1 ) Max(a 2,a n+2 ) Min(a 2,a n+2 ) Min(a n,a 2n ) Max(a n,a 2n ) Max(a n-1,a 2n-1 ) Min(a n-1,a 2n-1 ) H L merge Bitonic sequences

5. B ITONIC S ORTING A LGORITHM [G]

6. H OW TO MAP IT TO A PARALLEL COMPUTER ? What parallel computer are you using? After you answer this question you can “map” bitonic sorting to that parallel computer Well-known examples [G]: Mapping bitonic sorting to hypercubes Mapping bitonic sorting to trees Mapping bitonic sorting to Mesh networks ….etc

7. W HAT IS THE POINT ? Bitonic sorting( & odd-even merge sorting) is the fastest oblivious sorting algorithm we know so far It maps naturally to hypercubes Minimal communication cost Performs pretty well on parallel computers with symmetric architectures Communications patterns are easy to implement Cool for sorting in masses!

8. E VALUATE B ITONIC SORTING MAPPING If you map bitonic sorting to some new parallel computer, then you want to know how good that is in terms of both: Computation time Communication time You would also want to compare this mapping to other mappings of bitonic sorting How would you do that??

9. I N THE I DEAL W ORLD You would have all the parallel computers you want code bitonic sorting on each computer do real experimentation (test with huge input sizes for real) and find results

10. I N THE R EAL W ORLD You don’t have the parallel computers you want No decent simulators for interconnection networks exist Or at least not accessible Go For Simulation!! GOOD LUCK You really need it here by the way ;-)

11. W HY IS IT A TROUBLE ? You end up writing a multi-threaded one thread  one PE You need to count for EVERYTHING many threads compete for one CPU You run out of memory, go to HD?? Don’t you even think about it!! Thread synchronization overshadows execution Result Limit your simulation to relatively small processor numbers What about your original goal???

12. C AN WE GO TO THE G RID ?? What if we can get the Grid to help us distribute these threads among different computers? Capital, but? Threads access each others memories, what would we do about this??

13. R EFERENCES [B] Batcher K., (1968), "Sorting Networks and their Applications", Spring Joint Computer Conference, AFIPS Proc., vol. 32, April 30 th - May 2 nd, pp. 307-314. [G]Grama A., Gupta A., Karypis G., and Kumar V. Introduction to Parallel Computing, Addison Wesley, 2003. [K] Knuth D., (1998), "The Art of Computer Programming: Volume 3 Sorting and Searching", Addison-Wesley Longman, USA, 2 nd Edition, pp.225-228.

14. T HANKS