1. High Performance Communication using MPJ Express 1 Presented by Jawad Manzoor National University of Sciences and Technology, Pakistan 29 June 2015

2. Presentation Outline 2 Introduction Parallel computing HPC Platforms Software programming models MPJ Express Design Communication devices Performance Evaluation 29 June 2015

3. Serial vs Parallel Computing 3 Serial Computing Parallel Computing 29 June 2015

4. HPC Platforms There are three kind of High performance computing (HPC) platforms. ▫Distributed Memory Architecture  Massively Parallel Processor (MPP) ▫Shared Memory Architecture  Symmetric Multi processor (SMP), Multicore computers ▫Hybrid Architecture  SMP Clusters Most of the modern HPC hardware is based on hybrid models Distributed Memory Shared Memory Hybrid 4 29 June 2015

5. Software Programming Models Shared Memory Models ▫Process has direct access to all memory ▫Pthreads, OpenMP Distributed Memory Models ▫No direct access to memory of other processes ▫Message Passing Interface (MPI) Process 5 29 June 2015

6. Message Passing Interface (MPI) Message Passing Interface is the defacto standard for writing applications on parallel hardware Primarily designed for distributed memory machines but it is also used for shared memory machines 6 29 June 2015

7. MPI Implementations OpenMPI It is an open source production quality implementation of MPI- 2 in C Existing high performance drivers TCP/IP, Shared memory, Myrinet, Quadrics, Infiniband MPICH 2 It is the implementation of MPI on SMPs, clusters, and massively parallel processors POSIX shared memory, SysV shared memory, Windows shared memory, Myrinet, Quadrics, Infiniband, 10 Gigabit Ethernet MPJ Express Implements the high level functionality of MPI in pure Java Provides flexibility to update the layers or add new communication devices TCP/IP, Myrinet, Threads shared memory, SysV shared memory 7 29 June 2015

8. Presentation Outline 8 Introduction Parallel computing HPC Platforms Software programming models MPJ Express Design Communication devices Performance Evaluation 8 29 June 2015

9. 9

10. Java NIO Device Uses non-blocking I/O functionality, Implements two communication protocols: ▫Eager-send  For small messages (< 128 Kbytes),  May incur additional copying. ▫Rendezvous:  Exchange of control messages before the actual transmission,  For long messages (  128 Kbytes). 10 29 June 2015

11. Standard mode with eager send protocol (small messages) 11 29 June 2015

12. Standard mode with rendezvous protocol (large messages) 12 29 June 2015

13. Threads based MPJ process is represented by a Java thread and data is communicated using shared data structures. ▫sendQueue and recvQueue SysV based MPJ process is represented by a Unix process and data is communicated using shared data structures. ▫Java Module -The xdev API implementation for shared memory communication ▫C Module - Unix SysV Inter Process Communication methods ▫JNI Module – Bridge between C and Java. 13 Shared Memory Communication Device 29 June 2015

14. 14 MPI communication using sockets MPI communication using shared memory 29 June 2015

15. Key Implementation aspects Critical operations include: ▫ Initialize ▫ Point to point  Send  Receive ▫ Finalize 15 29 June 2015

16. 16 Process 0’s shared memory segment Process 1’s shared memory segment Process 2’s shared memory segment Process 3’s shared memory segment Initialization 29 June 2015

17. 17 Point-to-point communication  Communication between two processes.  Source process sends message to destination process.  Source and destination processes are identified by their rank 29 June 2015

18. 18 Blocking Send Only return from sub routine call when the operation has completed Non Blocking Send Return straight away and allow sub program to continue to perform other work. At some time later check for the completion of the process Send Modes 29 June 2015

19. Sending a message 19  Memory space of each process is divided into sub-sections equal to the number of processes.  Each subsection is used for communication with one process. 29 June 2015

20. Receiving a message 20  Destination process attaches itself to the shared memory segment of source process and starts reading messages from the sub- section allocated to it using offset 29 June 2015

21. Finalization 21  When the communication is completed, barrier method is called at the end which synchronizes all process.  Then the finalize method is called which destroys the shared memory allocated to the processes. 29 June 2015

22. Presentation Outline 22 Introduction Parallel computing HPC Platforms Software programming models Design and Implementation Design Communication devices Performance Evaluation 22 29 June 2015

23. Performance Evaluation A ping pong program was written in which two processes repeatedly pass a message back and forth. ▫ Timing calls to measure the time taken for one message. ▫ We used a warm up loop of 10K iterations and the average time was calculated for 20K iterations after warm up. We present latency and throughput graphs ▫ Latency is the delay between the initiation of a network transmission by a sender and the receipt of that transmission by a receiver ▫ Throughput is the amount of data that passes through a network connection over time as measured in bits per second. We have plotted the latency graph from message size of 1 byte up to 2KB and bandwidth graph from 2KB to 16MB 23 29 June 2015

24. Latency on Fast Ethernet 24 29 June 2015

25. Throughput on Fast Ethernet 25 29 June 2015

26. Latency on Gigabit Ethernet 26 29 June 2015

27. Throughput on GigE 27 29 June 2015

28. Latency on Myrinet 28 29 June 2015

29. Throughput on Myrinet 29 29 June 2015

30. Q ? 30 29 June 2015

31. Further Reading Parallel Computing https://computing.llnl.gov/tutorials/parallel_comp/ MPI www.mcs.anl.gov/mpi MPJ Express http://mpj-express.org/ MPICH2 http://www.mcs.anl.gov/research/projects/mpich2/ OpenMPI http://www.open-mpi.org/ 29 June 2015 31