1. MPI and RDMA Yufei 10/15/2010

3. MPI over uDAPL: abstract MPI: most popular parallel computing standard MPI needs the ability to deliver high performace Implement MPI with both portability and portable high performance using uDAPL. Two important techs: – Improve the startup time and communication performance – Multi-stream MPI design

4. MPI over uDAPL: background InfiniBand Vitual Interface Architecture iWARP uDAPL MPI MPI applicationapplication

5. MPI over uDAPL: Efficient Connection Establishment Multi-threads vs. Non-thread n processes need establish n*(n-1)/2 connections. If each connection needs a thread … – Thread creation and switching overhead – Portable problem of thread How to synchronize each process and step?

6. MPI over uDAPL: Multi-Stream MPI Design Deal with out-of-order messages: sequence number

7. MPI over uDAPL: Memory Management Should all the EPs share the same set of RDMA buffers or should they have separate buffers? But how could different streams share the buffers?

8. MPI over uDAPL: Connection Selection Policy

9. MPI over uDAPL: Result: threadless connection establishment

10. MPI over uDAPL: Result: bandwidth over infiniband on 10Gb

12. RDMA-Based MPI: Introduction InfiniBand Architecture – Channel semantics (communication) – Memory semantics (RDMA READ/WRITE) MPI is the standard for writing parallel apps Key techs – Persistent buffer association – RDMA polling set

13. RDMA-Based MPI: Send/Receive vs. RDMA Write Latency shows RDMA Writer is better than Send/Receive?

14. RDMA-Based MPI: Use RDMA Write instead of Send/Receive RDMA Write designed to implement Control Message – The RDMA destination address must be known before communication Persistent buffer association – The receiver side must detect the arrival of incoming message RDMA polling set

15. RDMA-Based MPI: Persistent buffer association

16. RDMA-Based MPI: RDMA polling set But how to poll the RDMA Polling Set (or Memory Region)

17. RDMA-Based MPI: RDMA Buffer Structure Waste CPU or Waste Time?

18. Q & A