1. Message Passing Computing 1 iCSC2015,Helvi Hartmann, FIAS Message Passing Computing Lecture 2 Message Passing Helvi Hartmann FIAS Inverted CERN School of Computing, 23-24 February 2015

2. Message Passing Computing 2 iCSC2015,Helvi Hartmann, FIAS Cluster Computing parallel computing on systems with distributed memory

3. Message Passing Computing 3 iCSC2015,Helvi Hartmann, FIAS ZMQ, PVM, MPI, HPX, OpenMP Any questions?

4. Message Passing Computing 4 iCSC2015,Helvi Hartmann, FIAS What Methods of Message Passing do you know?

5. Message Passing Computing 5 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe

6. Message Passing Computing 6 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP

7. Message Passing Computing 7 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ

8. Message Passing Computing 8 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Data is send to specific sockets Message Passing PVM MPI

9. Message Passing Computing 9 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Data is send to specific sockets Message Passing PVM MPI Processes outsource work to subroutines Remote Procedure CallZMQ

10. Message Passing Computing 10 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Data is send to specific sockets Message Passing PVM MPI Processes outsource work to subroutines Remote Procedure CallZMQ

11. Message Passing Computing 11 iCSC2015,Helvi Hartmann, FIAS OpenMP

12. Message Passing Computing 12 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Data is send to specific sockets Message Passing PVM MPI Processes outsource work to subroutines Remote Procedure CallZMQ

13. Message Passing Computing 13 iCSC2015,Helvi Hartmann, FIAS Socket Programming

14. Message Passing Computing 14 iCSC2015,Helvi Hartmann, FIAS ZMQ automatically reconnects to peers queues messages at sender and receiver routes messages handles sockets errors does all I/O in background threads lock free techniques for communication

15. Message Passing Computing 15 iCSC2015,Helvi Hartmann, FIAS ZMQ – Socket Types Request/Reply blocking

16. Message Passing Computing 16 iCSC2015,Helvi Hartmann, FIAS ZMQ – Socket Types Push/Pull nonblocking

17. Message Passing Computing 17 iCSC2015,Helvi Hartmann, FIAS ZMQ – Socket Types Publish/Subscribe

18. Message Passing Computing 18 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Data is send to specific sockets Message Passing PVM MPI Processes outsource work to subroutines Remote Procedure CallZMQ

19. Message Passing Computing 19 iCSC2015,Helvi Hartmann, FIAS Message Passing

20. Message Passing Computing 20 iCSC2015,Helvi Hartmann, FIAS PVM Master Slave Process 1Process 0 Process 2 Process 0Process 1Process 2

21. Message Passing Computing 21 iCSC2015,Helvi Hartmann, FIAS MPI Processes node0 node1 Process 0Process 1 ressource manager

22. Message Passing Computing 22 iCSC2015,Helvi Hartmann, FIAS MPI - Blocking Processes send receive Processes send receive

23. Message Passing Computing 23 iCSC2015,Helvi Hartmann, FIAS MPI - Blocking Synchronous Send (MPI_Ssend)

24. Message Passing Computing 24 iCSC2015,Helvi Hartmann, FIAS MPI - Blocking Buffered Send (MPI_Bsend)

25. Message Passing Computing 25 iCSC2015,Helvi Hartmann, FIAS MPI - Blocking

26. Message Passing Computing 26 iCSC2015,Helvi Hartmann, FIAS MPI - Blocking Synchronous Send (MPI_Ssend) Buffered Send (MPI_Bsend)

27. Message Passing Computing 27 iCSC2015,Helvi Hartmann, FIAS MPI - Blocking Processes send receive Processes send receive function returns immediately

28. Message Passing Computing 28 iCSC2015,Helvi Hartmann, FIAS MPI - Nonblocking

29. Message Passing Computing 29 iCSC2015,Helvi Hartmann, FIAS MPI - Nonblocking

30. Message Passing Computing 30 iCSC2015,Helvi Hartmann, FIAS MPI – Collectives Broadcast

31. Message Passing Computing 31 iCSC2015,Helvi Hartmann, FIAS MPI – Collectives ScatterGather

32. Message Passing Computing 32 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Data is send to specific sockets Message Passing PVM MPI Processes outsource work to subroutines Remote Procedure CallZMQ

33. Message Passing Computing 33 iCSC2015,Helvi Hartmann, FIAS Futures ? locality 1 future locality 1 Time ? locality 2

34. Message Passing Computing 34 iCSC2015,Helvi Hartmann, FIAS Futures ? locality 1 2 Time ? locality 2 ? locality 3 ? locality 4 34

35. Message Passing Computing 35 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Data is send to specific sockets Message Passing PVM MPI Processes outsource work to subroutines Remote Procedure CallZMQ

36. Message Passing Computing 36 iCSC2015,Helvi Hartmann, FIAS

37. Message Passing Computing 37 iCSC2015,Helvi Hartmann, FIAS 05.11.2014: http://creatia2013.wordpress.com/2013/03/12/dna-is-like-a-computer-program-but- far-far-more-advanced-than-any-software-weve-ever-created-bill-gates/ http://www.lhc-facts.ch/index.php?page=datenverarbeitung http://www.computerhistory.org/fellowawards/hall/bios/Gene,Amdahl/ http://www.heise.de/newsticker/meldung/AMD-bringt-neuen-Notebook-Prozessor- Llano-1259681.html http://en.wikipedia.org/wiki/TBR1 http://www.adpic.de/lizenzfreie_bilder/Geld%20_und_%20Boerse/Euro/goldener_g itter_globus_253863.html