Message Passing Computing 1 iCSC2015,Helvi Hartmann, FIAS Message Passing Computing Lecture 2 Message Passing Helvi Hartmann FIAS Inverted CERN School of Computing, February 2015

Message Passing Computing 2 iCSC2015,Helvi Hartmann, FIAS Cluster Computing parallel computing on systems with distributed memory

Message Passing Computing 3 iCSC2015,Helvi Hartmann, FIAS ZMQ, PVM, MPI, HPX, OpenMP Any questions?

Message Passing Computing 4 iCSC2015,Helvi Hartmann, FIAS What Methods of Message Passing do you know?

Message Passing Computing 5 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe

Message Passing Computing 6 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP

Message Passing Computing 7 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ

Message Passing Computing 8 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Data is send to specific sockets Message Passing PVM MPI

Message Passing Computing 9 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Data is send to specific sockets Message Passing PVM MPI Processes outsource work to subroutines Remote Procedure CallZMQ

Message Passing Computing 10 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Data is send to specific sockets Message Passing PVM MPI Processes outsource work to subroutines Remote Procedure CallZMQ

Message Passing Computing 11 iCSC2015,Helvi Hartmann, FIAS OpenMP

Message Passing Computing 12 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Data is send to specific sockets Message Passing PVM MPI Processes outsource work to subroutines Remote Procedure CallZMQ

Message Passing Computing 13 iCSC2015,Helvi Hartmann, FIAS Socket Programming

Message Passing Computing 14 iCSC2015,Helvi Hartmann, FIAS ZMQ automatically reconnects to peers queues messages at sender and receiver routes messages handles sockets errors does all I/O in background threads lock free techniques for communication

Message Passing Computing 15 iCSC2015,Helvi Hartmann, FIAS ZMQ – Socket Types Request/Reply blocking

Message Passing Computing 16 iCSC2015,Helvi Hartmann, FIAS ZMQ – Socket Types Push/Pull nonblocking

Message Passing Computing 17 iCSC2015,Helvi Hartmann, FIAS ZMQ – Socket Types Publish/Subscribe

Message Passing Computing 18 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Data is send to specific sockets Message Passing PVM MPI Processes outsource work to subroutines Remote Procedure CallZMQ

Message Passing Computing 19 iCSC2015,Helvi Hartmann, FIAS Message Passing

Message Passing Computing 20 iCSC2015,Helvi Hartmann, FIAS PVM Master Slave Process 1Process 0 Process 2 Process 0Process 1Process 2

Message Passing Computing 21 iCSC2015,Helvi Hartmann, FIAS MPI Processes node0 node1 Process 0Process 1 ressource manager

Message Passing Computing 22 iCSC2015,Helvi Hartmann, FIAS MPI - Blocking Processes send receive Processes send receive

Message Passing Computing 23 iCSC2015,Helvi Hartmann, FIAS MPI - Blocking Synchronous Send (MPI_Ssend)

Message Passing Computing 24 iCSC2015,Helvi Hartmann, FIAS MPI - Blocking Buffered Send (MPI_Bsend)

Message Passing Computing 25 iCSC2015,Helvi Hartmann, FIAS MPI - Blocking

Message Passing Computing 26 iCSC2015,Helvi Hartmann, FIAS MPI - Blocking Synchronous Send (MPI_Ssend) Buffered Send (MPI_Bsend)

Message Passing Computing 27 iCSC2015,Helvi Hartmann, FIAS MPI - Blocking Processes send receive Processes send receive function returns immediately

Message Passing Computing 28 iCSC2015,Helvi Hartmann, FIAS MPI - Nonblocking

Message Passing Computing 29 iCSC2015,Helvi Hartmann, FIAS MPI - Nonblocking

Message Passing Computing 30 iCSC2015,Helvi Hartmann, FIAS MPI – Collectives Broadcast

Message Passing Computing 31 iCSC2015,Helvi Hartmann, FIAS MPI – Collectives ScatterGather

Message Passing Computing 32 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Data is send to specific sockets Message Passing PVM MPI Processes outsource work to subroutines Remote Procedure CallZMQ

Message Passing Computing 33 iCSC2015,Helvi Hartmann, FIAS Futures ? locality 1 future locality 1 Time ? locality 2

Message Passing Computing 34 iCSC2015,Helvi Hartmann, FIAS Futures ? locality 1 2 Time ? locality 2 ? locality 3 ? locality 4 34

Message Passing Computing 35 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Data is send to specific sockets Message Passing PVM MPI Processes outsource work to subroutines Remote Procedure CallZMQ

Message Passing Computing 36 iCSC2015,Helvi Hartmann, FIAS

Message Passing Computing 37 iCSC2015,Helvi Hartmann, FIAS : far-far-more-advanced-than-any-software-weve-ever-created-bill-gates/ Llano html itter_globus_ html