Collective Communication Computer Science 320 Collective Communication

Several Collective Patterns broadcast flood scatter gather, all-gather reduce, all-reduce all-to-all scan barrier

Broadcast One process, the root, sends the same message to all processes world.broadcast(root, buffer) The operation starts after all processes have called it

Flood Combines a flood receive in every process with a flood send in one process; blocking and nonblocking versions world.floodReceive(destinationBuffer) world.floodSend(sourceBuffer)

Scatter Distributes buffers from a root process to the other processes world.scatter(root, sourceBufferArray, destinationBuffer)

Gather Pulls in buffers from the other processes to a root process; often used after a scatter to collect the results of a parallel computation world.gather(root, sourceBuffer, destinationBufferArray)

Reduce Uses a reduction operator to reduce many buffers into one buffer world.reduce(root, buffer, op)

Scan Performs element-wise reductions over a set of buffers world.scan(buffer, op)