1. Collective Communication
Computer Science 320
Collective Communication

2. Several Collective Patterns
broadcast
flood
scatter
gather, all-gather
reduce, all-reduce
all-to-all
scan
barrier

3. Broadcast
One process, the root, sends the same message to all processes
world.broadcast(root, buffer)
The operation starts after all processes have called it

4. Flood
Combines a flood receive in every process with a flood send in one process; blocking and nonblocking versions
world.floodReceive(destinationBuffer)
world.floodSend(sourceBuffer)

5. Scatter Distributes buffers from a root process to the other processes
world.scatter(root, sourceBufferArray, destinationBuffer)

6. Gather
Pulls in buffers from the other processes to a root process; often used after a scatter to collect the results of a parallel computation
world.gather(root, sourceBuffer, destinationBufferArray)

7. Reduce
Uses a reduction operator to reduce many buffers into one buffer
world.reduce(root, buffer, op)

8. Scan Performs element-wise reductions over a set of buffers
world.scan(buffer, op)