Persistent Collective WG December 9, 2015

Basics Mirror regular nonblocking collective operations For each nonblocking MPI collective (including neighborhood collectives), add a persistent variant For every MPI_I, add MPI_ _init All parameters for the new functions will be identical to those for the corresponding nonblocking function All arguments “fixed” for subsequent uses Persistent collective calls cannot be matched with blocking or nonblocking collective calls

Init/Start The init function calls only perform initialization actions for their particular (collective) operation and do not start the communication needed to effect the operation Ex: MPI_Allreduce_init() – Produces a persistent request (not destroyed by completion) Works with MPI_Start (but NOT Startall) Only inactive requests can be started MPI_REQUEST_FREE can be used to free an inactive persistent collective request (similar to freeing persistent point-to-point requests) – persistent pt2pt allows but says not encouraged (cf, section 3.9) but NBC (section 5.12) says erroneous—which do we follow? We have to follow the path of NBC, since we are extending NBC… erroneous.

Ordering of Init and Start Inits are non-blocking collective calls and must be ordered (across the group of a communicator) Similarly, persistent collective operations must be started in the same order at all processes in the corresponding communicator Startall cannot be used with persistent non- blocking collectives due to arbitrary ordering in the Startall operation NOTE: The proposal is the same as we read last time, but we changed this based on feedback.

Example

Standardization Status Open ticket 466 – moved to Git issue #25 (done) Target: MPI-Next first standard release First reading – June, 2015 Got feedback from the Forum Second reading – March, 2016 (we weren’t quite ready this time… 2 week deadline passed) Init/Start semantics have been clarified since first reading

Prototyping underway OpenMPI-based Replicate the libNBC library approach to interfacing with OpenMPI with persistence Explore improving persistent performance of point to point communication as well

Future Work Orthogonalization of the standard desirable We will extend ticket to non-blocking I/O collective operations – Other areas TBD – Could be part of the current ticket(?)

Summary We want to get maximum performance when there are repetitive operations Evidence in the literature of efficacy Other approaches (e.g., with Info arguments) are possible too Persistent, nonblocking collective operations provides the path to applications raising performance and predictability when there is reuse

Straw polls / Questions Would you vote for this proposal as presented? Is March a 2 nd first reading? LATER… – Would you like to see the NBPC-I/O APIs in March as part of this revised ticket? – Or as a separate ticket? – Other parts of MPI we should apply persistent mode to?

QUESTIONS?