Application of Design Patterns to Geometric Decompositions V. Balaji, Thomas L. Clune, Robert W. Numrich and Brice T. Womack

Our prototype example A grid-based code –implements finite-difference operators over fields defined on a grid of coordinates representing physical space. Solution of these problems typically involve: decomposition of data distribution of data across a parallel architecture assignment of work to that data.

A typical Approach Partition the physical coordinate space into subdomains. subdomains are extended to include regions of overlap with neighbors. –i.e. - halo cells, ghost cells, or guard cells subdomains interact with each other in some manner so they know when to update their halo cells with fresh data from their neighbors. The size, number and distribution of subdomains needs to be balanced to keep each processor busy and allow efficient exchange of halo data

Gof Patterns Applied Here Our prototype uses the following OO design patterns to solve a geometric decomposition problem –Composite –Builder –Strategy –Mediator –Iterator –…

GoF Composite Pattern

Allows us to maintain a unified picture of subdomains whether we want to think of them in: –a global setting, the problem as a whole –a local setting, the problem as local pieces. Methods associated with the composite provide a common interface at each level of the decomposition and allow the software developer to move from one view of the problem to another in a consistent manner.

Composite Tree Structure Comp Map Shared Memory Node Group 1 P1 L1 L2 L3 P2 L4 L5 L6 P3 L7 L8 L9 Shared Memory Node Group 2 P4P1 L10 L11 L12 P5 L13 L14 L15 P6 L16 L17 L18 Shared Memory Node Group …N P1P… L…

Composite features Application code is written to expect a block of data from a leaf. It shouldn’t assume any specific : –index order –data layout –or decomposition Blocks of data are assigned to processors by the object map structure –migration of work is done by simply moving one or more leaves to a different location within the composite Pieces of work (leaves) can be adaptively refined(AMR splitting/combining of sub elements) by replacing a leaf with a composite containing the newly refined leaves or the reverse operation

GoF Builder Pattern By applying the Builder pattern, we can isolate the composite's creation steps to produce composites that are: –Optimal for a target compute architecture –Customized for a particular numerical algorithm –Adapted for a particular set of physical or geometric constraints. –Etc.

GoF Builder Pattern

GoF Builder with Strategies

GoF Strategy Pattern Using the Strategy Pattern, we can provide a set of standard ways to build our composite object maps –Users can select from a set of predefined build strategies. –Users can easily customize the object map decomposition to improve performance. –Allows easy testing of application performance using different object map configurations

GoF Composite with Mediators

Data exchange w/Mediators Each local leaf uses a Mediator that is configured by the builder to manage its communication with neighbors All halo regions are specified in the local leaf’s context, not the neighbors A local halo region can transparently map to multiple neighbors –the user’s code doesn’t see any difference between 1 neighbor or 10 neighbors Mediator’s can use different communication methods for neighbors that are in local memory and neighbors that are outside of local memory

GoF Composite Applied

Request, Require & Release Request – non-blocking notification asking for data from neighbors Require – blocking request for data; does not continue until fulfilled Release – non-blocking notification of completion of work on local data Depending on the specific build strategy used to create the composite, the Mediator’s Request, Require and Release methods may be implemented using: –MPI or MPI-2 –SHMEM –Co-Array Fortran –ARMCI, –or Threads By abstracting the parallel communication mechanism, the builder has considerable freedom in configuring each Mediator to efficiently map communication implementation to the specific target parallel architecture.

Gof Iterators In a typical grid-code, each processor works only on the set of subdomains that are located in its own local memory. However, we also want to enable the use of global task queues for software architectures that permit low-latency task-switching when tasks stall for memory. We therefore see a need to provide iterators that allow: –Local iteration through all subdomains within local memory independent of the specific composition or the specific strategy used to build the composition. –Global iteration over subdomains that do not reside in the local memory of the physical processor doing the work.

Conclusions We are still working out implementation details, but, we have prototype Java code that is performing shallow water model equations based on these Gof Design patterns We are finding that application of the GoF design patterns can add flexibility and reduce the complexity of managing geometric decomposition problems We intend to use Fortran 2003’s OO features to implement this design when compilers are ready