1 M. Tudruj, J. Borkowski, D. Kopanski Inter-Application Control Through Global States Monitoring On a Grid Polish-Japanese Institute of Information Technology, Koszykowa 86, Warsaw, Poland

2  Arrows represent reliable, asynchronous communication channels P1 P2 P3 P4 S a state information S b control Processes can communicate with a number of Synchronizers. Synchronizers learn state information from processes and send back control information. synchronizers processes Monitoring global states

3 There is no global clock, no shared memory Synchronizer must be able to order properly incoming events to build Strongly Consistent Global States (SCGS) SCGS is a combination of process local states, one state from each process, such that the local states are pairwise concurrent. E.g. is a SCGS, is not. P1 P2 sync e1 e2 f1 f2 s1 t2 t1 m1m2 Monitoring consistent global states

4 Events must have timestamps to be able to order messages correctly. Logical vector clocks or real time intervals based on roughly synchronized local clocks can be used If process local clocks are synchronized with a known accuracy, then real time interval timestamps can be used to identify SCGS Strongly Consistent Global States

5 Computation activation and cancellation caused by predicate evaluation

6 A complete graphical programming environment for developing message passing applications designed at Parallel and Distributed Systems Laboratory of the SZTAKI Institute of Hungarian Academy of Sciences Application level specifies processes and their interconnections Process level defines control flow diagram of a process Text level is used to enter sequential C code into elements of a flow diagram GRADE system

7 standard message passing channels local state info transfer channels signal transfer channels GRADE extension – PS-GRADE State information monitoring

8 PS-GRADE - synchronizer

9 condition send signal reception of state variables PS-GRADE– synchronizer control flow window condition window

10 Start signal-sensitive region "watching- signal" End signal-sensitive region "endwatching- signal" Resume interrupted computations Cancel computations Send state End signal- insensitive region Start signal- insensitive region PS-GRADE– Process - control flow window

11 PS-GRADE– synchronizer hierarchy

12 The principles of Grid application control Control of Grid application by: Data control flow (similarly to P-GRADE Workflow implemented by SZTAKI), based on input and output files for cluster application Grid Synchronizer : Collects information (vector of state) about application state Detects SCGS or OGS Computes conditions Sends signals to the application

13 A Grid-level synchronizer inserted into a workflow graph A1 1 A2 A3 1 1 A A5 1 2 Synch A

14 A Grid-level synchronizer and an application GRID Synchronizer Application A2 Application A3 Application A4 Application A5

15 Grid PS-GRADE environment general structure

16 Simple workflow. A selected application starts executing after a set of selected applications is completed. Example: complicated scientific computations performed layer by layer in different computer networks. Organizing Grid-level program execution control The following co-operation schemes included into the proposed Grid environment will be discussed:

17 Alternative workflow. One of several applications is selected for execution depending on the results (state) of former applications. Example: one of two available program packages is run depending on computation results performed so far. Partial canceling of workflow: Applications that become superfluous from the point of view of the general purpose of computation are stopped. Example: The exhaustive parallel search on a Grid for the optimal solution in a solution space is stopped or restricted when the search provides a satisfactory solution.

18 Supporting workflow: A set of currently executed applications require activation of auxiliary applications, which will provide useful results. Example: In a coarse grain simulation of a system of moving objects a collision that appears, stimulates a change in the Grid application configuration. An application which models the collision in a detailed way (with a fine granularity of events) is activated. After detailed simulation of the collision the coarse grain of the simulation process is restored. Workflow coupling : A common (global) status of many applications is monitored and control directives are distributed to particular applications of needed. Applications compute parameters that are subject to mutual exchange. Some parameters in meta-level applications are updated with the use of results of some auxiliary computations

19 Example – A Grid TSP application structure

20 The TSP application Synch1 conditions: DataRequest, MinDist, FinishCond (left to right)

21 The TSP application B&B part: condition window Heuristic search Application structure search process application structure B&B part: communication diagram

22 The paper has presented how the synchronization-based parallel application control can be extended and ported onto the GRID level. With the use of the proposed method we can create an advanced control of many applications running in the GRID environment. Inter–application coordination between programs, which are executed on different GRID sites, is supported. We identify five types of Grid-level program execution control The presented example shows that the new programming environment provides convenient means for designing complicated Grid applications control. Being on the Grid, we can extend the time consuming parts of the applications and run it on any available clusters during the middle stage of the algorithm. The best results from the heuristic part of the application obtained in a shorter time than by the B&B computations can support faster finding of the exact solution. Conclusions