1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski *, M. Tudruj * x * Polish-Japanese Institute of Information Technology, 86 Koszykowa Str., Warsaw, Poland x Institute of Computer Science, Polish Academy of Sciences 21 Ordona Str Warsaw, Poland {janb, damian, GRID applications control based on synchronizers

2 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 Introduction The principles of application control Implementation issues Conclusions Contents

3 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 Monitoring global states Strongly Consistent Global States (SCGS) Observed Global States (OGS) Activation and cancellation P-GRADE system PS-GRADE system Introduction

4 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 Arrows represent reliable, asynchronous communication channels P1 P2 P3 P4 S a state information S b control Processes can communicate with a number of Synchronizers. Synchronizers learn state information from processes and send back control information. synchronizers processes Monitoring global states

5 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 There is no global clock, no shared memory Synchronizer must be able to order properly incoming events to build Strongly Consistent Global States (SCGS) SCGS is a combination of process local states, one state from each process, such that the local states are pairwise concurrent. E.g. is a SCGS, is not. P1 P2 sync e1 e2 f1 f2 s1 t2 t1 m1m2 Monitoring consistent global states

6 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 Events must have timestamps to be able to order messages correctly. Logical vector clocks or real time intervals based on roughly synchronized local clocks can be used If process local clocks are synchronized with a known accuracy, then real time interval timestamps can be used to identify SCGS Strongly Consistent Global States

7 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 Observed states P1 P2 sync e1 e2 f1 f2 - f1 e1 f1 e1 f2 e2 f2 We dont need a clock synchronization We dont need QoS (Quality Of Service) The reaction is fired immediately

8 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 Computation activation and cancellation caused by predicate evaluation

9 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 A complete graphical programming environment for developing message passing applications designed at Parallel and Distributed Systems Laboratory of the SZTAKI Institute of Hungarian Academy of Sciences Application level specifies processes and their interconnections Process level defines control flow diagram of a process Text level is used to enter sequential C code into elements of a flow diagram GRADE system

10 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 standard message passing channels local state info transfer channels signal transfer channels GRADE extension – state information acquisition

11 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 GRADE extension - synchronizer

12 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 GRADE extension – synchronizer Condition Window

13 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 condition send signal reception of state variables GRADE extension – synchronizer control flow window

14 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 Start signal-sensitive region "watching- signal" End signal-sensitive region "endwatching- signal" Resume interrupted computations Cancel computations Send state End signal- insensitive region Start signal- insensitive region GRADE extension – Process - control flow window

15 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 GRADE extension – synchronizer hierarchy

16 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 The principles of application control Control of GRID application by: Data control flow (similarly to P-GRADE Workflow implemented by SZTAKI), based on input and output files for cluster application GRID Synchronizer : Collects information (vector of state) about application state Detects SCGS or OGS Computes conditions Sends signals to the application

17 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 A GRID-level synchronizer inserted into a workflow graph A1 1 A2 A3 1 1 A A5 1 2 Synch A

18 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 A GRID-level synchronizer and an application (example) GRID Synchronizer Application A2 Application A3 Application A4 Application A5

19 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 Implementation concepts GT3 core architecture and PS-GRADE service We use the Globus Toolkit v3 (GT3) to implement web service infrastructure User Defined Service layer will be used for signal and message delivery = Grade-Globus-Web-Service (GGWS) and for the application maintenance service on the GRID= Grade-Globus-Maintenance-Service (GGMS) Inter-grid communication with SOAP protocol We also use : GridFTP services for input, output files and program code transfer GRAM serviced for local job managment

20 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 Grade-Globus-Maintenance-Service GGMS will be the central maintenance service of GRID- Grade application. Main function of GGMS : Checks the start conditions for applications and if they are true, initiates execution of application Makes the GRID execution map with information about current running applications and sends it to the GGSW (if GGSW need it) Makes the global (model) clock available to all GGSW GGCSW use : GridFTP services for input, output files and program code transfer GRAM services for job managment SOAP for communication with GGSW

21 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 GGMS – model of execution Start = execute GGCSW on selected GLOBUS Server LOAD Data = All executions, input files, execution Map to the selected GLOBUS Server with the use of GridFTP Starts the Global Synchronizers Main LOOP - Until all applications complete Checks all starting conditions If any fulfilled then Run the ready Grid applications Updates the GRID execution map Sends the execution map to the GGSW (if GGSW need it) Checks the state of all running application If any application completes then Gets the output files Updates the GRID execution map Sends the execution map to the GGSW Maintains the time synchronisation of all running GGSW Stops the Global Synchronizers Waits for output data request

22 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 GGMS – an application starting pattern Makes a new instance of GGSW on selected GLOBUS server Sends the execution file and input file to the GLOBUS server throught GridFtp service Executes program by the use of the relevant GRAM service in interactive mode or in the batch mode Creates then communication interface between the GGSW and the started application

23 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 Globus Resource Allocation Manager – GRAM service GRAM simplifies the use of remote systems by providing a single standard interface for requesting and using remote system resources for the execution of jobs. The most common use of GRAM is remote job submission and control. Grid Service Factory Pattern Create Service Service instance is created The request is validated Users job request is *ready* to be started Start operation Users job request is started Service instance monitors job request Updates request SDE ( Service Data Element) Job control Ensures client received a handle to the job before resources have been consumed GGMS and GRAM

24 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 The clock synchronisation If an application uses the Strongly Consistent Global States we have to implement a clock synchronisation mechanism We will use the GGMSs clock as a reference clock After clock synchronisation call, all the GGWSs measure the difference between their local clocks and the referenced clock When the local synchronizer sends a state message to the global synchronizer, GGWS does the timestamp correction by adding to the timestamp the value of time difference versus the reference clock.

25 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 Grade-Globus-Web-Service (GGWS) Main functions of GGSW Implements communication between local synchronizers and the global synchronizer (with SOAP Messaging) Corrects timestamps attached to state messages(in the SCGS mode) Maintains the GRID CENTER time-synchronisation process Monitors running application

26 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 Inter GRID Communication concepts

27 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 The paper has presented how the synchronization-based parallel application control can be extended and ported onto the GRID level. With the use of the proposed method we can create an advanced control of many applications running in the GRID environment. Inter–application coordination between programs, which are executed on different GRID sites, is supported. We have employed the Globus Toolkit as the required middleware implementation platform. Full implementation of the described extension of the PS-GRADE system will be done in co-operation with P-GRADE authors i.e. SZTAKI Institute of Hungarian Academy of Sciences Conclusions