GRM + Mercury in P-GRADE Monitoring of P-GRADE applications in the Grid using GRM and Mercury

Target ● Collect trace about a ● P-GRADE application ● running on one cluster but ● Migrating among several clusters. ● P-GRADE uses Grid Resource Broker to submit the application (E.g. Condor)

Tools ● GRM – Instrumentation library to generate trace – Prove visualisation tool – Main process to gather traces from Mercury, maintain trace file and communicate with Prove ● Mercury Monitor – Monitoring service on each cluster – Receive trace records from the application and – Deliver them to GRM (the user's machine)

Structure ● GRM – Instrumentation library in application processes – Main process and Prove on user's machine ● Mercury Monitor – Local Monitor as service on each machine of each cluster – Main Monitor on the frontend node of a cluster (connecting to all LM's on that cluster)

GRM and Mercury GRM Application Process Cluster 1 User's Host Host 1Host 2Host 1 Local Monitor LM Cluster 2 Local Monitor LM Main Monitor MM Appl. Process PROVE P-GRADE

Issues of monitoring ● Start-up – P-GRADE submits the job. – Resource Broker places the job somewhere. – Currently we do not use any Information Service to find out, where the job is running (in DataGrid we use R-GMA for this purpose). – Therefore, GRM should subscribe for trace at each Mercury main monitor (i.e. connect all clusters at the beginning which is not scalable for large grids).

Issues of monitoring ● Collecting trace – Mercury stops the process if it generates trace and there is no subscription for it. – Therefore, there is NO timing problem, i.e., GRM can start in advance or after the application start. – Mercury currently supports the push model therefore, GRM (and PROVE) receives trace as stream (opposite to the pull model of the original GRM in P- GRADE). – I.e. On-line monitoring now, not semi-on-line

Monitoring the migration ● Goal: In the visualisation the same process line is continued for the migrated process. A colored line shows the time of migration. ● In P-GRADE the processes has their own ID, which is used in their communication as well as in GRM instrumentation. This ID remains the same in the migrated process. ● The migration function generates special trace records for the start and the end of the migration. ● Thus, the generated trace satisfies our goal.

Monitoring the migration (cont.) ● Mercury monitor needs – The migrating process should close the connection to the local monitor – The migrated process should connect as new process to the local monitor (on a different machine, on different cluster). ● Mercury monitor – Takes care of delivering data received in the new connection to the original subscriptor (GRM) – Note: GRM is subscribed at each cluster at the beginning.