1. Author: Laurence Field (CERN)
Monitoring Overview
Author: Laurence Field (CERN)

2. Introduction There are many Monitoring Systems around
Most are end to end
They do not interoperate
This was highlighted in the LCG/OSG interoperations activity
Focus here more on LCG but applicable for other grids
What do we need to have seamless monitoring across grids?
Install one solution everywhere?
Which one?
Lets take a look at them all and look for similarities
This may lead us to a generic solution
What are we trying to achieve?

3. VO/Grid Specific Middleware
Hourglass Model
VO/Grid Specific Middleware
Security
File Transfer
Job Submission
Service Discovery
Monitoring
Site Specific Systems

4. Terminology Information, Monitoring and Accounting.
All essentially the same
Different properties
Information (Service Discovery)
Usually static
Latest query (pull model)
No history
Monitoring
Usually dynamic
Continuous query (push model)
Historically information available (history query)
Accounting
History
No data loss
It’s all Data!
“Data >> Information >> Knowledge” K. Peach CHEP 2004.

5. Globus MDS GIIS Site GIIS Cache Node GRIS Provider

6. GridIce GridIce Site GRIS Provider DB Node Lemon Lemon Server Sensor

7. R-GMA R-GMA Site Producer Servlet Cache Node R-GMA API Sensor

8. Apel R-GMA Site Producer Servlet Cron Node DB Script

9. Grid Cat Grid Cat GridCat Site GRAM SQL Scripts Cron

10. MonaLisa MonaLisa Site MonaLisa Service DB Node API Module

11. Abstract View All systems have a similar architecture Sensors
Obtain the raw data
Process this ready to be inserted into the system
Site Level Cache/Database
Site Level Transport
Moves data from the node with the sensor to the site cache
External transport.
Whatever happens once the data leaves the site
System wide schema
Central repository for visualisation

12. Interoperability Matrix
Sensors
Local
Transport
Site Cache
External
Schema
Repository
MDS
Information
Providers
LDAP
Memory
LDAP DB
R-GMA
Various
HTTP
SQL
MySQL
Apel
Custom
Scripts
GridICE
Lemon
Lemmon
Server
Postgresql
Grid Cat
Cron
SQL Lite
GRAM
GridCat
MonaLisa
Monitoring Module
???
SQL DBs
Java Objects

13. General System SQL DB at site stores all the data
Need a common schema to define what data should be there
Common interface to extract information
GSI enabled queries for pull
Register and notification system for push
Sensors
Must produce data in agreement with schema
Common sensors would reduce effort and errors
But common sensors are not compulsory
Site level transport
Completely hidden
Sites own system could be used
Could be easily interfaced to common sensors

14. External Query Interface
General System
External Query API
External Query Interface
Site Boundary
SQL
Node
Insert Interface
Insert API
Sensor

15. Current LCG System R-GMA Grid ICE MON GIN R-GMA Producer MySQL
GridICE GIIS
APEL Cron
Lemon Server SE CE
Lemon
Lemon
GIN
GIN
GIP
GIP
APEL Script
GRIS
GridFTP Mon
BDII
GRIS
BDII

16. Possible LCG System R-GMA Grid ICE BDII MON BDII R-GMA Producer MySQL
Lemon Server SE CE
Lemon
Lemon
R-GMA API
R-GMA API
Sensor
Sensor

17. Possible LCG System MON Interface SQL XXX Server SE CE XXX API XXX API
Sensor
Sensor

18. Summary Monitoring interoperability between grids is difficult
Even within a single grid there is a problem!
Don’t need another system but a consolidation of current systems
Common answer, just deploy one implementation
Is this the right answer, if so which one do we choose?
Most systems seem to be similar
Data from a sensor is stored in a sites cache (SQL DB)
Focus on the cache and the metrics we need
Sensors follow the schema
Generic sensors would save effort.
Agree on the interface.
Both for pull, a simple query.
And push, register and notify
Leverage experience and effort from all systems
Leave freedom to chose implementations