1. Grid Event Management Using R-GMA Monitoring Framework
ISGC April 27-29, 2005
Min Tsai ASCC/CERN

2. Overview Introduction to R-GMA R-GMA Monitoring Framework
Event Management System

3. R-GMA Introduction (Slides from Steve Fisher)
Background
R-GMA developed in EDG
Deployed by LCG
Re-engineered for the EGEE project
Uniform data transport mechanism for:
Measurement
Resource information
Logging
Monitoring

4. R-GMA – one cloud A relational implementation of GMA (from GGF)
Producer
A relational implementation of GMA (from GGF)
Powerful data model and query language
All data modelled as tables
SQL can express complex queries in one expression
Creates impression that you have one RDBMS
R-GMA
Consumer

5. Example Deployment Server Schema Service Registry Site A Server
Site B
Server
Server
Producer
Service
Consumer
Service
Producer
Service
Consumer
Service
API
control
data
Key
API
Consumer
application
Producer
application
Site C
Site D

6. The Producer Service Producer created control data Key declare API
application
Producer
Service

7. The Producer Service Registry Service contacted to register producer
control
data
Key
Registry
Service
register producer
declare
API
Producer
application
Producer
Service

8. The Producer Service
Data transferred from the application to Producer Service
If there are no consumers then the data goes no further than the Producer Service
control
data
Key
Registry
Service
register producer
declare
API
Producer
application
Producer
Service
insert

9. Producers Secondary Producers (SP) Primary Producers (PP)
Initial source of data
Data published by user code stored by Producer Service
Secondary Producers (SP)
Used to republish data to:
co-locate information to speed up queries
reduce network traffic PP PP SP

10. The Consumer Service Query issued by user code
Consumer Service carries out all of the work on its behalf
control
data
Key
query
API
Consumer
application
Consumer
Service

11. The Consumer Service
Registry Service contacted to identify relevant producers
Mediator works out list of producers to answer query
control
data
Key
Registry
Service
Mediator
list of producers
query
query
API
Consumer
application
Consumer
Service

12. The Consumer Service Consumer Service contacts relevant producers
control
data
Key
Registry
Service
Mediator
list of producers
query
query
query
API
Consumer
application
Consumer
Service
Producer
Service
Producer
Service
Producer
Service
Producer
Service

13. The Consumer Service
Data transferred directly from Producer Services to the Consumer Service
control
data
Key
Registry
Service
Mediator
list of producers
query
query
query
API
Consumer
application
Consumer
Service
Producer
Service
Producer
Service
Producer
Service
tuples
Producer
Service
tuples

14. The Consumer Service Query types: Continuous queries History Queries
as soon as new data becomes available it is broadcast to all interested parties
History Queries
return time sequenced data
Latest Queries
correspond to intuitive idea of current information

15. Using R-GMA Three steps to Producing Three Steps to Consuming
Create Producer
Declare table
Insert data
Three Steps to Consuming
Create a Consumer with your query
Start it
Read data
More information
JRA1-UK web site:

16. R-GMA Monitoring Framework

17. Problems with Monitoring Tools
Inconsistent site configuration sources
Results in different site coverage
Difficult to correlate test results
Different site Identification methods
Search through many web pages
Time consuming especially with 100’s sites!

18. Unified Monitoring Site configurations are consistent
Only from GOCDB
Data is sent to single data transport (R-GMA)
Shared data schema
Shared client applications (UI, Alarm Sys.)
Data can be selectively collected to create custom view (CIC, ROC, site admin)
Application can share results and build dependencies
Application can more easily be distributed using R-GMA to aggregate the data

19. Unified Schema TestDef TestDefRelation Test are registered here
Provides information describing each test
testName, friendlyName, isVirtual
dataType, Unit
testHelp, testTitle
TestDefRelation
Defines logical test groups and organize tests
superTestName, subTestName
displayPriority, flavour

20. Unified Schema (cont.) TestData TestDataRelation
Holds data produced by test
testName
nodeName
summaryData
detailedData
Status
0:NA, 10:OK, 20:INFO, etc.
TestDataRelation
Defines test data dependencies
superTestName, superNodeName
subTestName, subNodeName

21. Monitoring Console

22. Event Management System (EMS)

23. Introduction to EMS
Goal of Operation Centers to maximize availability of Grid services
EMS will help facilitate this by providing:
immediate notification as faults and performance degradation is identified
Shared EMS services that enables
Correlation and analysis of historical events of disparate monitoring tools
Reduce redundant development of notification modules for Grid monitoring applications

24. EMS Prototype
Web Services event transport and command client developed as a proof of concept
Individual monitoring application send events directly to a centralized EMS Core Service.
Client applications can then query the EMS Core for new events
Events are stored in an XML format and filtered using XPath expressions

25. Migration to R-GMA Transport
Already fully deployed in LCG2
R-GMA benefits
No additional work required by applications using R-GMA Monitoring Framework
Built-in data archiving
Automatic timestamps
Flexible SQL queries
Planned features
Security
Fully redundant directory and schema service

26. Event Management System

27. EMS Java Client

28. Additional Details and Help Information

29. Sorting by Columns

30. What Lies Ahead Not ready for operations use yet Display
Filtering mechanism to customize event received
Display list of current active alarms
Event reduction
Based on test dependencies
Damping of flapping events
Notification
Audio
Communication
Broadcast comments for a particular event

31. Questions? R-GMA Monitoring Framework EMS Piotr Nyczyk Judit Novak
Piotr Nyczyk
Judit Novak
Andrey Kiryanov
Dave Kant
EMS
Antun Gusev
Doug Chen
Mark Ho
Jeng-Hsueh Wu