1. GridICE monitoring for the EGEE infrastructure
Sergio Andreozzi
INFN-CNAF (Italy)
EGEE User Forum, CERN, Geneva, Switzerland, March 1-3, 2006

2. Outline GridICE Architecture overview
Presentation Layer from the VO perspective
EGEE User Forum, CERN, Geneva, Switzerland, March 1-3, 2006

3. VO Monitoring A VO wants to monitor a Grid in order to:
to observe the composition, state and features of available resources available to its users
to analyze their behavior and performance
to track user activity as regards computing/storage/network resource usage
to detect fault situations
In the context of Grid computing, two important categories of monitoring systems are:
Application monitoring
Infrastructure monitoring
We focus on
EGEE User Forum, CERN, Geneva, Switzerland, March 1-3, 2006

4. BalticGrid EELA BeGrid
Overview
GridICE:
a distributed monitoring tool for Grid systems
started in late 2002 (EU-DataTAG project)
is evolving in the context of EU-EGEE
100% open source
fully integrated with the LCG-2.x Middleware
Metering and publishing of data can be configured via LCG standard installation mechanisms
Self-configurable collection and presentation
Installed servers are monitoring Grid resources in the scope of:
EGEE EGEE-SWE RDIG EGEE-SEE Grid.it GILDA CMS ATLAS EUMedGrid EUChinaGRID
BalticGrid EELA BeGrid
EGEE User Forum, CERN, Geneva, Switzerland, March 1-3, 2006

5. Main Phases of the Monitoring Process
abstract the huge number of received events in order to enable the consumer to draw conclusions about the operation of the monitored system
Presenting
transmission of the events from the source to any interested parties
Distributing
Processing
sensors enquiring entities and encoding the measurements according to a schema
Generation
(e.g., fairly static as software and hardware configuration or dynamics as current processor load)
Dynamics: (e.g., fairly static as software and hardware configuration or dynamics as current processor load)
Timing: (e.g., periodic or on demand)
e.g., filtering according to some predefined criteria, or summarising a group of events
EGEE User Forum, CERN, Geneva, Switzerland, March 1-3, 2006

6. Architecture at a glance
GridICE
GridICE Server
Lemon or other fabric mon. tools
charts
HTML
XML
notification
data aggregation and abstraction
Grid Inf. Service
persistent storage
discovery
consumers
scheduler
Grid Discovery Service
Site – Administrative domain
Monitored Entity
Site Collector
local publisher
site consumer
site publisher
sensor
site persistent storage
EGEE User Forum, CERN, Geneva, Switzerland, March 1-3, 2006

7. Generation
In GridICE, available measurements are defined by the GLUE Schema extensions
Extensions:
available
fabric-level information
job monitoring
summary info for computing resources
network connectivity from a Grid viewpoint [4]
on-going work
WMS, file transfer, file access (open + I/O per file/dataset)
Scheduled on a periodic fashion
EGEE User Forum, CERN, Geneva, Switzerland, March 1-3, 2006

8. We focus on the following categories of users:
Presentation Layer
We focus on the following categories of
users:
VO manager
actual set of resources accessible to VO members
Grid operator
all resources under responsibility of a Grid Operator Center
Site administrator
site resources offered to a Grid
EGEE User Forum, CERN, Geneva, Switzerland, March 1-3, 2006

9. Multi-dimensional data
Same measurements, different aggregation dimensions:
Time
Management
User
VO:
“WallTime of all jobs submitted by my users”
“WallTime of all jobs submitted by my users in Feb06”
User, Time
“WallTime of all jobs submitted by my users in Feb06 at INFN-T1”
User, Time, Management
EGEE User Forum, CERN, Geneva, Switzerland, March 1-3, 2006

10. Presentation Layer /1
EGEE User Forum, CERN, Geneva, Switzerland, March 1-3, 2006

11. Presentation Layer /2
EGEE User Forum, CERN, Geneva, Switzerland, March 1-3, 2006

12. Presentation /3 UPCOMING CHARTS Computing resources Summary Info
EGEE User Forum, CERN, Geneva, Switzerland, March 1-3, 2006

13. Ongoing and Future Work
Short term (3 months):
New service-specific sensors (i.e., WMS, SRM)
Extend chart section
Integration with the FCR tool
Medium term (3-9 months):
Security and Privacy concerns
Dealing with heterogeneous publisher interfaces
Improve flexibility in selecting different grain for the aggregation dimensions
EGEE User Forum, CERN, Geneva, Switzerland, March 1-3, 2006

14. Conclusion
GridICE is a production service monitoring the LCG/EGEE infrastructure
The set of measurements is continuously extended in order to consider VO-specific needs
The metering and collection infrastructure is stable
Need to improve access to collected data (more charts, more flexibility in browsing through dimensions)
EGEE User Forum, CERN, Geneva, Switzerland, March 1-3, 2006

15. Dissemination: http://grid.infn.it/gridice
References
Dissemination:
[1] S. Andreozzi, N. De Bortoli, S. Fantinel, A. Ghiselli, G. L. Rubini, G. Tortone, M. C. Vistoli GridICE: a monitoring service for Grid systems, Future Generation Computer System 21 (2005) 559–571
[2] C. Aiftimiei, S. Andreozzi, G. Cuscela, N. De Bortoli, G. Donvito, S. Fantinel, E. Fattibene, G. Misurelli, A. Pierro, G.L. Rubini, G.Tortone. GridICE: Requirements, Architecture and Experience of a Monitoring Tool for Grid Systems. In Proceedings of the International Conference on Computing in High Energy and Nuclear Physics (CHEP2006), Mumbai, India February 2006.
[3] C. Aiftimiei, S. Andreozzi, G. Cuscela, N. De Bortoli, G. Donvito, S. Fantinel, E. Fattibene, G. Misurelli, A. Pierro, G.L. Rubini, G.Tortone. Flexible notification service for Grid monitoring events. In Proceedings of the International Conference on Computing in High Energy and Nuclear Physics (CHEP2006), Mumbai, India February 2006.
[4] S. Andreozzi, A. Ciuffoletti, A. Ghiselli, C. Vistoli. Monitoring the Connectivity of a Grid. In Proceedings of the 2nd International Workshop on Middleware for Grid Computing (MGC 2004) in conjunction with the 5th ACM/IFIP/USENIX International Middleware Conference, Toronto, Canada, October 2004.
EGEE User Forum, CERN, Geneva, Switzerland, March 1-3, 2006