CE-Monitor Luigi Zangrando INFN-Padova Milan, IT-CZ JRA1 meeting, 19-20 July 2004 CE-Monitor Luigi Zangrando INFN-Padova www.eu-egee.org EGEE is a project funded by the European Union under contract IST-2003-508833

Monitor structure The CE-Monitor is the CE component responsible for monitoring CE characteristics and status Job monitoring Possibly something else… The Monitor architecture is organized in various components, including a web-services layer and a serie of sensors. Each sensor implements a specialized monitoring service (e.g. CE-Sensor).

Monitor architecture MONITOR WS Job monitoring Sensor CE monitoring Other Sensor Job Monitor Sensor

Monitor architecture The Monitor manages the web-services requests and the data coming from the sensors. A client (ISM, a user, …) can get monitoring information from the CE (about jobs, CE status, …) Via asynchronous notifications (after having subscribed for them) “Directly” querying the CE Monitor to get the last info provided by the sensor Subscription, notification, “direct” query operations are made via Web Services. They are implemented according to WS-BaseNotification specifications. Advantages: WSRF Specification asynchronous notifications. Use of topics to select the sensor info a user wishes to be notified of.

Monitor sensors The sensors are elements that can be plugged to the Monitor They are used to perform a specific monitoring task (job monitoring, CE monitoring, …). They communicate with the Monitor using a standard interface. AbstractSensor class to be extended by specific sensor implementation They inform the Monitor of every event they register, provinding it with all the relevant data. The Monitor will then notify all the interested subscribers; they are selected using the policies they stated in the subscription.

CE - Sensor The CE-Sensor is the sensor specialized in monitoring the CE resources. The CE-Sensor service gets the CE info (according to GlueSchema) and passes it to the Monitor At the moment it relies on the EDG WP4/LCG scripts. All configuration data (updating time, script URI, ...) are specified in a configuration file. CE status information right now encompassed in a single string in the message Which format is needed?

Job monitoring sensor It monitors the job status changes The Monitor informs it about the jobs to monitor It notifies the Monitor, which in turn notifies the clients according to specified policies at subscription time We are implementing the Job monitoring sensors for PBS and LSF jobs

Status First version of CE Monitor + CE sensor implemented Code ready to be committed in CVS Compliant with SCM Preliminary tests already performed Implemented test clients to send subscriptions and get notifications LSF and PBS Job monitoring sensors to be tested Is it possible to integrate them with LCG-2 based job submission or they are usable only when the new CE is implemented? How can I get the gridjobid-localjobid mapping?

Some next steps Stress tests Allow complex subscription policies E.g. many clients querying the CE and/or subscribed for notifications Allow complex subscription policies E.g. allow specifying notification rate (at the present time it is the same of the sensor) Integration of job monitoring sensors (possibly also with LCG-2 submission). Allow on-the-fly plugging of sensors. Implemention of WS-Addressing specifications