Lavoisier : a way to integrate heteregeneous monitoring systems. 30/03/10 30/03/10 mardi 30 mars 2010 Lavoisier : a way to integrate heteregeneous monitoring systems. Cyril L’Orphelin IN2P3/CNRS Computing Centre, Lyon, France 1

Outlines - Motivations - Main benefits of using Lavoisier 30/03/10 Outlines - Motivations - Main benefits of using Lavoisier - Concrete example : the Operations Dashboard - Future developments HEPiX Spring 2010 Workshop

EGEE / LCG information providers 30/03/10 EGEE / LCG information providers CIC-DB - VO Information VO Contacts User Contacts SQL VO repository (static) HTTP GOC-DB - Site Information - Site Contacts - Downtimes SQL Site repository (static) GGUS - Create ticket - Update ticket - View ticket SOAP Ticketing System Monitoring Framework Nagios Notifications about problem on services AMQ SAM - Test results - Status per nodes X-SQL Job Submission Framework - GIIS status per site - Storage Information - Jobs Information GSTAT HTTP GIIS browser BDII LDAP Dynamic information per nodes , per sites ... LDAP Database for sites VOMS HTTP + Script - VOMS certificates. - VOMS users VO User repository (dynamic) HEPiX Spring 2010 Workshop 30/03/10 3

30/03/10 Motivations - Context Multiple heterogeneous sources and technologies : No common language No direct cross queries For a web application , it means lot of computations and treatment and a dependency to the technologies proposed The information might be : Not usable Not available Too long to be queried HEPiX Spring 2010 Workshop

http://grid.in2p3.fr/lavoisier Our solution : Lavoisier 30/03/10 30/03/10 Our solution : Lavoisier An answer developped at CC-IN2P3: the Lavoisier service A good mechanism to build unified views from heterogeneous data sources http://grid.in2p3.fr/lavoisier "Nothing is lost, nothing is created, all is transformed" Antoine Laurent Lavoisier (1743 - 1794) HEPiX Spring 2010 Workshop 30/03/10 5 5

30/03/10 Accesses to data HEPiX Spring 2010 Workshop

30/03/10 Separation of roles HEPiX Spring 2010 Workshop

30/03/10 Other main features HEPiX Spring 2010 Workshop

30/03/10 Web Console HEPiX Spring 2010 Workshop

The Operations dashboard 30/03/10 The Operations dashboard HEPiX Spring 2010 Workshop

The Synoptic view https://operations-portal.in2p3.fr/ 30/03/10 HEPiX Spring 2010 Workshop

30/03/10 Access to details HEPiX Spring 2010 Workshop

Future developments (Lavoisier 1.x) 30/03/10 Future developments (Lavoisier 1.x) RDBMS adaptor (on-going development) user see the entire database as a huge XML document structure described by Hibernate or Doctrine ORM mapping file query with XPath / response as XML but of course this huge XML document is never built XPath query is translated to SQL query only the result set is converted to XML data XQuery and XPath as alternatives to XSLT available through SOAP and REST operations available as adapters (=> cacheable processing result) 20/07/201830/03/10

Future developments (Lavoisier 2.0) 30/03/10 Future developments (Lavoisier 2.0) Refactoring of core engine based on JMS to increase flexibility for notifications within a Lavoisier instance (through in-memory messages) between Lavoisier instances (though OpenWire protocol) improve extensibility with adapter interfaces for… Data protocol connectors Data format transformers Cache content managers 4 hard-coded managers Cache refresh triggers 8 hard-coded triggers Cache refresh scheduler No scheduler (overload risk) Lavoisier 2.0 Lavoisier 1.x Adapters do both (less reusable) 20/07/201830/03/10

Lavoisier : Dashboard : Links http://grid.in2p3.fr/lavoisier/ 30/03/10 Links Lavoisier : http://grid.in2p3.fr/lavoisier/ Dashboard : https://operations-portal.in2p3.fr (you need to be registered in GOC DB) https://forge.in2p3.fr/projects/show/opsportaluser HEPiX Spring 2010 Workshop