E-infrastructure shared between Europe and Latin America 12th EELA Tutorial for Users and System Administrators gLite Information System Yubiryn Ramírez ULA - Mérida 12th EELA Tutorial Users Lima,

E-infrastructure shared between Europe and Latin America 2 Lima, 12th EELA Tutorial, This presentation is based on previous gLite presentations

E-infrastructure shared between Europe and Latin America 3 Lima, 12th EELA Tutorial, How to discover resource? Grid Information System Glue Schema gLite LCG Information System –Hierarchy –LDAP model RGMA –Overview –Server Architecture –Consumer-Producer model –RGMA Concepts References Outline

E-infrastructure shared between Europe and Latin America 4 Lima, 12th EELA Tutorial, How to discover resource? Once an user is logged into an User Interface (s)he is ready to take advantage of the Grid Power for his/her own application. But what are the available resources to accomplish his/her tasks? The answer to this question comes through the interactions with the Information System (IS).

E-infrastructure shared between Europe and Latin America 5 Lima, 12th EELA Tutorial, What is? –System to collect information on the state of resources Why? –To discover resources of the grid and their nature –To have useful data in order to who is in charge of managing the workload to do it more efficiently. –To check for health status of resources. How? –Monitoring state of resources locally and publishing fresh data on the information system. –Adopting a data model that MUST be well known to all components that want to access monitored information Grid Information System Overview

E-infrastructure shared between Europe and Latin America 6 Lima, 12th EELA Tutorial, About Measures –Measures SHOULD be sensitive to the aim the users want to achieve. –Measures SHOULD be enough accurate to be considered valid. –Rate of taking measures MUST be adequate to be used. About the gathering of Information –How and when collected info should be published? –Where should collected info be stored? –How long should this info be maintained in the storage? Querying the Information System –Where should queries be sent to have a response? –What syntax and protocols have to be adopted to make queries? –What is the adopted data model to describe resources? Security –Who is allowed to execute queries against the IS and what type of queries is he allowed to do? –Management of user rights and credentials. Designing of Information System

E-infrastructure shared between Europe and Latin America 7 Lima, 12th EELA Tutorial, Basic data model. –Grid Laboratory Uniform Environment (GLUE) Schema. Two architectures in glite3 –LCG Information System  BDII over Globus MDS.  OpenLDAP interface.  Preserves LCG-2 compatibility. –Relational Grid Monitoring Architecture (R-GMA)  Based on the GMA standard from the Grid Global Forum  Information in SQL relational databases  Web Services.  Future replacement of LCG IS. gLite Information System

E-infrastructure shared between Europe and Latin America 8 Lima, 12th EELA Tutorial, GLUE Schema

E-infrastructure shared between Europe and Latin America 9 Lima, 12th EELA Tutorial, A schema of objects and attributes describing Grid resources and its relationships. –Originally a EU-DataTAG and US-iVDGL coordinated effort. –Current participants: EGEE, OSG, Globus and NorduGrid. –A way to describe Grid info  Statically and dynamically supplied  Hierarchically represented  Independently of the framework (LDAP, XML, SQL…) Actual release (1.2) is mapped into –LDAP –XML –Old ClassAd (used by Condor Matchmaking Glue Schema

E-infrastructure shared between Europe and Latin America 10 Lima, 12th EELA Tutorial, Site Element

E-infrastructure shared between Europe and Latin America 11 Lima, 12th EELA Tutorial, Computing Element

E-infrastructure shared between Europe and Latin America 12 Lima, 12th EELA Tutorial, LCG Information System

E-infrastructure shared between Europe and Latin America 13 Lima, 12th EELA Tutorial, Resource level: Grid Resource Information Server (GRIS) –One GRIS on top of each CE, SE, RB, MyProxy (no WNs). –Sensors and scripts get status of concrete resources statically (e.g. GlueCEUniqueID) or dynamically (e.g. GlueCEStateWaitingJobs) LCG Information System Levels

E-infrastructure shared between Europe and Latin America 14 Lima, 12th EELA Tutorial, Site level: Grid Index Information Server (GIIS) –Compiles all the information of the different GRISes in a site. –LCG-2 recommends using a BDII instead of a GIIS  Improves robustness and stability.  Called the site BDII LCG Information System Levels

E-infrastructure shared between Europe and Latin America 15 Lima, 12th EELA Tutorial, Top level: Berkeley DB Information Index (BDII) –Keeps all Grid information about the VOs (generally only one). –Stores information from local BDIIs or GIISes in its database. –Only queries sites that are included in a configuration file. LCG Information System Levels

E-infrastructure shared between Europe and Latin America 16 Lima, 12th EELA Tutorial, LCG Information System Hierarchy

E-infrastructure shared between Europe and Latin America 17 Lima, 12th EELA Tutorial, Way of collecting info –Pull model (higher level servers periodically query lower level servers) –All servers are based on LDAP  Inherit hierarchical structure (tree-like)  LDAP Data Information Format (LDIF) Users get info with –Generic applications  ldapsearch (MDS: 2135 and BDII:2170 ports)  Graphical (BDII web; LDAP GUIs)  Always can get information about specific resources (maybe more up-to-date) by querying directly the site BDIIs, GIISes or GRISes. –Querying VO info with lcg-infosites or lcg-info LDAP Model

E-infrastructure shared between Europe and Latin America 18 Lima, 12th EELA Tutorial, LDAP Browser show BDII information

E-infrastructure shared between Europe and Latin America 19 Lima, 12th EELA Tutorial, R-GMA

E-infrastructure shared between Europe and Latin America 20 Lima, 12th EELA Tutorial, Introduction to R-GMA Relational Grid Monitoring Architecture (R-GMA) –Developed as part of the EuropeanDataGrid Project (EDG) –Now as part of the EGEE project. –Based the Grid Monitoring Architecture (GMA) standart from the Global Grid Forum (GGF).

E-infrastructure shared between Europe and Latin America 21 Lima, 12th EELA Tutorial, R-GMA Overview Information in SQL relational databases (a DB per VO) Query syntax is a SQL subset Simple consumer-producer model Web Services oriented CLI and Web user interface Allows self-logging applications Future replacement of LCG IS

E-infrastructure shared between Europe and Latin America 22 Lima, 12th EELA Tutorial, R-GMA Overview R-GMA offers a global view of the VO information –In one large relational DB: virtual database. –Registry stores localization tuples (database rows) published by producers:  Standard Tables: CE state in GLUE Schema (by R-GMA-GIN)  Applications specific tables (e.g. self-logging with Log4j)  Access by SQL queries through a WS interface. –Always updated  Consumers get information directly from the resource  Can be cached by a secondary producer from single o multiple producers to improve access.

E-infrastructure shared between Europe and Latin America 23 Lima, 12th EELA Tutorial, R-GMA Server Architecture

E-infrastructure shared between Europe and Latin America 24 Lima, 12th EELA Tutorial, R-GMA Server Description R-GMA Servers –Server (one or several) –Schema Server (There is one common global schema server for the grid) –Registry Server (one or several) –Browser (shows grid information via web browser)

E-infrastructure shared between Europe and Latin America 25 Lima, 12th EELA Tutorial, R-GMA Server Description R-GMA Clients –Generic Client (generic set of APIs for different languages and CLIs) –Servicetool (used by all services to publish its existence and status) –Site Publisher  Client to publish the existence of a site.  Each site must have one R-GMA site publisher. –GadgetIN (GIN)  Client to extract information from MDS and to republish it to R-GMA.  used by the CE. –Data archiver (flexible archiver)  Client to make the data that is coming from the R-GMA site-publisher, servicetool and GIN constantly available.  By default the GLUE tables and service tables are archived, however this can be configured.

E-infrastructure shared between Europe and Latin America 26 Lima, 12th EELA Tutorial, R-GMA Server per Node Type –The gLite R-GMA Server should deployed the first as part of a gLite grid, because all services require it to publish service information. Deployment moduleContainsUsed / included by R-GMA ServerR-GMA server R-GMA registry server R-GMA schema server R-GMA browser R-GMA site publisher R-GMA archiver R-GMA servicetool MON-box R-GMA clientRGMA client APIsUI, WN, SD R-GMA servicetool R-GMA server,VOMS, LB, WMS, CE, DC, FTS, FTA, Hydra, I/O- Server,Torque Server R-GMA GINR-GMA GadgetINCE

E-infrastructure shared between Europe and Latin America 27 Lima, 12th EELA Tutorial, RGMA Producer – Consumer model PRODUCERREGISTRY Store location Lookup location Transfer Data The Producer stores its location (URL) in the Registry. The Consumer looks up producer URLs in the Registry. The Consumer contacts the Producer to get all the data or the Consumer can listen to the Producer for new data. CONSUMER

E-infrastructure shared between Europe and Latin America 28 Lima, 12th EELA Tutorial, RGMA Multiple Producers The Consumer will get all the URLs that could satisfy the query. The Consumer will connect to all the Producers. Producers that can satisfy the query will send the tuples to the Consumer. The Consumer will merge these tuples to form one result set. Consumer Producer 1 Registry TableName Value 1Value2 Value 3Value 4 TableName Value 1Value 2 TableNameURL 1 TableNameURL 2 Producer 2 TableName Value 3Value 4

E-infrastructure shared between Europe and Latin America 29 Lima, 12th EELA Tutorial, Select * from CPULoad CPULoad (Producer 3) CHCERNATLAS CHCERNCDF CPULoad (Producer 1) UKRALCDF UKRALATLAS CPULoad (Producer 2) UKGLACDF UKGLAALICE CPULoad (Consumer) CountrySiteFacilityLoadTimestamp UKRALCDF UKRALATLAS UKGLACDF UKGLAALICE CHCERNALICE CHCERNCDF

E-infrastructure shared between Europe and Latin America 30 Lima, 12th EELA Tutorial, Joins Service URIVOtype Contactsite ServiceStatus URIVOtypeupstatus gppse01aliceSEySE is running gppse01atlasSEySE is running gppse02cmsSEnSE ERROR 101 lxshare0404aliceSEySE is running lxshare0404atlasSEySE is running Result Set (Consumer) URI Contact SELECT Service.URI Service. Contact FROM Service S, ServiceStatus SS WHERE (S.URI= SS.URI and SS.up=‘n’)

E-infrastructure shared between Europe and Latin America 31 Lima, 12th EELA Tutorial, R-GMA Concepts Producers –Primary producer The producer creates and stores locally the informatiion –Secondary producer The producer stores locally the information coming from primary producers –On Demand The producer returns the information only under the request of the consumer Consumer –Continuous As soon as new data becomes available it is broadcast to all interested parties –Latest Corresponds to the intuitive idea of “current information” –History –Returns time sequenced data

E-infrastructure shared between Europe and Latin America 32 Lima, 12th EELA Tutorial, LCG-2 User Guide – R-GMA overview page. – R-GMA in EGEE – R-GMA Documenation – GLUE Schema – References

E-infrastructure shared between Europe and Latin America 33 Lima, 12th EELA Tutorial, Questions …