1. www.eu-eela.org E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA-026409 gLite Information System Pedro Rausch IF - UFRJ Ninth EELA Tutorial Bogotá, 06.03.2007

2. E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA-026409 2 Bogotá, Ninth EELA Tutorial, 06.03.2007 How to discover resource? Information System –Design of Information System –Adopted Information System LCG Information System –Collecting Information –BDII RGMA –Introduction to R-GMA –Grid Monitoring Architecture (GMA) –R-GMA –R-GMA: Schema - Registry - Mediator –R-GMA: Producer – Consumer GLUE SCHEMA Outline

3. E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA-026409 3 Bogotá, Ninth EELA Tutorial, 06.03.2007 How to discover resource? Once an user is logged into an User Interface (s)he is ready to take advantage of the Grid Power for his/her own application. But what are the available resources to accomplish his/her tasks? The answer to this question comes through the interactions with the Information System (IS).

4. E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA-026409 4 Bogotá, Ninth EELA Tutorial, 06.03.2007 What is? –System to collect information on the state of resources Why? –To discover resources of the grid and their nature –To have useful data in order to who is in charge of managing the workload to do it more efficiently. –To check for health status of resources. How? –Monitoring state of resources locally and publishing fresh data on the information system. –Adopting a data model that MUST be well known to all components that want to access monitored information –Using different approaches that we are going to investigate in the next slides Information System

5. E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA-026409 5 Bogotá, Ninth EELA Tutorial, 06.03.2007 About Measures –Measures SHOULD be sensitive to the aim the users want to achieve. –Measures SHOULD be enough accurate to be considered valid. –Rate of taking measures MUST be adequate to be used. About the gathering of Information –How and when collected info should be published? –Where should collected info be stored? –How long should this info be maintained in the storage? Querying the Information System –Where should queries be sent to have a response? –What syntax and protocols have to be adopted to make queries? –What is the adopted data model to describe resources? Security –Who is allowed to execute queries against the IS and what type of queries is he allowed to do? –Management of user rights and credentials. Designing of Information System

6. E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA-026409 6 Bogotá, Ninth EELA Tutorial, 06.03.2007 The BDII (Berkeley DB Information Index) –has been adopted in LCG middleware as the Information System provider. –It is an evolution of the Monitoring and Discovery Service (MDS) –It is based on Lightweight Directory Access Protocol (LDAP) servers. The Relational Grid Monitoring Architecture (R-GMA) –Is an implementation of the Grid Monitoring Architecture (GMA) standardized by the Global Grid Forum (GGF) –It is a relational implementation of the GMA –It is strongly Web Services Oriented –It uses standard SQL query syntax Adopted Information System

7. E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA-026409 7 Bogotá, Ninth EELA Tutorial, 06.03.2007 LCG Information System

8. E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA-026409 8 Bogotá, Ninth EELA Tutorial, 06.03.2007 LCG adopts a combination of solutions –Globus MDS  At the lowest level of the information system  To discover and monitor resources and publish information  Caching –BDII  At the highest level of the system  Because MDS had some troubles in terms of scalability  Used by the Resource Broker for the matchmaking process  Can be configured by each VO  Queries underlying systems periodically (30s - 2 minutes) LCG Information System

9. E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA-026409 9 Bogotá, Ninth EELA Tutorial, 06.03.2007 Gathering of information at different levels –Lower level: Grid Resource Information Server (GRIS)  Collects information on the state of a given resource  One GRIS on top of each resource  A set of scripts and sensors that try to extract useful info on the resource –Medium level: Grid Index Information Server (GIIS)  Collects information on resources of a given site  One GIIS for each site  Also called Site BDII –Higher level: BDII  Collects information on resources of a given VO  One BDII for each VO (suggested solution) Way of collecting info –Pull model (higher level servers periodically query lower level servers) –LDAP query model Collecting Information

10. E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA-026409 10 Bogotá, Ninth EELA Tutorial, 06.03.2007 The Berkeley Database Information Index (BDII) –Developed within the context of LCG project –Solves problems of instability of the MDS occurring when the number of sites grows too much –Stays on top of GIIS sites –One for each VO –Centralized system –Three levels of hierarchy –Accessed by the Workload Management System Way of working –One GRIS for each resource –One GIIS for each site collecting info from below GRIS systems –One BDII for a given VO collecting information from below GIIS systems –Two LDAP servers, one for write access and one for read access –Every two minutes a cron-job runs a script and collects info from a list of GIIS sites –The list of GIIS is placed in the configuration file of the BDII BDII

11. E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA-026409 11 Bogotá, Ninth EELA Tutorial, 06.03.2007

12. E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA-026409 12 Bogotá, Ninth EELA Tutorial, 06.03.2007 R-GMA

13. E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA-026409 13 Bogotá, Ninth EELA Tutorial, 06.03.2007 Introduction to R-GMA Relational Grid Monitoring Architecture (R-GMA) –Developed as part of the EuropeanDataGrid Project (EDG) –Now as part of the EGEE project. –Based the Grid Monitoring Architecture (GMA) from the Global Grid Forum (GGF). Uses a relational data model. –Data are viewed as tables. –Data structure defined by the columns. –Each entry is a row (tuple). –Queried using Structured Query Language (SQL).

14. E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA-026409 14 Bogotá, Ninth EELA Tutorial, 06.03.2007 Grid Monitoring Architecture (GMA) PRODUCER CONSUMER REGISTRY Store location Lookup location Transfer Data The Producer stores its location (URL) in the Registry. The Consumer looks up producer URLs in the Registry. The Consumer contacts the Producer to get all the data or the Consumer can listen to the Producer for new data.

15. E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA-026409 15 Bogotá, Ninth EELA Tutorial, 06.03.2007 R-GMA The Relational Grid Monitoring Architecture (R-GMA) –Adopts a database model with tables and relations between tables –Implements a virtual database –The user queries the R-GMA as he/she was querying to a classical database (SQL string) –Implements different type of queries The information –Produced and accessed locally to its site –Always new –Can be collected by an entity (secondary producer) to be accessed faster

16. E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA-026409 16 Bogotá, Ninth EELA Tutorial, 06.03.2007 R-GMA: Schema-Registry-Mediator MEDIATOR: a set of rules for deciding which data providers to contact for any given query. REGISTRY: It holds the details of all producers that are publishing to tables in the virtual database and it also holds the details of “continuous” consumers. SCHEMA : it holds the names and definitions of all of the tables in the virtual database, and their authorization rules.

17. E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA-026409 17 Bogotá, Ninth EELA Tutorial, 06.03.2007 R-GMA: Producer-Consumer P1 P2 C1 SQL “INSERT” SQL “SELECT” Producers: are the data providers for the virtual database. Writing data into the virtual database is known as publishing, and data is always published in complete rows, known as tuples. There are three types of producer: Primary, Secondary and On-demand. Consumer: represents a single SQL SELECT query on the virtual database. The query is matched against the list of available producers in the Registry. The consumer service then selects the best set of producers to contact and sends the query directly to each of them, to obtain the answer tuples. C2

18. E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA-026409 18 Bogotá, Ninth EELA Tutorial, 06.03.2007 GLUE Schema

19. E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA-026409 19 Bogotá, Ninth EELA Tutorial, 06.03.2007 Grid Laboratory Uniform Environment (GLUE) Schema –It is a data model to describe in a meaningful way information on grid resources (static and dynamic info) –As result of a collaboration between the EU-DataTAG and iVDGL projects –EGEE, NorduGrid, LCG and Grid3/OSG contributed to the definition of the schema XML Schema –Now, GLUE Schema is being mapped to an XML representation –http://infnforge.cnaf.infn.it/glueinfomodel/Spec/V12/R1http://infnforge.cnaf.infn.it/glueinfomodel/Spec/V12/R1 GLUE Schema

20. E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA-026409 20 Bogotá, Ninth EELA Tutorial, 06.03.2007 LCG-2 User Guide –https://edms.cern.ch/file/454439//LCG-2-UserGuide.htmlhttps://edms.cern.ch/file/454439//LCG-2-UserGuide.html R-GMA overview page. –http://www.r-gma.org/http://www.r-gma.org/ R-GMA in EGEE –http://hepunx.rl.ac.uk/egee/jra1-uk/http://hepunx.rl.ac.uk/egee/jra1-uk/ R-GMA Documenation –http://hepunx.rl.ac.uk/egee/jra1-uk/glite-r1/http://hepunx.rl.ac.uk/egee/jra1-uk/glite-r1/ GLUE Schema –http://infnforge.cnaf.infn.it/glueinfomodel/http://infnforge.cnaf.infn.it/glueinfomodel/ References

21. E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA-026409 21 Bogotá, Ninth EELA Tutorial, 06.03.2007 Questions …