FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America Information System (IS) Valeria Ardizzone INFN Catania First EELA Grid tutorial for users and system administrators Madrid, 20-24th February 2006

E-infrastructure shared between Europe and Latin America lcg-infosites & lcg-info

E-infrastructure shared between Europe and Latin America How to discover resources ? Once an user is logged into an User Interface (s)he is ready to take advantage of the Grid Power for his/her own application. But what are the available resources to accomplish his/her tasks? The answer to this question comes through the interactions with the Information System (IS). The Information System (IS) provides information about the LCG-2 Grid resources and their status.

E-infrastructure shared between Europe and Latin America How to discover resources (cont) The data published in the IS conforms to the GLUE (Grid Laboratory for a Uniform Environment) Schema. The GLUE Schema aims to define a common conceptual data model to be used for Grid resources. In LCG-2, the BDII (Berkeley DB Information Index), based on an updated version of the Monitoring and Discovery Service (MDS), was adopted as main provider of the Information Service. In gLite, R-GMA (Relational Grid Monitoring Architecture) is adopted as IS.

E-infrastructure shared between Europe and Latin America Monitoring and Discovery Service (1) Computing and storage resources at a site implement an entity called Information Provider, which generates the relevant information of the resource (e.g.: the used space in a SE). This information is published via an LDAP server by Grid Resource Information Servers, or GRISes.

E-infrastructure shared between Europe and Latin America In each site an element called the Site Grid Index Information Server (GIIS) collects all the information of the different GRISes and publishes it. This BDII queries the GIISes and acts as a cache, storing information about the Grid status in its database. Monitoring and Discovery Service (2)

E-infrastructure shared between Europe and Latin America Querying the BDII a user or a service has all the available information about the status of the grid resources. Moreover in order to get more up-to-date information it is possible to querying directly the GIISes or GRISes. Monitoring and Discovery Service (3)

E-infrastructure shared between Europe and Latin America How to query the IS? In order to query directly the IS elements two higher level tools are provided. lcg-infositeslcg-info These tools should be enough for most common user needs and will usually avoid the necessary of raw LDAP queries.

E-infrastructure shared between Europe and Latin America lcg-infosites The lcg-infosites command can be used as an easy way to retrieve information on Grid resources for the most use cases. USAGE: lcg-infosites --vo options -v --is

E-infrastructure shared between Europe and Latin America lcg-infosites options

E-infrastructure shared between Europe and Latin America lcg-info intro This command can be used to list either CEs or the SEs that satisfy a given set of conditions, and to print the values of a given set of attributes. The information is taken from the BDII specified by the LCG_GFAL_INFOSYS environment variable. The query syntax is like this: attr1 op1 valueN,... attrN opN valueN where attrN is an attribute name op is =, >= or <=, and the cuts are ANDed. The cuts are comma-separated and spaces are not allowed. After the upgrading of the new GLUE SCHEMA it’s not possible use the operator ‘>’ and ‘<‘ After the upgrading of the new GLUE SCHEMA it’s not possible use the operator ‘>’ and ‘<‘

E-infrastructure shared between Europe and Latin America USAGE lcg-info --list-ce [--bdii bdii] [--vo vo] [--sed] [--query query] [--attrs list] lcg-info --list-se [--bdii bdii] [--vo vo] [--sed] [--query query] [--attrs list] lcg-info --list-attrs lcg-info --help lcg-info usage

E-infrastructure shared between Europe and Latin America lcg-info options

E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, References LCG-2 User Guide Manual Series UserGuide.html

E-infrastructure shared between Europe and Latin America SECOND PART R-GMA

E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, Outline  Introduction to R-GMA and Grid Monitoring Architecture (GMA).  R-GMA within Testbed  R-GMA in depth: - Schema, Registry, Producer and Consumer - Query and Storage Types - R-GMA Browser  Security

E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, Introduction to R-GMA Relational Grid Monitoring Architecture (R-GMA) –Developed as part of the EuropeanDataGrid Project (EDG) –Now as part of the EGEE project. –Based the Grid Monitoring Architecture (GMA) from the Global Grid Forum (GGF). Uses a relational data model. –Data are viewed as tables. –Data structure defined by the columns. –Each entry is a row (tuple). –Queried using Structured Query Language (SQL).

E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, Grid Monitoring Architecture(GMA) PRODUCER CONSUMER REGISTRY Store location Lookup location Transfer Data The Producer stores its location (URL) in the Registry. The Consumer looks up producer URLs in the Registry. The Consumer contacts the Producer to get all the data or the Consumer can listen to the Producer for new data.

E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, R-GMA within Testbed

E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, R-GMA: Schema-Registry-Mediator VIRTUAL DATABASE TABLE 1, Colum defs TABLE 2, Colum defs TABLE 3, Colum defs TABLE 4, Colum defs SCHEMA TABLE 1,Producer P1 details TABLE 2,Producer P1 details TABLE 2,Producer P2 details TABLE 2,Producer P3 details TABLE 3,Producer P2 details TABLE 3,Producer P1 details TABLE 3,Producer P3 details REGISTRY MEDIATOR R-GMA Server MEDIATOR: a set of rules for deciding which data providers to contact for any given query. REGISTRY: It holds the details of all producers that are publishing to tables in the virtual database and it also holds the details of “continuous” consumers. SCHEMA : it holds the names and definitions of all of the tables in the virtual database, and their authorization rules.

E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, R-GMA: Producer-Consumer VIRTUAL DATABASE TABLE 1, Colum defs TABLE 2, Colum defs TABLE 3, Colum defs TABLE 4, Colum defs SCHEMA TABLE 1,Producer P1 details TABLE 2,Producer P1 details TABLE 2,Producer P2 details TABLE 2,Producer P3 details TABLE 3,Producer P2 details TABLE 3,Producer P1 details TABLE 3,Producer P3 details REGISTRY MEDIATOR P1 P2 P3 C1C2 SQL “INSERT” SQL “SELECT” Producers: are the data providers for the virtual database. Writing data into the virtual database is known as publishing, and data is always published in complete rows, known as tuples. There are three types of producer: Primary, Secondary and On-demand. Consumer: represents a single SQL SELECT query on the virtual database. The query is matched against the list of available producers in the Registry. The consumer service then selects the best set of producers to contact and sends the query directly to each of them, to obtain the answer tuples. R-GMA Server

E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, Producer Types Primary Producer Secondary Producer On-Demand Producer User Code Producer API Producer Service Tuple Storage C Control only Queries Tuples SELECT * Tuples P User Code Producer API Producer Service C Control only Queries Tuples Queries User Code Producer API Producer Service Tuple Storage C Control and inserted tuples Queries Tuples

E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, Query and Storage Types Continuous: as soon as new data becomes available it is broadcast to all interested parties. Latest: correspond to intuitive idea of current information. History: return time sequenced data. TABLE 1,Producer P1 details TABLE 2,Producer P1 details TABLE 2,Producer P2 details TABLE 2,Producer P3 details TABLE 3,Producer P2 details TABLE 3,Producer P1 details TABLE 3,Producer P3 details REGISTRY P1 Latest-store Continuous&History-store P1 LATEST RETENTION PERIOD (LRP) and HISTORY RETENTION PERIOD (RTP) allow producers to periodically purge old tuples, and to give a precise meaning to the “current state”. Tuple-store can be in Memory or Database

E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, Continuous Producer Servlet Registry Store location Lookup location Continuous Store table description Producer API SQL “CREATE TABLE” Result Set TableName Value 1Value 2 TableNameURLPredicate Schema TableNameColumn TableName Value 1Value 2 Insert TableName UKRALAlice Consumer ServletConsumer API SQL “SELECT” TableName Value 1Value 2 TableName Value 1Value 2 Query SQL “INSERT”

E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, History or Latest Producer Servlet Registry Store location Lookup location Query Store table description Producer API SQL “CREATE TABLE” Result Set TableName Value 1Value 2 TableNameURLPredicate Schema TableNameColumn TableName Value 1Value 2 Insert TableName UKRALAlice Consumer ServletConsumer API SQL “SELECT” TableName Value 1Value 2 TableName Value 1Value 2 Query SQL “INSERT”

E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators,

E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, Security: Requirements Consumer users: want to be able to run queries on IS of any VO they belong. Producer users: want to be able to publish data to the IS of one or more VOs. Site administrators: want to control who creates R-GMA services on their systems and which services are run. Virtual Organizations: want to control who can add virtual tables to their schema and who can remove them. R-GMA Consumer Producer Site Admin VO

E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, Security: Solution R-GMA Consumer Provider Site Admin VO Mutual Autentication: guaranteeing who is at each end of an exchange of messages. Encryption: using an encrypted transport protocol (HTTPS). Authorization: it must be mutual. It can be implicit or explict.

E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, More information R-GMA overview page. – R-GMA documentation in EGEE – R-GMA in GILDA –