gLite Information System overview Africa 3 2010 - Joint EUMEDGRID-Support/EPIKH School for Application Porting Elisa Ingrà – Consortium GARR Algiers, 4-16 July 2010

Outline Motivation / Introduction Background GLUE Schema LDAP MDS Architecture Monitoring Information system in practice lcg-infosites lcg-info

Introduction

The power Grid paradigm wind coal Electric Power Grid System Users can access electrical power coming from different (and heterogeneous) sources nuclear hydroelectric

The computing Grid approach System Users can access storage and computing resources coming from different (and heterogeneous) sources The resources shared within the Grid can be physical objects (CPUs, storage devices) or logical resources (computing queues, distributed file systems)

Information System What is? Why? How? System to collect information on the state of resources Why? To discover resources of the grid and their nature To have useful data in order to who is in charge of managing the workload to do it more efficiently. To check for health status of resources. How? Monitoring state of resources locally and publishing fresh data on the information system. Adopting a data model that MUST be well known to all components that want to access monitored information Using different approaches that we are going to investigate in the next slides

The Info. system scope Discover new added resources Make available updated information about the resources Monitor resources load and status How does it work? Collecting information from the resources and publishing fresh data periodically. Adopting a data model that MUST be well known to all Grid components.

How it is used and by whom? Users Retrieve information about resources: Where can I run my job? Where can I copy my files? Which software packages are available on a given CE? Site managers Publish information about the resources and services they provide Middleware WMS: matching job requirements and allocating the resources Monitoring Services: retrieving information about the status and availability of resources.

GLUE Schema LDAP MDS Architecture Background GLUE Schema LDAP MDS Architecture

GLUE Schema - Overview Stands for “Grid Laboratory Uniform Environment” Provides a standardized description of a Grid computing system It is not tied to any particular implementation The Grid components are represented as objects which have attributes and relations to other objects

GLUE Schema – Why we need it? The Grid is a paradigm of distributed computing that enables the coordination of resources and services not subject to centralized control. These resources are geographically dispersed, span multiple trust domains and are heterogeneous. Resources can be dynamically contributed by different owner institutions, so a precise and shared description of resources among information consumers and resource providers is necessary. This description should also be common to different Grid infrastructures in order to contribute to the interoperability among them.

GLUE Schema – Core entities

GLUE Schema – Site properties Site entity – Properties: Property Type Description UniqueID string Unique Identifier of the Site Name Human-readable name Short description of this site EmailContact The main email contact for the site. Syntax rule: "mailto:" followed by a list of email addresses separated by a comma UserSupportContact E-mail addresses of the support service SysAdminContact E-mail addresses of the system administrator SecurityContact E-mail addresses of the security manager Location Geographical location of this site (e.g., city, state, country) Latitude real32 The position of a place north or south of the equator measured from -90º to 90º, with positive values going north and negative values going south Longitude The position of a place east or west of Greenwich, England measured from -180º to 180º with positive values going east and negative values going west Web uri The URI identifying a web page with more information about this site Sponsor VO sponsoring the site; the syntax should allow the expression of the percentage of sponsorship OtherInfo This attribute is to be used to publish info that does not fit in any other attribute of the site entity

GLUE Schema – Service properties Service entity – Properties: Property Type Description UniqueID string Unique Identifier of this service Name Human-friendly name serviceType_t The service type Version Version of the service: <major version number>.<minor version number>.<patch version number> Endpoint uri Network endpoint for this service Status serviceStatus_t Status of the service. String enumeration: OK, Warning, Critical, Unknown, Other StatusInfo Textual explanation for the status of the service WSDL URI of the WSDL describing the service Semantics URL of detailed description StartTime dateTime_xs_t The timestamp related to last start time of this service Owner Owner of the service AccessControlBase.Rule ACL_t Authorization rule for this entity

GLUE Schema – Cluster entity

GLUE Schema – CE entity

LDAP - Overview Stands for “Lightweight Directory Access Protocol” It is a protocol that defines the method by which directory data is accessed Optimized for reading, browsing and searching information (‘write-once-read-many-times’ service) Data is represented as a hierarchy of objects (entities) forming a tree structure Data Information Tree (DIT)

LDAP – Data Information Tree Attribute Type String representation Distinguished Name (DN) Unique name (path) that unambiguously identifies a single entry dc=grid (root of the DIT) Attribute Type String domainComponent DC CountryName C OrganizationName O OrganizationUnitName OU CommonName CN c= Brazil c=Italy c=Spain o=INFN ou=Catania ou=Roma cn=Elisa Ingra cn=Elisa Ingra,ou=Catania,o=INFN,c=Italy,dc=grid

LDAP – Data Information Tree The tree hierarchy is described by textual files following the LDIF format (LDAP Data Interchange Files) dn: dc=example,dc=root dc: example description: My company objectClass: dcObject objectClass: organization o: Example, Inc. ## FIRST Level hierarchy - people # this is an ENTRY sequence and is preceded by a BLANK line dn: ou=people, dc=example,dc=root ou: people description: All people in organisation objectClass: organizationalUnit ## SECOND Level hierarchy - people entries dn: cn=Joe Schmo,ou=people,dc=example,dc=root objectclass: inetOrgPerson cn: Joe Schmo sn: Schmo uid: jschmo mail: joe@example.com ou: sales

LDAP – Referrals An individual LDAP server might not store the entire DIT Servers need to be linked together in order to form a distributed directory that contains the whole DIT All client requests start at the global directory LDAP 1 LDAP 1 server LDAP 2 server LDAP 3 server c=USA,dc=example o=IBM,c=USA,dc=example dc=example referral c= USA o=CISCO o=IBM ou=IT ou=Web referral

MDS gLite adopts the Globus Monitoring and Discovery Service (MDS) architecture as its Information System It implements the GLUE Schema using OpenLDAP Grid Resource Information Server (GRIS) LDAP server at Grid Elements level Local CEs and SEs run an Information Provider software, which collects info about the resource Grid Index Information Server (or site BDII) (GIIS) LDAP server at a site level each Grid site to collect information by all GRISes present in that site Berkeley Database Information Index (BDII) Top level database Store and publish data gathered from the site Uses LDAP as protocol and GLUE Schema as data model

MDS - Architecture Site level BDII Grid Index Information Server (GIIS)

MDS - Architecture BDII consists of 2 servers: one contains a ready-only database and the other a write-only database Every 2 minutes a cron-job runs a script and collects info from lower-level servers (GIIS) Once updated, the 2 servers change roles

LDAP browser showing BDII info

lcg-infosites In this section, two higher level tools will be presented: lcg-infosites and lcg-info. These tools should be enough for most common user needs and will usually avoid the necessity of raw LDAP queries. The lcg-infosites command is actually just a perl script wrapping a series of LDAP commands and was developed to allow users to retrieve information on Grid resources for the most common cases

Praticals: lcg-infosites USAGE: lcg-infosites --vo options -v --is -h/--help: help option --vo: VO name (mandatory) --is: it's possible to specify a not default Top BDII Some options: se: The names of the SEs supported by the user's VO ce: The information relative to number of CPUs, running jobs,etc. rb: Names of the Rbs available for each VO sitenames: Names of the LCG sites tag: The names of the tags relative to the software installed in site is printed together with the corresponding CE closeSE: The names of the CEs where the user's VO is allowed to run together with their corresponding closest SEs are provided

lcg-info The lcg-info command can be used to list either CEs or the SEs that satisfy a given set of conditions on their attributes, and to print, for each of them, the values of a given set of attributes. The information is taken from the BDII specified by the LCG GFAL INFOSYS environment variable or in the command line.

Praticals: lcg-info -h/--help: to see the help USAGE: lcg-info [--list-ce | --list-se] [--query ] [--attrs ] -h/--help: to see the help –list-attrs: print the list of the possible attributes --list-ce: lists the CEs which satisfy a query, or all the CEs if no query is given. --list-se: lists the SEs which satisfy a query, or all the SEs if no query is given. --bdii: allows to specify a BDII in the form <hostname>:<port>. If not given, the value of the environmental variable LCG_GFAL_INFOSYS is used. If that is not defined, the command returns an error. --vo: restricts the output to CEs or SEs where the given VO is authorized.

References gLite documentation LDAP Concepts & Overview http://glite.web.cern.ch/glite/documentation/ LDAP Concepts & Overview http://www.zytrax.com/books/ldap/ch2/ OpenLDAP community http://www.openldap.org GLUE Schema Specification version 1.3 Final http://forge.gridforum.org/sf/go/doc14185?nav=1 Lilongwe, EELA-2 Workshop, 13.11.2008

export LCG_GFAL_INFOSYS=wms-01.eumedgrid.eu:2170 HANDS ON Information System BDII (use eumed instead of gilda) If will be necessary enter: export LCG_GFAL_INFOSYS=wms-01.eumedgrid.eu:2170 https://grid.ct.infn.it/twiki/bin/view/GILDA/InformationSystems Lilongwe, EELA-2 Workshop, 13.11.2008