1. Tecnologia dei Servizi Grid e cloud computing - Lezione 9b 0 Lezione 9b - 16 Dicembre 2009 Il materiale didattico usato in questo corso è stato mutuato da quello utilizzato da Paolo Veronesi per il corso di Griglie Computazionali per la Laurea Specialistica in Informatica tenuto nellanno accademico 2008/09 presso lUniversità degli Studi di Ferrara. Paolo Veronesi paolo.veronesi@cnaf.infn.it paolo.veronesi@cnaf.infn.it, pveronesi@unife.it http://www.cnaf.infn.it/~pveronesi/unife/ Università degli Studi di Bari – Corso di Laurea Specialistica in Informatica Tecnologia dei Servizi Grid e cloud computing A.A. 2009/2010 Giorgio Pietro Maggi giorgio.maggi@ba.infn.it, http://www.ba.infn.it/~maggi giorgio.maggi@ba.infn.it

2. Tecnologia dei Servizi Grid e cloud computing - Lezione 9b 1 Todays focus: Information Services Security Cross-organizational users Trust nobody Authorized access only Security Cross-organizational users Trust nobody Authorized access only Information Services Registry Notification Logging/auditing Information Services Registry Notification Logging/auditing Execution Management Job description & submission Scheduling Resource provisioning Execution Management Job description & submission Scheduling Resource provisioning Data Services Common access facilities Efficient & reliable transport Replication services Data Services Common access facilities Efficient & reliable transport Replication services Self-Management Self-configuration Self-optimization Self-healing Self-Management Self-configuration Self-optimization Self-healing Resource Management Discovery Monitoring Control Resource Management Discovery Monitoring Control OGSA OGSA profiles Web services foundation DONE

3. Tecnologia dei Servizi Grid e cloud computing - Lezione 9b 2 Outline What is the Information System Data Model: the GLUE Schema Overview Core entities OpenLDAP server introduction LCG Information Service Architecture Top BDII and Site BDII Information upgrade process

4. Tecnologia dei Servizi Grid e cloud computing - Lezione 9b 3 GLUE Schema Structure Collection of resources owned by a sinle organisation. Contains info on the location, the administrator, web page and so on Site Description of deployed service ServiceStorageElement Set of heterogeneous resources. Contains info on shared directory Cluster 111 * * * Set of homogeneous resources. Contains the size of the set Sub-Cluster ComputingElement Contains details of hardware (features and performance) and software Host 1 * 1 JobVOviewStatePolicyInfo * *

5. Tecnologia dei Servizi Grid e cloud computing - Lezione 9b 4 BDII top-level BDII site-level BDII resource BDII Resource provider WMS WN UI FTS Queries Site - Based on ldap - Standardized information provider (GIP) - GLUE-1.3 schema - Top level Used with 230+ sites - Roughly 60 instances in EGEE 2 minutes Berkeley Database Information Index Information & Monitoring Services

6. Tecnologia dei Servizi Grid e cloud computing - Lezione 9b 5 BDII overview Every node (except UI and WNs) has a bdii service in order to publish its informations A node in every site collects all site BDIIs and publishes them using a site BDII; The top BDII collects all site BDIIs User can run a set of commands to query the top BDII.

7. Tecnologia dei Servizi Grid e cloud computing - Lezione 9b 6 Generic Information Provider The Generic Information Provider, GIP, is a highly configurable information provider that makes a separation between static and dynamic information. It can be used to produce any kind of information for use with LDAP based grid information systems.GIPLDAP An information provider in its simplest form is a script that prints an LDIF file to standard out. The LDIF file should conform to the schema used in the GIP.LDIFGIP The problem is with dynamic information. Usually there are only a few attributes that have to be found dynamically. The idea with the GIP is to use dynamic plug-ins to obtain these values and use a common framework for everything else.GIP

8. Tecnologia dei Servizi Grid e cloud computing - Lezione 9b 7 GIP configuration file The GIP configuration file contains all the configuration parameters for the GIP.GIP temp_dir The directory to use as the GIPs cache.GIPs plugin_dir The directory where to put links to the dynamic plugins. static_dir The directory where to put links to the static ldif files. provider_dir The directory where to put links to the information providers. freshness How long to use the cache before running the dynamic plugins again. cache_ttl How long the cache is valid. response How long the GIP will wait for dynamic plugins before continuing.GIP timeout Timeout for the dynamic plugins in seconds.

9. Tecnologia dei Servizi Grid e cloud computing - Lezione 9b 8 GIP 1. The GIP script reads in all the ldif files from the static_dir ;GIP 2. run all the providers in the plugin_dir. 3. It will then run any dynamic plug-ins found in the plugin_dir to obtain the dynamic values. 4. It will print the LDIF to standard out but use the any dynamic values found from the plugins.

10. Tecnologia dei Servizi Grid e cloud computing - Lezione 9b 9 R-GMA: New System ? Why a new protocol? Disadvantages of the old system: ¤ LDAP does not allow to query information from different entries ¤ MDS/BDII is not flexible enough to allow for dynamic publication of data from user applications Advantages of the new system: ¤ R-GMA is quite flexible and allows cross queries between different entries ¤ Anyone can introduce new information in the system in a very easy way ¤ It is quite dynamic with new Producers of information being notified by existing Consumers

11. Tecnologia dei Servizi Grid e cloud computing - Lezione 9b 10 R-GMA: Characteristics GMA (Grid Monitoring Architecture) From GGF (Global Grid Forum) Very simple; it does not define: Data model Data transfer mechanism Registry implementation R-GMA (Relational GMA): Relational implementation Powerful data model and query language All data modeled as tables SQL as query language. It can express most queries in one expression You have a Relational DB for each VO

12. Tecnologia dei Servizi Grid e cloud computing - Lezione 9b 11 R-GMA: Design REGISTRY M Box Data Base Data Base Data Base SITE Client Collector of Information Info ??? Info

13. Tecnologia dei Servizi Grid e cloud computing - Lezione 9b 12 R-GMA Architecture Producer Consumer Registry Store Location Lookup Location Transfer Data Producers: Register themselves with the Registry and describe the type and structure of the information they want to make available to Grid Consumers: Query the Registry to find out the information available and locate Producers which provide such information. They can connect directly the Producers Registry: General collector, its arrow line represents the main flow of data

14. Tecnologia dei Servizi Grid e cloud computing - Lezione 9b 13 R-GMA tools: Browser The user can retrieve the R-GMA information via the browser servlet http://lcgic01.grid pp.rl.ac.uk:8080/R- GMA/index.html It shows the schema, the registered producers and allows simple queries

15. Tecnologia dei Servizi Grid e cloud computing - Lezione 9b 14 References gLite doc http://glite.web.cern.ch/glite/documentation/default.asp gLite userGuide https://edms.cern.ch/file/722398//gLite-3-UserGuide.pdf, Section 5. https://edms.cern.ch/file/722398//gLite-3-UserGuide.pdf EGEE: The Information System https://twiki.cern.ch/twiki/bin/view/EGEE/InformationSystemOverview https://twiki.cern.ch/twiki/bin/view/EGEE/InformationSystemOverview Generic Information Provider https://twiki.cern.ch/twiki//bin/view/EGEE/GIPhttps://twiki.cern.ch/twiki//bin/view/EGEE/GIP Usage of Glue Schema v1.3 for WLCG Installed Capacity information: https://twiki.cern.ch/twiki/pub/LCG/WLCGCommonComputingReadinessChalleng es/WLCG_GlueSchemaUsage-1.8.pdf https://twiki.cern.ch/twiki/pub/LCG/WLCGCommonComputingReadinessChalleng es/WLCG_GlueSchemaUsage-1.8.pdf