Information System in gLite Valeria Ardizzone Istituto Nazionale di Fisica Nucleare Corso introduttivo al Grid Computing Catania, 03.04.2008

Outline What is the Information System Data Model: the GLUE Schema Overview Core entities OpenLDAP server introduction LCG Information Service Architecture Top BDII and Site BDII Information upgrade process Grid Computing Course, Physics Dept, Univ. of Catania

Information System What is? Why? How? System to collect information on the state of resources Why? To discover resources of the grid and their nature To have useful data that helps who is in charge of managing the workload to do it more efficiently. To check for health status of resources. How? Monitoring state of resources locally and publishing right information on the information system. Adopting a data model that MUST be well known to all components that want to access monitored information Using different approaches that we are going to investigate in next slides Grid Computing Course, Physics Dept, Univ. of Catania

Design of Information Systems About Measures Measures SHOULD be sensitive to the aim the users want to achieve. Measures SHOULD be enough accurate to be considered valid. Rate of taking measures MUST be adequate to be used. About the gathering of Information How and when collected info should be published? Where should collected info be stored? How long should this info be maintained in the storage? Querying the Information System Where should queries be sent to have a response? What syntax and protocols have to be adopted to make queries? What is the adopted data model to describe resources? Security Who is allowed to execute queries against the IS and what type of queries is he allowed to do? Management of user rights and credentials. Grid Computing Course, Physics Dept, Univ. of Catania

Adopted Information Systems The BDII (Berkley DB Information Index) has been adopted in LCG middleware as the Information System provider. It is an evolution of the Globus Meta Directory System (MDS) LCG-2 actually adopts BDII as Information System. It is based on Lightweight Directory Access Protocol (LDAP) servers. The Relational Grid Monitoring Architecture (R-GMA) Is an implementation of the Grid Monitoring Architecture (GMA) standardized by the Global Grid Forum (GGF) It is a relational implementation of the GMA It is strongly Web Services Oriented It will be adopted by next releases of the gLite middleware Grid Computing Course, Physics Dept, Univ. of Catania

The Data Model: GLUE Schema Grid Computing Course, Physics Dept, Univ. of Catania

GLUE: overview GLUE: Grid Laboratory Uniform Environment It’s an information model that describe all those resources that partecipate in the Grid system and that are requested to be discoverable and monitored The same information can be retrieved from different GISs relying on different technology (e.g. R-GMA) Grid Computing Course, Physics Dept, Univ. of Catania

Site Element Grid Computing Course, Physics Dept, Univ. of Catania 8

GLUE: site GlueSiteUniqueID: TRIGRID-INFN-CATANIA GlueSiteName: TRIGRID-INFN-CATANIA GlueSiteDescription: LCG Site GlueSiteUserSupportContact: mailto: grid-prod@ct.infn.it GlueSiteSysAdminContact: mailto: grid-prod@ct.infn.it GlueSiteSecurityContact: mailto: grid-prod@ct.infn.it GlueSiteLocation: Catania, Italy GlueSiteLatitude: 37.54866 GlueSiteLongitude: 15.036076 GlueSiteWeb: http://www.trigrid.it GlueSiteOtherInfo: TIER 1 GlueSiteOtherInfo: Trigrid Team Grid Computing Course, Physics Dept, Univ. of Catania

GLUE: service GlueServiceUniqueID: infn-rb-01.ct.trigrid.it:7772 GlueServiceName: INFN-CATANIA-rb GlueServiceType: ResourceBroker GlueServiceVersion: 1.2.0 GlueServiceEndpoint: infn-rb-01.ct.trigrid.it:7772 GlueServiceURI: unset GlueServiceAccessPointURL: not_used GlueServiceStatus: OK GlueServiceStatusInfo: No Problems GlueServiceWSDL: unset GlueServiceSemantics: unset GlueServiceStartTime: 1970-01-01T00:00:00Z GlueServiceOwner: trigrid GlueServiceOwner: cometa GlueServiceOwner: inaf GlueServiceOwner: alice GlueServiceAccessControlRule: trigrid GlueServiceAccessControlRule: cometa GlueServiceAccessControlRule: inaf GlueServiceAccessControlRule: alice GlueForeignKey: GlueSiteUniqueID=INFN-CATANIA Grid Computing Course, Physics Dept, Univ. of Catania

Cluster Element Grid Computing Course, Physics Dept, Univ. of Catania 11

GLUE: cluster and subcluster GlueClusterName: infn-ce-01.ct.trigrid.it GlueClusterService: infn-ce-01.ct.trigrid.it:2119/jobmanager-lcglsf-short GlueClusterService: infn-ce-01.ct.trigrid.it:2119/jobmanager-lcglsf-long GlueClusterService: infn-ce-01.ct.trigrid.it:2119/jobmanager-lcglsf-infinite GlueClusterService: infn-ce-01.ct.trigrid.it:2119/jobmanager-lcglsf-cert GlueClusterService: infn-ce-01.ct.trigrid.it:2119/jobmanager-lcglsf-cometa GlueClusterService: infn-ce-01.ct.trigrid.it:2119/jobmanager-lcglsf-inaf GlueClusterService: infn-ce-01.ct.trigrid.it:2119/jobmanager-lcglsf-alice [..] GlueSubClusterPhysicalCPUs: 4 GlueSubClusterLogicalCPUs: 4 GlueSubClusterTmpDir: /tmp GlueSubClusterWNTmpDir: /tmp Grid Computing Course, Physics Dept, Univ. of Catania

GLUE: Host GlueHostApplicationSoftwareRunTimeEnvironment: GLITE-3_0_0 GlueHostApplicationSoftwareRunTimeEnvironment: INFN-CATANIA GlueHostApplicationSoftwareRunTimeEnvironment: MPICH [..] GlueHostArchitectureSMPSize: 4 GlueHostBenchmarkSF00: 1937 GlueHostBenchmarkSI00: 1483 GlueHostMainMemoryRAMSize: 4096 GlueHostMainMemoryVirtualSize: 8192 GlueHostNetworkAdapterInboundIP: TRUE GlueHostNetworkAdapterOutboundIP: TRUE GlueHostOperatingSystemName: Scientific Linux CERN GlueHostOperatingSystemRelease: 3.0.6 GlueHostOperatingSystemVersion: SLC GlueHostProcessorClockSpeed: 2392 GlueHostProcessorModel: Dual Core Opteron 280 GlueHostProcessorVendor: AMD Grid Computing Course, Physics Dept, Univ. of Catania

Computing Element Grid Computing Course, Physics Dept, Univ. of Catania 14

GLUE: Host Grid Computing Course, Physics Dept, Univ. of Catania GlueCEName: cometa GlueCEUniqueID: infn-ce-01.ct.trigrid.it:2119/jobmanager-lcglsf-cometa GlueCEInfoGatekeeperPort: 2119 GlueCEInfoHostName: infn-ce-01.ct.trigrid.it GlueCEInfoLRMSType: lsf GlueCEInfoLRMSVersion: 6.1 GlueCEInfoTotalCPUs: 98 GlueCEInfoJobManager: lcglsf GlueCEInfoContactString: infn-ce-01.ct.trigrid.it:2119/jobmanager-lcglsf-cometa GlueCEInfoApplicationDir: /opt/exp_soft GlueCEInfoDataDir: unset GlueCEInfoDefaultSE: infn-se-01.ct.trigrid.it GlueCEStateEstimatedResponseTime: 61713 GlueCEStateFreeCPUs: 26 GlueCEStateRunningJobs: 70 GlueCEStateStatus: Production GlueCEStateTotalJobs: 70 GlueCEStateWaitingJobs: 0 GlueCEStateWorstResponseTime: 123427 GlueCEStateFreeJobSlots: 26 GlueCEPolicyMaxCPUTime: 2880 GlueCEPolicyMaxRunningJobs: 98 GlueCEPolicyMaxTotalJobs: 0 GlueCEPolicyMaxWallClockTime: 2880 GlueCEPolicyPriority: -10 GlueCEPolicyAssignedJobSlots: 98 GlueCEAccessControlBaseRule: VO:cometa Grid Computing Course, Physics Dept, Univ. of Catania

Computing Element Storage Element Storage Area Access protocols Information about the service (like Name,Port,URL) Storage Area Contains info of available and used disk space,file policies, access rules,etc. Access protocols Contains info about the protocols used to transfer files Grid Computing Course, Physics Dept, Univ. of Catania 16

GLUE: Storage Element GlueSEUniqueID: infn-se-01.ct.trigrid.it GlueSEName: TRIGRID-INFN-CATANIA:srm_v1 GlueSEPort: 2811 GlueSESizeTotal: 16350 GlueSESizeFree: 16350 GlueSEArchitecture: multidisk GlueInformationServiceURL: ldap://infn-se-01.ct.trigrid.it:2135/mds-vo-name=lo cal,o=grid Grid Computing Course, Physics Dept, Univ. of Catania

GLUE: Storage Area GlueSARoot: cometa:/dpm/ct.trigrid.it/home/cometa GlueSAPath: /dpm/ct.trigrid.it/home/cometa GlueSAType: permanent GlueSALocalID: cometa GlueSAPolicyMaxFileSize: 10000 GlueSAPolicyMinFileSize: 1 GlueSAPolicyMaxData: 100 GlueSAPolicyMaxNumFiles: 10 GlueSAPolicyMaxPinDuration: 10 GlueSAPolicyQuota: 0 GlueSAPolicyFileLifeTime: permanent GlueSAStateAvailableSpace: 16350000000 GlueSAStateUsedSpace: 0 GlueSAAccessControlBaseRule: cometa Grid Computing Course, Physics Dept, Univ. of Catania

GLUE: Access Protocols GlueSEAccessProtocolLocalID: gsiftp GlueSEAccessProtocolType: gsiftp GlueSEAccessProtocolEndpoint: gsiftp://infn-se-01.ct.trigrid.it GlueSEAccessProtocolCapability: file transfer GlueSEAccessProtocolVersion: 1.0.0 GlueSEAccessProtocolPort: 2811 GlueSEAccessProtocolSupportedSecurity: GSI GlueSEAccessProtocolLocalID: rfio GlueSEAccessProtocolType: rfio GlueSEAccessProtocolEndpoint: httpg://infn-se-01.ct.trigrid.it GlueSEAccessProtocolCapability: byte access GlueSEAccessProtocolPort: 5001 GlueSEAccessProtocolSupportedSecurity: RFIO Grid Computing Course, Physics Dept, Univ. of Catania

LCG Information System Grid Computing Course, Physics Dept, Univ. of Catania

LCG Information System LCG adopts a combination of solutions Globus MDS At the lowest level of the information system To discover and monitor resources and publish information Grid Information Security (GSI) credentials Caching BDII At the highest level of the system Because MDS had some troubles in terms of scalability Used by the Resource Broker for the matchmaking process Can be configured by each VO Queries underlying systems periodically (2 minutes) Hierarchical system Information is collected on the leaves of a hierarchical tree and travels towards the root Clients can query the hierarchical tree at every level The higher the level against which queries are made, the older is the obtained information Grid Computing Course, Physics Dept, Univ. of Catania

Collecting Information Gathering of information at different levels Lower level: Grid Resource Information Server (GRIS) Collects information on the state of a given resource One GRIS on top of each resource A set of scripts and sensor that try to extract useful info on the resource Medium level: Grid Index Information Server (GIIS) Collects information on resources of a given site One GIIS for each site Higher level: BDII Collects information on resources of a given VO One BDII for each VO (suggested solution) Way of collecting info Pull model (higher level servers periodically query lower level servers) LDAP query model Grid Computing Course, Physics Dept, Univ. of Catania

BDII overview The Berkley Database Information Index (BDII) Developed within the context of LCG project Solves problems of instability of the MDS occurring when the number of sites grows too much Stays on top of GIIS sites One for each VO Centralized system Three levels of hierarchy Accessed by the Workload Management System Way of working One GRIS for each resource One GIIS for each site collecting info from below GRIS systems One BDII for a given VO collecting information from below GIIS systems Two LDAP servers, one for write access and one for read access Every two minutes a cron-job runs a script and collects info from a list of GIIS sites The list of GIIS is placed in the configuration file of the BDII Grid Computing Course, Physics Dept, Univ. of Catania

LCG Information System Hierarchy Top BDII RB Site BDII A SE GRIS CE GRIS Site A Site BDII C SE GRIS CE GRIS Site C Site BDII B SE GRIS CE GRIS Site B Grid Computing Course, Physics Dept, Univ. of Catania

BDII overview Every node (except UI,BDII and Wns) has a GRIS service in order to publish its informations A node in every site collects all site GRISes and publishes them using a site BDII, by default installed on the CE The top BDII collects all site BDIIs User can run a set of commands to query the top BDII. Grid Computing Course, Physics Dept, Univ. of Catania

Top BDII vs Site BDII Site BDII By default it is installed on the CE It collects all site GRISes* (for example SE,RB,LFC,etc..) The name of the service is bdii Top BDII It collects all site BDIIs** (CE1,CE2 ..... CEn) It gives to the RB all needed informations to match and dispatch user's jobs It can run in the same machine where the RB is running (it's more fast in answer) *GRIS=Grid Resource Information Service **BDII=Berkely Database Infomatin Index Grid Computing Course, Physics Dept, Univ. of Catania

LCG Information System Hierarchy BDII-A BDII-B BDII-C Site BDII Site BDII Site BDII SE Local GRIS SE Local GRIS CE Local GRIS SE Local GRIS CE Local GRIS SE Local GRIS RB Local GRIS Site 1 Site 2 Site 3 Grid Computing Course, Physics Dept, Univ. of Catania

Generic Information Provider The Generic Information Provider, GIP, is a highly configurable information provider that makes a separation between static and dynamic information. It can be used to produce any kind of information for use with LDAP based grid information systems. An information provider in its simplest form is a script that prints an LDIF file to standard out. The LDIF file should conform to the schema used in the GIP. The problem is with dynamic information. Usually there are only a few attributes that have to be found dynamically. The idea with the GIP is to use dynamic plug-ins to obtain these values and use a common framework for everything else. Grid Computing Course, Physics Dept, Univ. of Catania

GIP configuration file The GIP configuration file contains all the configuration parameters for the GIP. temp_dir The directory to use as the GIPs cache. plugin_dir The directory where to put links to the dynamic plugins. static_dir The directory where to put inks to the static ldif files. provider_dir The directory where to put links to the information providers. freshness How long to use the cache before running the dynamic plugins again. cache_ttl How long the cache is valid. response How long the GIP will wait for dynamic plugins before continuing. timeout Timeout for the dynamic plugins in seconds. Grid Computing Course, Physics Dept, Univ. of Catania

GIP The GIP script reads in all the ldif files from the static_dir and run all the providers in the plugin_dir. It will then run any dynamic plug-ins found in the plugin_dir to obtain the dynamic values. It will print the LDIF to standard out but use the any dynamic values found from the plugins. Plugins and providers should not be put directly into the directory, insted a symbolic link or wrapper script should be used. Grid Computing Course, Physics Dept, Univ. of Catania

SiteInfo Information Provider dn: in=host.invalid/siteinfo,Mds-Vo-name=local,o=grid objectClass: SiteInfo objectClass: DataGridTop objectClass: DynamicObject siteName: SITE-NAME sysAdminContact: admin@host.invalid userSupportContact: support@host.invalid siteSecurityContact: security@host.invalid dataGridVersion: LCG-2_2_0 installationDate: 20040106120000Z Grid Computing Course, Physics Dept, Univ. of Catania

SiteInfo Schema objectClass: DataGridTop objectClass: SiteInfo in: information siteName: Name of site sysAdminContact: Contact details for local system administrator userSupportContact: Contact details for local user support siteSecurityContact: Contact details for site security dataGridVersion: Overall DataGrid software version installationDate: Date of installation cpuResourceDescription: Description of available CPU resources diskResourceDescription: Description of available disk resources supportedFileSystem: Supported file systems AFS, NFS, local batchSystem: Available batch system, PBS, LSF, BQS, Condor, fork etc massStorageDescription: Description of available mass storage experimentalSoftware: Availablity of software, ATLAS, CMS, ESA etc… Grid Computing Course, Physics Dept, Univ. of Catania

BDII Update Process (1) The BDII consists of two or more standard LDAP databases that are populated by an update process. The update process obtains LDIF (LDAP Data Interchange Format ) file from either doing an ldapsearch on LDAP URLs or by running a local script that generates LDIF. The LDIF is then inserted into the LDAP database. Options exist to update the the list of LDAP URLs from a web page and to use an LDIF file from a web page to modify the data before inserting it into the database. The daemon script will start the update process is bdii-update. Grid Computing Course, Physics Dept, Univ. of Catania

BDII Update Process (2) The update process does the following in an endless loop: Update configuration file that contains the LDAP URLs from the web. Fork off processes to obtain the LDIF from the URLs. Write the LDIF files in the temporary directory. Try to obtain an LDIF file that will modify (filter) the data. Modify the data read from the temporary files. Kill the oldest database instance along with any stale clients. Clear the corresponding database. Add the fresh data to a new database. Start the new database. Forward traffic from the BDII read port to the new database. Sleep for a while. Grid Computing Course, Physics Dept, Univ. of Catania

lcg-info Grid Computing Course, Physics Dept, Univ. of Catania 36

Praticals: lcginfo -h/--help: to see the help –list-attrs: print the list of the possible attributes --list-ce: lists the CEs which satisfy a query, or all the CEs if no query is given. --list-se: lists the SEs which satisfy a query, or all the SEs if no query is given. --bdii: allows to specify a BDII in the form <hostname>:<port>. If not given, the value of the environmental variable LCG_GFAL_INFOSYS is used. If that is not defined, the command returns an error. --vo: restricts the output to CEs or SEs where the given VO is authorized. Grid Computing Course, Physics Dept, Univ. of Catania

lcg-infosites Grid Computing Course, Physics Dept, Univ. of Catania 38

Praticals: lcg-infosites -h/--help: help option --vo: VO name (mandatory) --is: it's possible to specify a not default Top BDII Some options: se: The names of the SEs supported by the user's VO ce: The information relative to number of CPUs, running jobs,etc. rb: Names of the Rbs available for each VO sitenames: Names of the LCG sites tag: The names of the tags relative to the software installed in site is printed together with the corresponding CE closeSE: The names of the CEs where the user's VO is allowed to run together with their corresponding closest SEs are provided Grid Computing Course, Physics Dept, Univ. of Catania

References Esercitazione LCG-2 User Guide GLUE Schema http://repository.gs.ba.infn.it/twiki/bin/view/GILDA/InformationSystems#Information_System_BDII LCG-2 User Guide https://edms.cern.ch/file/454439//LCG-2-UserGuide.html GLUE Schema http://infnforge.cnaf.infn.it/glueinfomodel/ Grid Computing Course, Physics Dept, Univ. of Catania