Presentation is loading. Please wait.

Presentation is loading. Please wait.

gLite Information System overview

Similar presentations


Presentation on theme: "gLite Information System overview"— Presentation transcript:

1 gLite Information System overview
Africa Joint EUMEDGRID-Support/EPIKH School for Application Porting Maha Metawei – Electronics Research Institute ERI Cairo, 24 October 2010

2 Outline Motivation / Introduction Background Need for an information system GLUE Schema [data model for the grid resources] LDAP [protocol used to query the information system] MDS Architecture [It implements the GLUE Schema using OpenLDAP] Information system in practice lcg-infosites lcg-info Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

3 Introduction Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

4 The power Grid paradigm
wind coal Electric Power Grid System Users can access electrical power coming from different (and heterogeneous) sources nuclear hydroelectric Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

5 The computing Grid approach
System Users can access storage and computing resources coming from different (and heterogeneous) sources Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

6 Information System What is? Why? How?
System to collect information on the state of resources Why? To discover resources of the grid and their nature To have useful data in order to who is in charge of managing the workload to do it more efficiently. To check for health status of resources. How? Monitoring state of resources locally and publishing fresh data on the information system. Adopting a data model that MUST be well known to all components that want to access monitored information Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

7 The Info. system scope Discover new added resources
Make available updated information about the resources Monitor resources load and status How does it work? Collecting information from the resources and publishing fresh data periodically. Adopting a data model that MUST be well known to all Grid components. Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

8 How it is used and by whom?
Users Retrieve information about resources: Where can I run my job? Where can I copy my files? Which software packages are available on a given CE? Grid Site Admins Publish information about the resources and services they provide Middleware WMS: matching job requirements and allocating the resources Monitoring Services: retrieving information about the status and availability of resources. Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

9 GLUE Schema LDAP MDS Architecture
Background GLUE Schema LDAP MDS Architecture Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

10 GLUE Schema - Overview GLUE Stands for “Grid Laboratory Uniform Environment” Provides a standard data model to describe information on static and dynamic grid resources It is not tied to any particular implementation The Grid components are represented as objects which have attributes and relations to other objects Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

11 GLUE Schema – Why we need it?
The Grid is a paradigm of distributed computing that enables the coordination of resources and services not subject to centralized control. These resources are geographically dispersed, span multiple trust domains and are heterogeneous. Resources can be dynamically contributed by different owner institutions, so a precise and shared description of resources among information consumers and resource providers is necessary. This description should also be common to different Grid infrastructures in order to contribute to the interoperability among them. Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

12 GLUE Schema – Core entities
Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

13 GLUE Schema – Site properties
Site entity – Properties: Property Type Description UniqueID string Unique Identifier of the Site Name Human-readable name Short description of this site Contact The main contact for the site. Syntax rule: "mailto:" followed by a list of addresses separated by a comma UserSupportContact addresses of the support service SysAdminContact addresses of the system administrator SecurityContact addresses of the security manager Location Geographical location of this site (e.g., city, state, country) Latitude real32 The position of a place north or south of the equator measured from -90º to 90º, with positive values going north and negative values going south Longitude The position of a place east or west of Greenwich, England measured from -180º to 180º with positive values going east and negative values going west Web uri The URI identifying a web page with more information about this site Sponsor VO sponsoring the site; the syntax should allow the expression of the percentage of sponsorship OtherInfo This attribute is to be used to publish info that does not fit in any other attribute of the site entity Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

14 GLUE Schema – Service properties
Service entity – Properties: Property Type Description UniqueID string Unique Identifier of this service Name Human-friendly name serviceType_t The service type Version Version of the service: <major version number>.<minor version number>.<patch version number> Endpoint uri Network endpoint for this service Status serviceStatus_t Status of the service. String enumeration: OK, Warning, Critical, Unknown, Other StatusInfo Textual explanation for the status of the service WSDL URI of the WSDL describing the service Semantics URL of detailed description StartTime dateTime_xs_t The timestamp related to last start time of this service Owner Owner of the service AccessControlBase.Rule ACL_t Authorization rule for this entity Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

15 GLUE Schema – Cluster entity
Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

16 GLUE Schema – CE entity Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

17 LDAP - Overview LDAP Stands for “Lightweight Directory Access Protocol” It is the protocol used to query the information system Optimized for reading, browsing and searching information (‘write-once-read-many-times’ service) Data is represented as a hierarchy of objects (entities) forming a tree structure Data Information Tree (DIT) Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

18 LDAP – Data Information Tree
Attribute Type String representation Distinguished Name (DN) Unique name (path) that unambiguously identifies a single entry dc=grid (root of the DIT) Attribute Type String domainComponent DC CountryName C OrganizationName O OrganizationUnitName OU CommonName CN c= Brazil c=Italy c=Spain o=INFN ou=Catania ou=Roma cn=Elisa Ingra cn=Elisa Ingra,ou=Catania,o=INFN,c=Italy,dc=grid Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

19 LDAP – Data Information Tree
The tree hierarchy is described by textual files following the LDIF format (LDAP Data Interchange Files) dn: dc=example,dc=root dc: example description: My company objectClass: dcObject objectClass: organization o: Example, Inc. ## FIRST Level hierarchy - people # this is an ENTRY sequence and is preceded by a BLANK line dn: ou=people, dc=example,dc=root ou: people description: All people in organisation objectClass: organizationalUnit ## SECOND Level hierarchy - people entries dn: cn=Joe Schmo,ou=people,dc=example,dc=root objectclass: inetOrgPerson cn: Joe Schmo sn: Schmo uid: jschmo mail: ou: sales Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

20 LDAP – Referrals An individual LDAP server might not store the entire DIT Servers need to be linked together in order to form a distributed directory that contains the whole DIT All client requests start at the global directory LDAP 1 LDAP 1 server LDAP 2 server LDAP 3 server c=USA,dc=example o=IBM,c=USA,dc=example dc=example referral c= USA o=CISCO o=IBM ou=IT ou=Web referral Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

21 Components of information system
Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

22 lcg-infosites In this section, two higher level tools will be presented: lcg-infosites and lcg-info. These tools should be enough for most common user needs and will usually avoid the necessity of raw LDAP queries. The lcg-infosites command is actually just a perl script wrapping a series of LDAP commands and was developed to allow users to retrieve information on Grid resources for the most common cases Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

23 Praticals: lcg-infosites
USAGE: lcg-infosites --vo options -v --is -h/--help: help option --vo: VO name (mandatory) --is: it's possible to specify a not default Top BDII Some options: se: The names of the SEs supported by the user's VO ce: The information relative to number of CPUs, running jobs,etc. rb: Names of the Rbs available for each VO sitenames: Names of the LCG sites tag: The names of the tags relative to the software installed in site is printed together with the corresponding CE closeSE: The names of the CEs where the user's VO is allowed to run together with their corresponding closest SEs are provided Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

24 Account Password: cairoschool
Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

25 HANDS ON Login to a UI machine and do the following:
Password: cairoschool Initiate a proxy to be able to run the grid commands ~]$ voms-proxy-init --voms gilda Enter GRID pass phrase: [CAIRO] Your identity: /C=IT/O=GILDA/OU=Personal Certificate/L=CAIRO/CN=CAIRO01 Creating temporary proxy Done Contacting voms.ct.infn.it:15001 [/C=IT/O=INFN/OU=Host/L=Catania/CN=voms.ct.infn.it] "gilda" Done Creating proxy Done Your proxy is valid until Sat Oct 23 02:46: Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

26 View the list of CE available for gilda VO
~]$ lcg-infosites --vo gilda ce valor del bdii: gilda-bdii.ct.infn.it:2170 #CPU Free Total Jobs Running Waiting ComputingElement ce-edu.grid.acad.bg:2119/jobmana ger-pbs-gilda ce.hpc.iit.bme.hu:2119/jobmanage r-lcgpbs-gilda ce1-egee.srce.hr:2119/jobmanager sge-prod grid010.ct.infn.it:2119/jobmanag er-lcgpbs-gilda gilda-01.pd.infn.it:2119/jobmana ger-lcgpbs-gilda sirius-ce.ct.infn.it:2119/jobman ager-lcgpbs-gilda grisuce.scope.unina.it:2119/jobm anager-lcgpbs-grisu_short grisuce.scope.unina.it:2119/jobm anager-lcgpbs-grisu_long iceage-ce-01.ct.infn.it:2119/job manager-lcgpbs-gilda dgt01.ui.savba.sk:2119/jobmanage r-lcgpbs-gilda gn0.hpcc.sztaki.hu:2119/jobmanag er-lcgpbs-gilda ce.scope.unina.it:2119/jobmanage r-lcgpbs-egee_short ce.scope.unina.it:2119/jobmanage r-lcgpbs-egee_long Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

27 You can list more info about CE
~]$ lcg-infosites --vo gilda ce -v 2 valor del bdii: gilda-bdii.ct.infn.it:2170 RAMMemory Operating System System Version Processor Subcluster name ScientificCERNSLC Beryllium Xeon ce-edu.grid.acad.bg ScientificSL Beryllium PIV ce.hpc.iit.bme.hu CentOS Final Opteron ce1-egee.srce.hr ScientificSL Beryllium GenuineIntel grid010.ct.infn.it ScientificCERNSLC Beryllium PIII gilda-01.pd.infn.it ScientificSL SL Xeon sirius-ce.ct.infn.it ScientificSL SL Xeon grisuce.scope.unina.it ScientificSL Beryllium Opteron iceage-ce-01.ct.infn.it ScientificSL SL P dgt01.ui.savba.sk ScientificSL Beryllium PentiumD gn0.hpcc.sztaki.hu ScientificSL SL Xeon ce.scope.unina.it Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

28 View the list of SEavailable for gilda VO
~]$ lcg-infosites --vo gilda se Avail Space(Kb) Used Space(Kb) Type SEs n.a n.a n.a se-edu.grid.acad.bg n.a aliserv6.ct.infn.it n.a gilda-02.pd.infn.it n.a sirius-se.ct.infn.it n.a grisuse.scope.unina.it n.a iceage-se-01.ct.infn.it n.a dgt02.ui.savba.sk n.a se.scope.unina.it Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

29 List of Software tags on each CE
~]$ lcg-infosites --vo gilda tag Name of the CE: grid010.ct.infn.it VO-gilda-ASTRA VO-gilda-BLAST VO-gilda-BLENDER-2.42a VO-gilda-CSOUND5.12.1 VO-gilda-FALIGN VO-gilda-GATE-4.0.0 VO-gilda-GNUPLOT_4_2_4 VO-gilda-LUXRENDER-0.6_ia32 VO-gilda-MMCS VO-gilda-MPEG2_V12 VO-gilda-OCTAVE VO-gilda-PARAVIEW-2.4 VO-gilda-SFK09 VO-gilda-root_v g VO-gilda-scons Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

30 List of CE and SE that are close together
~]$ lcg-infosites --vo gilda closeSE valor del bdii: gilda-bdii.ct.infn.it:2170 Name of the CE: ce-edu.grid.acad.bg:2119/jobmanager-pbs-gilda se-edu.grid.acad.bg Name of the CE: ce.hpc.iit.bme.hu:2119/jobmanager-lcgpbs-gilda se.hpc.iit.bme.hu Name of the CE: ce1-egee.srce.hr:2119/jobmanager-sge-prod se1-egee.srce.hr Name of the CE: grid010.ct.infn.it:2119/jobmanager-lcgpbs-gilda aliserv6.ct.infn.it Name of the CE: gilda-01.pd.infn.it:2119/jobmanager-lcgpbs-gilda gilda-02.pd.infn.it Name of the CE: sirius-ce.ct.infn.it:2119/jobmanager-lcgpbs-gilda sirius-se.ct.infn.it Name of the CE: grisuce.scope.unina.it:2119/jobmanager-lcgpbs-grisu_short grisuse.scope.unina.it Name of the CE: grisuce.scope.unina.it:2119/jobmanager-lcgpbs-grisu_long Name of the CE: iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-gilda iceage-se-01.ct.infn.it Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

31 lcg-info The lcg-info command can be used to list either CEs or the SEs that satisfy a given set of conditions on their attributes, and to print, for each of them, the values of a given set of attributes. The information is taken from the BDII specified by the LCG GFAL INFOSYS environment variable or in the command line. Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

32 Praticals: lcg-info -h/--help: to see the help
USAGE: lcg-info [--list-ce | --list-se] [--query ] [--attrs ] -h/--help: to see the help –list-attrs: print the list of the possible attributes --list-ce: lists the CEs which satisfy a query, or all the CEs if no query is given. --list-se: lists the SEs which satisfy a query, or all the SEs if no query is given. --bdii: allows to specify a BDII in the form <hostname>:<port>. If not given, the value of the environmental variable LCG_GFAL_INFOSYS is used. If that is not defined, the command returns an error. --vo: restricts the output to CEs or SEs where the given VO is authorized. Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

33 List number of FreeCPU for a CE that has MPI
lcg-info --vo gilda --list-ce --query 'Tag=MPICH' --attrs 'FreeCPUs‘ - CE: ce-edu.grid.acad.bg:2119/jobmanager-pbs-gilda - FreeCPUs - CE: ce.scope.unina.it:2119/jobmanager-lcgpbs-egee_long - FreeCPUs - CE: ce.scope.unina.it:2119/jobmanager-lcgpbs-egee_short - CE: ce1-egee.srce.hr:2119/jobmanager-sge-prod - FreeCPUs - CE: dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-gilda - FreeCPUs - CE: grid010.ct.infn.it:2119/jobmanager-lcgpbs-gilda - FreeCPUs - CE: iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-gilda - FreeCPUs Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

34 Another query lcg-info --vo gilda --list-ce --query 'TotalCPUs>=10' --attrs 'RunningJobs,FreeCPUs‘ - CE: ce.hpc.iit.bme.hu:2119/jobmanager-lcgpbs-gilda - RunningJobs - FreeCPUs - CE: ce.scope.unina.it:2119/jobmanager-lcgpbs-egee_long - FreeCPUs - CE: ce.scope.unina.it:2119/jobmanager-lcgpbs-egee_short - RunningJobs - CE: ce1-egee.srce.hr:2119/jobmanager-sge-prod - RunningJobs - FreeCPUs - CE: gilda-01.pd.infn.it:2119/jobmanager-lcgpbs-gilda - FreeCPUs - CE: gn0.hpcc.sztaki.hu:2119/jobmanager-lcgpbs-gilda - FreeCPUs Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

35 Useful link Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

36 For EUMED VO Login to a UI machine and do the following:
Password: cairoschool Put your user certificates files on /home/cairoxx/.globus/usercert.pem /home/cairoxx/.globus/userkey.pem Initiate a proxy to be able to run the grid commands ~]$ voms-proxy-init --voms eumed Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

37 View the list of CE available for EUMED VO
~]$ lcg-infosites --vo eumed ce valor del bdii: bdii.eumedgrid.eu:2170 #CPU Free Total Jobs Running Waiting ComputingElement gridsrv2-4.dir.garr.it:8443/cream-pbs-eumed ce-01.roma3.infn.it:2119/jobmanager-lcgpbs-eumed grid012.ct.infn.it:2119/jobmanager-lcglsf-eumed kalkan1.ulakbim.gov.tr:2119/jobmanager-lcgpbs-eumed ce.eumedgrid.eun.eg:2119/jobmanager-lcgpbs-eumed infnlns-ce-01.ct.pi2s2.it:8443/cream-lsf-eumed infn-ce-01.ct.pi2s2.it:8443/cream-lsf-eumed ce1.cnrst.magrid.ma:2119/jobmanager-lcgpbs-eumed ce01.grid.um.edu.mt:8443/cream-pbs-eumed grid2.cck.rnu.tn:8443/cream-pbs-eumed ce01.isabella.grnet.gr:2119/jobmanager-pbs-eumed ce01.junet.edu.jo:8443/cream-pbs-eumed ce01.grid.arn.dz:8443/cream-pbs-eumed ce02.grid.arn.dz:2119/jobmanager-lcgpbs-eumed Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

38 You can list more info about CE
~]$ lcg-infosites --vo eumed ce -v 2 valor del bdii: bdii.eumedgrid.eu:2170 RAMMemory Operating System System Version Processor Subcluster name ScientificSL Boron Xeon gridsrv2-4.dir.garr.it ScientificSL Boron Xeon ce-01.roma3.infn.it ScientificSL Beryllium Dual Core AMD Opteron(tm) Processor grid012.ct.infn.it ScientificSL Boron Xeon kalkan1.ulakbim.gov.tr Scientific Linux SL PIII ce.eumedgrid.eun.eg Scientific Linux SL Dual Core Opteron infnlns-ce-01.ct.pi2s2.it Scientific Linux SL Dual Core Opteron infn-ce-01.ct.pi2s2.it ScientificCERNSLC Beryllium Xeon ce1.cnrst.magrid.ma Scientific Linux Boron Opteron ce01.grid.um.edu.mt ScientificSL Boron XEON grid2.cck.rnu.tn ScientificSL Beryllium Xeon ce01.isabella.grnet.gr ScientificSL Boron Core_2_DUO ce01.junet.edu.jo ScientificSL Boron Xeon ce01.grid.arn.dz ce02.grid.arn.dz Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

39 View the list of SEavailable for eumed VO
~]$ lcg-infosites --vo eumed se Avail Space(Kb) Used Space(Kb) Type SEs n.a n.a storm-01.roma3.infn.it n.a torik1.ulakbim.gov.tr n.a n.a prod-se-02.ct.infn.it n.a se1.cnrst.magrid.ma Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

40 List of Software tags on each CE
~]$ lcg-infosites --vo eumed tag valor del bdii: bdii.eumedgrid.eu:2170 Name of the CE: gridsrv2-4.dir.garr.it VO-eumed-GATE-6.0.0 Name of the CE: ce-01.roma3.infn.it VO-eumed-ASTRA VO-eumed-SCILAB-5.2.2 Name of the CE: grid012.ct.infn.it Name of the CE: kalkan1.ulakbim.gov.tr Name of the CE: ce.eumedgrid.eun.eg Name of the CE: infnlns-ce-01.ct.pi2s2.it Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

41 List of CE and SE that are close together
~]$ lcg-infosites --vo eumed closeSE valor del bdii: bdii.eumedgrid.eu:2170 Name of the CE: gridsrv2-4.dir.garr.it:8443/cream-pbs-eumed gridsrv3-4.dir.garr.it Name of the CE: ce-01.roma3.infn.it:2119/jobmanager-lcgpbs-eumed storm-01.roma3.infn.it Name of the CE: grid012.ct.infn.it:2119/jobmanager-lcglsf-eumed prod-se-02.ct.infn.it Name of the CE: kalkan1.ulakbim.gov.tr:2119/jobmanager-lcgpbs-eumed torik1.ulakbim.gov.tr Name of the CE: ce.eumedgrid.eun.eg:2119/jobmanager-lcgpbs-eumed se.eumedgrid.eun.eg my-admin-node Name of the CE: infnlns-ce-01.ct.pi2s2.it:8443/cream-lsf-eumed infn-se-01.ct.pi2s2.it Name of the CE: infn-ce-01.ct.pi2s2.it:8443/cream-lsf-eumed Name of the CE: ce1.cnrst.magrid.ma:2119/jobmanager-lcgpbs-eumed se1.cnrst.magrid.ma Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

42 List number of FreeCPU for a CE that has MPI
~]$ lcg-info --vo eumed --list-ce --query 'Tag=MPICH' --attrs 'FreeCPUs' - CE: ce01.grid.um.edu.mt:8443/cream-pbs-eumed - FreeCPUs - CE: ce01.isabella.grnet.gr:2119/jobmanager-pbs-eumed - FreeCPUs - CE: ce01.junet.edu.jo:8443/cream-pbs-eumed - FreeCPUs - CE: ce1.cnrst.magrid.ma:2119/jobmanager-lcgpbs-eumed - FreeCPUs - CE: grid012.ct.infn.it:2119/jobmanager-lcglsf-eumed - FreeCPUs - CE: gridsrv2-4.dir.garr.it:8443/cream-pbs-eumed - FreeCPUs Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

43 Another query ~]$lcg-info --vo eumed --list-ce --query 'TotalCPUs>=10' --attrs 'RunningJobs,FreeCPUs’ - CE: ce-01.roma3.infn.it:2119/jobmanager-lcgpbs-eumed - RunningJobs - FreeCPUs - CE: ce01.grid.arn.dz:8443/cream-pbs-eumed - RunningJobs - FreeCPUs - CE: ce01.grid.um.edu.mt:8443/cream-pbs-eumed - FreeCPUs - CE: ce01.isabella.grnet.gr:2119/jobmanager-pbs-eumed - FreeCPUs - CE: ce1.cnrst.magrid.ma:2119/jobmanager-lcgpbs-eumed - FreeCPUs - CE: grid012.ct.infn.it:2119/jobmanager-lcglsf-eumed - FreeCPUs Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,

44 References gLite documentation LDAP Concepts & Overview
LDAP Concepts & Overview OpenLDAP community GLUE Schema Specification version 1.3 Final Cairo, Joint EUMEDGRID-Support/EPIKH School for Application Porting ,


Download ppt "gLite Information System overview"

Similar presentations


Ads by Google