1. LCG Information and Monitoring System Jason Shih WLCG T2 Asia Workshop Dec 2, 2006: TIFR

2. Outline Components of IS Introduction to MDS, GRIS, GIIS and BDII Monitoring tools GridICE & GIIS monitor Practicals: Job Requirement, ranking and FCR High level tools lcg-infosites, lcg-info, glite-sd-query, lcg-ManageVOTag Data management Nice tool: LDAP browser APEL accounting system

3. Information System Provide information about the grid resources, and their status Offering data conforms GLUE (Grid Laboratory for Uniform Environment) Schema (common conceptual data model) Main component of Glue schema trying to describe: CE (computing element) SE (storage element) Binding information for SE and CE From users’ point of view: Where I can have available resource to accomplish my job (requirement, ranking etc.) Where I can register/replicate/copy files to/from (SE attributes) where I can find certain applications to run my job (VO Tag) Where are the nearest resources (computing, storage fabric etc.)

4. MDS, GRIS, GIIS and BDII

5. LDAP Protocol: the data model dn: objectclass: : dn: objectclass: : This is an entry; collection of attributes. It’s defined by a unique DN (Distinguished Name) Objectclass: special attributes a) Defines the tree structure of a certain entry b) Filters the entries of this objectclass White space to separate entries from each other The types and objectclass names should follow a schema (Glue Schema on LCG) The information is imported and exported by LDIF files (LDAP Data Interchange Format) which follow such structure

6. MDS (monitoring and discovery services) IP (information provide) running on computing and storage elements generate relevant information of the resources and published by the grid resource information server.

7. Globus.conf: MDS … Content in globus configuration file: …. [mds] globus_flavor_name=gcc32dbgpthr user=edginfo [mds/gris/provider/edg] [mds/gris/registration/site] regname=Taiwan-LCG2 reghn=lcg00125.grid.sinica.edu.tw …. /etc/globus.conf:

8. GRIS (Grid Resource Information Service) Local GRISes run on Computing/Storage elements and publish characteristic information and status of the services, which include: Static information For SE: Service endpoint, SE unique ID, Schema Ver., AccessProtocolType, SAPath and SARoot For CE: SiteWeb, SiteName, SiteUniqueID, SysAdminContact etc. Dynamic information For SE: AvailableSpace, UsedSpace wrt all supported VOs For CE: FreeCPUs, RunningJobs, TotalJobs, WaitingJobs, EstimatedResponseTime

9. Local GRIS (Cont’) ldapsearch -x -H ldap://`hostname`:2135 -b mds-vo-name=local,o=grid Query from target host with “–h” associated with proper port no., with “–p” -x indicate that simple authentication will be used -b option is adopted to specify the initial entry from which starts the search in the LDAP tree

10. GIIS (Grid Index Information Service) The role of GIIS is to collect info from all GRISes, and sources published from other GIIS, but it has shown its scalability limits with growing no. of sites. Due to this, BDII (Berkeley DB Information Index) was introduced. GIIS has been kept at site level to collect information from GRISes implemented at each resource fabric.

11. BDII (Berkeley DB Information Index) BDII queries the GIISes and act as cache storing information about the grid status in its database. Each BDII contains information from site GIISes define by a configuration file in which BDII fork processes simultaneously and cutout unanswered queries after timeout. User and other Grid Services (such as RB) can interrogate with BDII to get information of grid status. Up2date information can be found directly from site GIISes or from local GRISes that run on specific resources.

12. BDII configuration file (Site GIIS) # more bdii.conf BDII_PORT_READ=2170 BDII_PORTS_WRITE="2171 2172 2173" BDII_USER=edguser BDII_BIND=mds-vo-name=Taiwan-LCG2,o=grid BDII_PASSWD= BDII_SEARCH_FILTER='*' BDII_SEARCH_TIMEOUT=30 BDII_BREATHE_TIME=60 BDII_AUTO_UPDATE=no BDII_AUTO_MODIFY=no BDII_DIR=/opt/bdii/ BDII_UPDATE_URL=http://grid-deployment.web.cern.ch/grid- deployment/gis/lcg2-bdii/dteam/lcg2-all-sites.conf BDII_UPDATE_LDIF=http:// SLAPD=/usr/sbin/slapd SLAPADD=/usr/sbin/slapadd

13. BDII update configurtion (Site GIIS) # more bdii-update.conf CE ldap://lcg00125.grid.sinica.edu.tw:2135/mds-vo-name=local,o=grid SE ldap://lcg00123.grid.sinica.edu.tw:2135/mds-vo-name=local,o=grid SE2 ldap://lcg00129.grid.sinica.edu.tw:2135/mds-vo-name=local,o=grid RB ldap://lcg00124.grid.sinica.edu.tw:2135/mds-vo-name=local,o=grid PX ldap://lcg00127.grid.sinica.edu.tw:2135/mds-vo-name=local,o=grid LFC ldap://lfc.grid.sinica.edu.tw:2135/mds-vo-name=local,o=grid CASTORSC ldap://lcg00115.grid.sinica.edu.tw:2135/mds-vo-name=local,o=grid DPM ldap://dpm01.grid.sinica.edu.tw:2135/mds-vo-name=local,o=grid

14. Top BDII (global) Config file: BDII_PORT_READ=2170 BDII_PORTS_WRITE="2171 2172 2173" BDII_USER=edguser BDII_BIND=mds-vo-name=local,o=grid BDII_PASSWD= BDII_SEARCH_FILTER='(|(objectClass=GlueSchemaVersion)(objectClass=GlueTop))' BDII_SEARCH_TIMEOUT=30 BDII_BREATHE_TIME=60 BDII_AUTO_UPDATE=yes BDII_AUTO_MODIFY=yes BDII_DIR=/opt/bdii/ BDII_UPDATE_URL=http://grid-deployment.web.cern.ch/grid-deployment/gis/lcg2- bdii/dteam/lcg2-all-sites.conf BDII_UPDATE_LDIF=https://goc.grid-support.ac.uk/gridsite/bdii/BDII/www/bdii- update.ldif SLAPD=/usr/sbin/slapd SLAPADD=/usr/sbin/slapadd

15. Top BDII update config file: # # Top Level BDII configuration file # --------------------------------- # This file is generated, DO NOT EDIT it directly #TW-ASGC-WS TW-ASGC-WS ldap://ws45.twgrid.org:2170/mds-vo-name=TW-ASGC-WS,o=grid #TW-ASGC-WS1 TW-ASGC-WS1 ldap://ws52.twgrid.org:2170/mds-vo-name=TW-ASGC-WS1,o=grid #TW-ASGC-WS2 TW-ASGC-WS2 ldap://ws62.twgrid.org:2170/mds-vo-name=TW-ASGC-WS2,o=grid #TW-ASGC-WS3 TW-ASGC-WS3 ldap://ws72.twgrid.org:2170/mds-vo-name=TW-ASGC-WS3,o=grid #TW-ASGC-WS4 TW-ASGC-WS4 ldap://ws82.twgrid.org:2170/mds-vo-name=TW-ASGC-WS4,o=grid

16. LCG Information System LCG Information System GIP (Generic Information Provider) Top level BDII MDS (Monitoring and Discovery System) Site BDII LDAP Protocol

17. Components of IS: GRIS, GIIS & BDII Each site can run a BDII. It collects the information coming from the GIISs % ldapsearch –x –h -p 2170 –b “o=grid” At each site, a site GIIS collects the information given by the GRISs % ldapsearch –x –h -p 2135 –b “mds-vo-name=,o=grid” Local GRISes run on CEs and SEs at each site and report dynamic and static information % ldapsearch –x –h -p 2135 –b “mds-vo-name=local,o=grid” Abbreviations: BDII: Berkeley DataBase Information Index GIIS: Grid Index Information Server GRIS: Grid Resource Information Server From LCG2.3.0 site GIIS has been replaced by “local” BDII ldapsearch –x –h -p 2170 –b “o=grid”

18. LCG Information System (cont’) We try to figure out how the information is generated and transported among the nodes. Let’s have a look to the files… After a while we will have such a basic picture in mind: globus-mdsThis file is new!!!! /opt/var lcg-static.conf lcg-dynamic libexec ldapadd ldapsearch Templates too!! Files generated by the system Files generated by a deamon Even more config files Where does path come from?!?!?!?! This file was not here yesterday! Template II Let’s try to understand it, element by element: 1. Not paying so much attention to the code 2. Just giving the fundamental FILES

19. How information generated? globus-mds: deamon script It’s a deamon, initialized at boot time. It runs the info-providers and has a 60s cache grid-info-resource-ldif.conf a) Created by the startup script of globus-mds b) It begins the chain of scripts generating the information of the node /opt/globus/etc/grid-info-resource-ldif.conf dn: Mds-Vo-name=local,o=grid objectclass: GlobusTop objectclass: GlobusActiveObject objectclass: GlobusActiveSearch type: exec path: /opt/lcg/libexec base: lcg-info-wrapper args: cachetime: 60 timelimit: 20 sizelimit: 250

20. Info generation: lcg-info-wrapper grid-info-resource-ldif.conf lcg-info-wrapper Interface between the info System and all providers #!/bin/sh /opt/lcg/libexec/lcg-info-generic \ /opt/lcg/var/gip/lcg-info-generic.conf Executed by the previous file lcg-info-generic: 1.Read the file containing the static information Where does this file comes from? 2. Execute dynamic plug-ins which produces the dynamic information Where is the script name included? 3. GIP (Generic Information Provider) uses dynamic information built in cache Where is the path name defined? This is included in its Argument: GIP Config File lcg-info-generic.conf GIP SCRIPT

21. Configuration file for SE ldif_file=/opt/lcg/var/gip/lcg-info-static.ldif generic_script=/opt/lcg/libexec/lcg-info-generic wrapper_script=/opt/lcg/libexec/lcg-info-wrapper temp_path=/opt/lcg/var/gip/tmp template=/opt/lcg/etc/GlueSite.template template=/opt/lcg/etc/GlueCE.template template=/opt/lcg/etc/GlueCESEBind.template template=/opt/lcg/etc/GlueSE.template template=/opt/lcg/etc/GlueService.template # Common for all GlueInformationServiceURL: ldap://lcg00123.grid.sinica.edu.tw:2135/mds-vo- name=local,o=grid dynamic_script=/opt/lcg/libexec/lcg-info-dynamic-se GlueSEType: disk GlueSEPort: 2811 GlueSESizeTotal: 0 GlueSESizeFree: 0 GlueSEArchitecture: disk GlueSAType: permanent GlueSAPolicyFileLifeTime: permanent GlueSAPolicyMaxFileSize: 10000 Contains the static Information in ldif format Templates to create the information in ldif Format: GIP Templates df –k /flatfiles/SE00/VO The dynamic information: used and available space, visible in $temp_path/lcg-info-synamic-classic.ldif

22. Configuration file for CE ldif_file=/opt/lcg/var/gip/lcg-info-static.ldif generic_script=/opt/lcg/libexec/lcg-info-generic wrapper_script=/opt/lcg/libexec/lcg-info-wrapper temp_path=/opt/lcg/var/gip/tmp template=/opt/lcg/etc/GlueSite.template template=/opt/lcg/etc/GlueCE.template template=/opt/lcg/etc/GlueCESEBind.template template=/opt/lcg/etc/GlueSE.template template=/opt/lcg/etc/GlueService.template # Common for all GlueInformationServiceURL: ldap://lcg00125.grid.sinica.edu.tw:2135/mds-vo-name=local,o=grid dn: GlueSiteUniqueID=Taiwan-LCG2,mds-vo-name=local,o=grid GlueSiteName: Taiwan-LCG2 GlueSiteDescription: LCG Site …. dynamic_script=/opt/lcg/libexec/lcg-info-dynamic-ce dynamic_script=/opt/lcg/libexec/lcg-info-dynamic-software /opt/lcg/var/gip/lcg-info- generic.conf Contains the static Information in ldif format Templates to create the information in ldif Format: GIP Templates

23. Dynamic information of CE /opt/lcg/libexec/lcg-info-dynamic-pbs /opt/lcg/libexec/lcg-info-dynamic-software GlueCEInfoLRMSVersion, GlueCEInfoTotal(Free)CPUs, GlueCEPolicyMaxCPUTime(RunningJobs)(WallClockTime), GlueCEStateTotal(Waiting)(Running)Jobs, GlueCEStateWorst(Estimate)ResponseTime Quantities calculated basically using: qstat –B –f pbsHost obtained from its argument: pbsnodes –a –s pbsHostlcg-info-generic.conf GlueHostApplicationSoftwareRunTimeEnvironment Obtained by reading the *list files stored under: /opt/edg/var/info/VO_name This files are created by running the script lcg-ManageVOTag

24. Configuration files of RB lcg-info-generic.conf: ldif_file = /opt/lcg/var/gip/lcg-info-static.ldif generic_script = /opt/lcg/libexec/lcg-info-generic wrapper_script = /opt/lcg/libexec/lcg-info-wrapper temp_path = /opt/lcg/var/gip/tmp (EMPTY) template = /opt/lcg/etc/GlueService.template (information relative to the node written following the Glue Schema) ALL RB Information is static

25. Interoperability of LCG IS RB Local GRIS SE Local GRIS CE Local GRIS BDII-A BDII-B MSS Local GRIS PX Local GRIS CE Local GRIS SE Local GRIS BDII-C CE Local GRIS CE Site GIIS CE Local GRIS CE Site GIIS CE Local GRIS CE Site GIIS Site 1 Site 2 Site 3

26. Monitoring Tools

27. GridICE (Gris View) http://gridice2.cnaf.infn.it:50080/gridice/

28. Gstat GOC DB SITE BSITE A GSTAT SERVER (Taiwan) Production BDII gstat runs each 15 minutes through a cron List of sites Site A under testing Agent 3 BDII Agent 2 R-GMA Agent 1 ldapsearch WEB PAGE Filter Data

29. GIIS Monitoring (global view) sitescountriestotalCPUfreeCPUrunJobwaitJobseAvail TBseUsed TB maxCPUavgCPU 2084828156127152075917070813086.98114.93975531301 http://goc.grid.sinica.edu.tw/gstat/

30. GIIS Monitoring (II): regional view

31. GIIS (III): Site level profile (I) (II) (III) (IV)

32. Practical consideration using LCG Information System

33. Job description lang.: requirement and ranking [ Executable = “myexe"; StdOutput = "std.out"; StdError = "std.err"; InputSandBox = {“myexe"}; OutputSandBox = {"std.out","std.err"}; rank = -other.GlueCEPolicyMaxCPUTime; DefaultRank = -other.GlueCEStateEstimatedResponseTime; VirtualOrganisation = "dteam"; requirements = ( other.GlueCEUniqueID == "lcg00125.grid.sinica.edu.tw:2119/jobmanager-lcgpbs-dteam" ) && ( other.GlueCEStateStatus == "Production" ) && ( other.RunTimeEnvironment == "my_own_program_version_minor_version" ) ; ]

34. Example (I): Job ranking and requirement Atlas Production Monitoring Page: Requirement: ( MaxCPUTime * CINT2000 >= 333333 ) && ( Memory > 600 ) && ( OutboundConnectivity == True ) && ( CEStatus == Production ) Ranking: if ( Waiting == 0 ){if ( Running == 0 ){ rank = FreeCPUs * 100 } else { rank = (FreeCPUs * 100) / ( Running + 1 ) } } else { rank = - Waiting / ( Running + 1 ) }

35. Example (II): FCR (Freedom of Choices for Resources) https://goc.grid-support.ac.uk/gridsite/bdii/site-apps/FCR-cgi/fcr.cgi

36. Example (III): High level cmd: lcg-infosites (I) $ lcg-infosites --vo dteam ce | grep sinica ************************************************************** These are the related data for dteam: (in terms of CEs) ************************************************************** #CPU Free Total Jobs Running Waiting ComputingElement ------------------------------------------------------------------------------------------------- 124 123 0 0 0 lcg00125.grid.sinica.edu.tw:2119/jobmanager-lcgpbs-dteam 0 0 1 1 0 testbed001.phys.sinica.edu.tw:2119/jobmanager-lcgpbs-dteam $ lcg-infosites --vo dteam ce -v 2 | head -10 ************************************************************** These are the related data for dteam: (in terms of CEs) ************************************************************** RAMMemory Operating System System Version Processor CE Name ---------------------------------------------------------------------------------------------------------------------------------------------- 2000 ScientificLinux 3 PIV lcg00125.grid.sinica.edu.tw 1025 ScientificLinux 3 PIII testbed001.phys.sinica.edu.tw

37. High level cmd: lcg-infosites (II) Retrieving information of SE: Where are the closeSE? $ lcg-infosites --vo twgrid se ************************************************************** These are the related data for twgrid: (in terms of SE) ************************************************************** Avail Space(Kb) Used Space(Kb) Type SEs ---------------------------------------------------------- 814664 13534300 se.cc.ncu.edu.tw 33479944 29051084 melon035.ngpp.ngp.org.sg 757116936 1200483004 lcg00123.grid.sinica.edu.tw 3960000000 12180000000 castor.grid.sinica.edu.tw 6740000000 580000000 castorsc.grid.sinica.edu.tw $ lcg-infosites --vo twgrid closeSE Name of the CE: ce.cc.ncu.edu.tw:2119/jobmanager-lcgpbs-twgrid Name of the close SE: se.cc.ncu.edu.tw Name of the CE: melon.ngpp.ngp.org.sg:2119/jobmanager-lcgpbs-twgrid Name of the close SE: melon035.ngpp.ngp.org.sg Name of the CE: lcg00125.grid.sinica.edu.tw:2119/jobmanager-lcgpbs-twgrid Name of the close SE: lcg00123.grid.sinica.edu.tw Name of the close SE: castor.grid.sinica.edu.tw Name of the close SE: castorsc.grid.sinica.edu.tw

38. Arguments for lcg-infosites

39. High level cmd: lcg-info This command can be used to list either CE or SE that satisfy a given set of conditions and to print the values of a given set of attributes Retrieved information from $LCG_GFAL_INFOSYS environment variable. E.g.: $ env |grep GFAL LCG_GFAL_INFOSYS=lcg00126.grid.sinica.edu.tw:2170 [Quote] The query syntax is like this: attr1 op1 valueN,... attrN opN valueN where attrN is an attribute name op is =, >= or <=, and the cuts are ANDed. The cuts are comma-separated and spaces are not allowed.

40. High level cmd: lcg-info (II) Usage lcg-info --list-ce [--bdii bdii] [--vo vo] [--sed] [--query query] [--attrs list] lcg-info --list-se [--bdii bdii] [--vo vo] [--sed] [--query query] [--attrs list] lcg-info --list-attrs lcg-info --help

41. High level cmd: lcg-info (III) Arguments of lcg-info:

42. High level cmd: lcg-info (IV) Attribute nameGlue object class Glue attribute name MaxTime GlueCE GlueCEPolicyMaxWallClockTime CEStatus GlueCE GlueCEStateStatus TotalJobs GlueCE GlueCEStateTotalJobs CEVOs GlueCE GlueCEAccessControlBaseRule TotalCPUs GlueCE GlueCEInfoTotalCPUs FreeCPUs GlueCE GlueCEStateFreeCPUs CE GlueCE GlueCEUniqueID WaitingJobs GlueCE GlueCEStateWaitingJobs RunningJobs GlueCE GlueCEStateRunningJobs CloseCE GlueCESEBindGroup GlueCESEBindGroupCEUniqueID CloseSE GlueCESEBindGroup GlueCESEBindGroupSEUniqueID SEVOs GlueSA GlueSAAccessControlBaseRule UsedSpace GlueSA GlueSAStateUsedSpace AvailableSpace GlueSA GlueSAStateAvailableSpace Type GlueSE GlueSEType SE GlueSE GlueSEUniqueID Protocol GlueSEAccessProtocol GlueSEAccessProtocolType ArchType GlueSL GlueSLArchitectureType Processor GlueSubCluster GlueHostProcessorModel OS GlueSubCluster GlueHostOperatingSystemName Cluster GlueSubCluster GlueSubClusterUniqueID Tag GlueSubCluster GlueHostApplicationSoftwareRunTimeEnvironment Memory GlueSubCluster GlueHostMainMemoryRAMSize

43. Check Sepc info from TWGrid VO: List available Tags, say CMS, from TWGrid VO: High level cmd: lcg-info (V) $ lcg-info --list-ce --vo twgrid --attrs 'CINT2000,CFP2000' - CE: ce.cc.ncu.edu.tw:2119/jobmanager-lcgpbs-twgrid - CINT2000 1107 - CFP2000 0 - CE: lcg00125.grid.sinica.edu.tw:2119/jobmanager-lcgpbs-twgrid - CINT2000 1378 - CFP2000 1583 - CE: melon.ngpp.ngp.org.sg:2119/jobmanager-lcgpbs-twgrid - CINT2000 381 - CFP2000 0 $ lcg-info --list-ce --vo twgrid --attrs 'Tag' | grep cms VO-cms-slc3_ia32_gcc323 VO-cms-OSCAR_3_6_5 VO-cms-ORCA_8_13_1 VO-cms-FAMOS_1_3_2 VO-cms-ORCA_8_7_4 VO-cms-OSCAR_3_9_8

44. glite-sd-query glite-sd-query help searching for suitable service type of storage element publish from specific site, and extensive information are also provided. Basic query, SRM service endpoint from specific SE: Print more details, including the SAPath and supported VOs associated to the mount point: [lcg00122] ~> glite-sd-query --host castorsc.grid.sinica.edu.tw -t SRM Name: httpg://castorsc.grid.sinica.edu.tw:8443/srm/managerv1 Type: SRM Endpoint: httpg://castorsc.grid.sinica.edu.tw:8443/srm/managerv1 Version: 1.1.0 [lcg00122] ~> glite-sd-query --host castorsc.grid.sinica.edu.tw -t SRM –x |grep Mount Key: cms:SEMountPoint - Value: /castor/grid.sinica.edu.tw/sc/cms Key: alice:SEMountPoint - Value: /castor/grid.sinica.edu.tw/sc/alice Key: atlas:SEMountPoint - Value: /castor/grid.sinica.edu.tw/sc/atlas Key: dteam:SEMountPoint - Value: /castor/grid.sinica.edu.tw/sc/dteam Key: apesci:SEMountPoint - Value: /castor/grid.sinica.edu.tw/sc/apesci Key: biomed:SEMountPoint - Value: /castor/grid.sinica.edu.tw/sc/biomed Key: twgrid:SEMountPoint - Value: /castor/grid.sinica.edu.tw/sc/twgrid

45. lcg-ManageVOTag lcg-ManageVOTag help retrieving all VO tag publish from site CE, that end users can easily get updated status of s/w installation/validation from IS directly. [lcg00122] ~> lcg-ManageVOTag -host lcg00125.grid.sinica.edu.tw -vo atlas --list VO-atlas-release-11.0.42 VO-atlas-release-11.0.5 VO-atlas-release-11.5.0 VO-atlas-production-12.0.3 VO-atlas-tier-T1 VO-atlas-cloud-TW VO-atlas-production-12.0.31

46. Example (IV): APEL Accounting http://goc.grid-support.ac.uk/gridsite/accounting/interoperate.html

47. APROC: Accounting

48. Example (V): GridView: http://lxgate24.cern.ch/GRIDVIEW/

49. LDAP browser (win + linux) Download from http://www-unix.mcs.anl.gov/~gawor/ldap/download.html (latest version is 2.8.2)http://www-unix.mcs.anl.gov/~gawor/ldap/download.html

50. LDAP Browser: SE (Castor)

51. LDAP Browser: VO Tag

52. lcg utilities (lcg-cr, lcg-rep etc.) lcg-cr -v --vo dteam -d lcg00123.grid.sinica.edu.tw file:///etc/group -l lfn:test By default, w/o –d, lcg-cr will adopt $VO_DTEAM_DEFAULT_SE as default target SE to register the file Adopt GlueSAPath from IS in which the file will be located e.g.: $ ldapsearch -x -H ldap://lcg00123.grid.sinica.edu.tw:2135 -b mds-vo-name=local,o=grid | grep dteam | grep GlueSAPath Before LFC introduced, RLS service endpoint have to be included in IS as well if specify “-l” option (troubleshooting). lcg-rep -v --vo dteam -d castor.grid.sinica.edu.tw lfn:test Name space (cf. to abs path) retrieved from IS, e.g.: GlueSAPath: /castor/grid.sinica.edu.tw/grid/dteam If service unique Id is missing from IP, replication fail (check gstat) Given “-p” to def relative path for the files being replicated to.

53. Summary LCG information system consist by: GIP, which generate static/dynamic information to: Site BDII as well as global (top) BDII. MDS BDII Users retrieve information from LDAP based IS High level command available in LCG Gstat have been widely used to profiling and monitoring site/global level IS problem, continuously provide grid resource status

54. References R-GMA overview page. http://www.r-gma.org/ R-GMA in EGEE http://hepunx.rl.ac.uk/egee/jra1-uk/ R-GMA Documenation http://hepunx.rl.ac.uk/egee/jra1-uk/glite-r1/ GLUE Schema http://infnforge.cnaf.infn.it/glueinfomodel/