1. WP3 RGMA Deployment Laurence Field / RAL Steve Fisher / RAL

2. WP3 Laurence Field 30/01/2003R-GMA Deployment2 R-GMA Uses the Grid Monitoring Architecture from Global Grid Forum R-GMA is a relational implementation Applied to both information and monitoring Creates impression that you have one RDBMS per Virtual Organisation Producer Consumer Registry Information flow Meta-data flow

3. WP3 Laurence Field 30/01/2003R-GMA Deployment3 SiteInfo Example dn: in=siteinfo,Mds-Vo-name=ralrgma,o=grid objectclass: SiteInfo objectclass: DataGridTop objectclass: DynamicObject siteName: RALRGMA sysAdminContact: edg-site-admin@gridpp.rl.ac.uk userSupportContact: edg-site-admin@gridpp.rl.ac.uk siteSecurityContact: edg-site-admin@gridpp.rl.ac.uk dataGridVersion: v1_4_3 installationDate: 20021111103000Z createTimestamp: 20030129111208Z modifyTimestamp: 20030129111208Z entryTtl: 1200 admin@gridpp.rl.ac.uk RAL in=siteinfo,Mds-Vo-name=ralrgma,o=grid sysAdminContactsiteNamedn GinGout

4. WP3 Laurence Field 30/01/2003R-GMA Deployment4 R-GMAMDS Gin Information Provider Uses the same information provider as MDS Can run MDS and R-GMA in parallel Can compare information systems edg-rgma Gin GRIS CircularBuffer Producer

5. WP3 Laurence Field 30/01/2003R-GMA Deployment5 Gout R-GMA Consumer (CE) Consumer (SE) Consumer (SiteInfo) RDBMS DataBase Producer LDAP Server GOUT Consumer API Archiver LDAP Server is queried by RB Two RBs should be used currently in the UK –one for MDS and one for R-GMA

6. WP3 Laurence Field 30/01/2003R-GMA Deployment6 CE and SE Tables ComputingElement dn CEId TotalCPUs FreeCPUs TotalJobs RunningJobs …… CloseStorage Element dn CEId CloseSE …… StorageElement status dn SEId SEfreespace …… “ Select a ComputingElement with at least 1 free CPU that also has a CloseStorageElement with at least 1000 MB of free space” SELECT DISTINCT ComputingElement.CEId FROM ComputingElement, CloseStorageElement,StorageElementStatus WHERE ComputingElement.FreeCPUs > 0 AND (ComputingElement.CEId = CloseStorageElement.CEId AND CloseStorageElement.CloseSE = StorageElementStatus.SEId AND StorageElementStatus.SEfreespace > 1000) R-GMA CLI Join

7. WP3 Laurence Field 30/01/2003R-GMA Deployment7 InfoCat One logical registry per VO –Replication/Distribution system ready soon At the moment only using one for all VOs Located at RAL –Infocat.gridpp.rl.ac.uk InfoCat machine also hosts BrowserServlet R-GMA Browser

8. WP3 Laurence Field 30/01/2003R-GMA Deployment8 Site A Information Catalogue Tomcat Registry Servlet Schema Servlet Client Config Gin Producer API Information Provider Resource Broker LDAP Gout Archiever API Tomcat Archiver Servlet Information flow MetaData flow Server Config Tomcat Producer Servlet Gin Producer API Information Provider job submission

9. WP3 Laurence Field 30/01/2003R-GMA Deployment9 R-GMA Tools R-GMA CLI –Command Line Interface (similar to MySQL) –Supports single query and interactive modes R-GMA Browser –JSP application dynamically generating web pages –Supports pre-defined and user-defined queries Pulse –R-GMA Java client-based GUI –Supports streaming and simple graphical displays Nagios –R-GMA plug-in instantiates an Archiver and uses that information to populate Nagios displays –Users can benefit from Nagios alert mechanism

10. WP3 Laurence Field 30/01/2003R-GMA Deployment10 Service Status R-GMA Browser Monitors EDG components Provides a standard interface for plug-in service status scripts –Return 0 or 1  OK or fail –Returns message from standard out Browsing via the web hostNamenameVOstatusmessagemeasurementTime gpp.rl.ac.ukR-GMAGridPPOKRunning20021111103000

11. WP3 Laurence Field 30/01/2003R-GMA Deployment11 How to upgrade Based on edg version 1_4_3 Install rpms need for R-GMA Configure via lcfg (or manually)

12. WP3 Laurence Field 30/01/2003R-GMA Deployment12 RPMS EDG RPMS edg-info-main-2.0.1-0 CE edg-info-siteinfo-1.0.0-1 SE edg-info-se-2.0.0-4 Externals j2sdk-1.4.1-fcs log4cpp-0.2.7-3 mm.mysql-2.0.6-2 xerces-c-1.7.0-1 MySQL-python2.1-0.9.2-1 python2-2.1.3-2 log4j-1.2.6-1jpp SWIG-1.3.11-edg1 R-GMA edg-rgma-api-c-2.2.8-1 edg-rgma-common-2.2.8-1 edg-rgma-api-cpp-2.2.8-1 edg-rgma-demo-2.2.8-1 edg-rgma-api-java-2.2.8-1 edg-rgma-servlets-2.2.8-5 edg-rgma-api-perl-2.2.8-1 edg-rgma-tools-2.2.8-5 edg-rgma-api-python-2.2.8-1 LCFG obj-rgma-2.2.8-5 obj-infoproviders-2.0.1-9

13. WP3 Laurence Field 30/01/2003R-GMA Deployment13 LCFG R-GMA rgma.servletlocation localhost rgma.registrylocation INFOCAT_HOSTNAME rgma.GOUT no rgma.service RGMA MDS GIN rgma.VO_RGMA SE_GDMP_VOS rgma.cmd_RGMA /opt/edg/bin/rgma-servlet-status /etc/tomcat4/server.xml rgma.VO_MDS SE_GDMP_VOS rgma.cmd_MDS /etc/rc.d/init.d/globus-mds status rgma.VO_GIN SE_GDMP_VOS rgma.cmd_GIN /etc/rc.d/init.d/edg-rgma-gin status

14. WP3 Laurence Field 30/01/2003R-GMA Deployment14 LCFG InfoProviders infoproviders.GRID_INFO_USER edginfo infoproviders.INFO_LOCATION info infoproviders.INFO_PROVIDER_LOG /opt/edg/var/info/info-provider-log infoproviders.SIenabled yes infoproviders.siteNameSITE_NAME_ infoproviders.sysAdminContactSITE_MAILROOT infoproviders.userSupportContactSITE_MAILROOT infoproviders.siteSecurityContactSITE_MAILROOT infoproviders.dataGridVersionSITE_EDG_VERSION infoproviders.installationDateSITE_INSTALLATION_DATE_ infoproviders.SITE_DN Mds-Vo-name=local,o=grid infoproviders.SITE_GIN_UPDATE_PERIOD600 infoproviders.CEenabled yes

15. WP3 Laurence Field 30/01/2003R-GMA Deployment15 BOSS job, edg-rgma- pulse CMS - BOSS Batch Object Submission System –A wrapper for any jobs a user wants to run A BOSS job is submitted containing the user’s job, whereupon BOSS carries out the following: –Forks the job off to a worker node –Parses (i.e. filters) ‘standard output’ and writes selected data from that output to a local database User application using R-GMA to stream data CircularBufferProducer used to publish parsed Standard Output Archiver used to collect and store information from multiple jobs

16. WP3 Laurence Field 30/01/2003R-GMA Deployment16 R-GMA Deployment and Performance Deployment of WP3 Demo Testbed (used in EU review) –Revealed R-GMA bug with multiple Tomcat instances – subsequently fixed –UK and INFN deployment following WP3 Demo Testbed R-GMA is being used by: –WP1 (logging and book keeping) –WP2 (replica manager using R-GMA as the information provider) –WP7 (network monitoring) –CMS (Imperial College, Brunel) (BOSS) Performance and Optimisation –Using NetLogger and profiling tools to identify possible bottlenecks

17. WP3 Laurence Field 30/01/2003R-GMA Deployment17 Programme for 2003 Functionality –Enhanced Mediator –Complete GRM integration for monitoring parallel applications –Nagios Integration Resilience and Scalability –Consider all failure modes –Complete Registry and Schema replication Security –Authorisation Performance Optimisation OGSA Compliant Implementation –GT3 inter-operability –Include wrapper for backwards compatibility

18. WP3 Laurence Field 30/01/2003R-GMA Deployment18 Summary R-GMA works! Need to deploy to enable R-GMA to be tested in a real grid environment –Within UK and Italy initially –Problems found will be fixed quickly Currently focusing on reliability, stability and performance