WP3 RGMA Deployment Laurence Field / RAL Steve Fisher / RAL

WP3 GridPP7 1/7/2003R-GMA Deployment2 R-GMA Use the GMA from GGF A relational implementation –Powerful data model and query language Applied to both information and monitoring Creates impression that you have one RDBMS per VO Producer Consumer Registry Store location Lookup location execute or stream

WP3 GridPP7 1/7/2003R-GMA Deployment3 Relational Data Model Not a general distributed RDBMS system, but a way to use the relational model in a distributed environment where global consistency is not important. Producers announce:SQL “CREATE TABLE” publish:SQL “INSERT” Consumers collect:SQL “SELECT” Some producers, the Registry and Schema make use of RDBMS as appropriate – but what is central is the relational model.

WP3 GridPP7 1/7/2003R-GMA Deployment4 Producer  Consumer Consumer can issue one-off queries –Similar to normal database query Consumer can also start a continuous query –Requests all data published which matches the query Can be seen as an alert mechanism

WP3 GridPP7 1/7/2003R-GMA Deployment5 3 Kinds of Query Tuple StreamProducer HistoryProducer LatestProducer Tuple

WP3 GridPP7 1/7/2003R-GMA Deployment6 Producers DataBaseProducer – Supports History Queries –Information not lost –Supports joins –Clean up strategy StreamProducer – Supports Continuous Queries –In memory data structure –Can define minimum retention period ResilientStreamProducer – Supports Continuous Queries –Like the StreamProducer but won’t lose data if system crashes –So slightly slower LatestProducer – Supports Latest Queries –Just holds the latest information for any “primaryish” key –Supports joins CanonicalProducer – Supports anything –Offers anything as relations

WP3 GridPP7 1/7/2003R-GMA Deployment7 Archiver (Re-publisher) It is a combined Consumer-Producer You just have to tell it what to collect and it does so on your behalf Re-publishes to any kind of “Insertable” (i.e. not to the CanonicalProducer) –Can support joins if archiving to a DataBaseProducer or a LatestProducer

WP3 GridPP7 1/7/2003R-GMA Deployment8 Functionality - mediator Queries posed against a virtual data base The Mediator must: –find the right Producers –combine information from them Hidden component – but vital to R-GMA Now supports Joins!

WP3 GridPP7 1/7/2003R-GMA Deployment9 Topologies Normally publish via a StreamProducer Archivers instantiated with a Producer and a Predicate Must avoid cycles in the graph A SP A A LP A HP SP A

WP3 GridPP7 1/7/2003R-GMA Deployment10 edg-rgma show tables describe Service show producers of Service latest select * from Service old continuous select * from Service Now produces decent error messages as it uses new C API internally.

WP3 GridPP7 1/7/2003R-GMA Deployment11 GIN and GOUT (Gadget IN and Gadget OUT) R-GMA Consumers LDAP InfoProvider GIN LDAP Server LDAP InfoProvider Stream Producer GIN Consumer (CE) Consumer (SE) Consumer (SiteInfo) RDBMS Latest Producer GOUT ConsumerA PI Archiver (LatestProducer) Stream Producer R-GMA GLUE Schema

WP3 GridPP7 1/7/2003R-GMA Deployment12 WP3 testbed Asked to create a WP3 testbed. –Integration testing before releasing to EDG Dev. –Building a complete testbed (including RB) Significant effort –Impacts a number of sysadmins –Hope to get a good return Bristol, Glasgow, Imperial College, NIKHEF, RAL x3

WP3 GridPP7 1/7/2003R-GMA Deployment13 Performance Testing Document – based on criteria from Steven Burke –No of sites R-GMA can scale to. –Availability of information –Accuracy of information –Propagation speed –etc. Analyzing code using OptimizeIt. –Looking at bottlenecks in code. –Memory, CPU usage etc. –Increasing performance of the code. Reducing huge number of threads in Gout Optimising and simplifying Gin

WP3 GridPP7 1/7/2003R-GMA Deployment14 Resilience Testing Looking at the Resilience of R-GMA. –Random Reboots, Power Cuts, network glitches etc. Developing a resilience testing framework. –Automating testing of different components. Will test the fail-over mechanisms. –Test registry replication. Will lead to more robust code.

WP3 GridPP7 1/7/2003R-GMA Deployment15 Current Status Jobs have been submitted on Dev testbed Minor bugs appear less frequently –Mostly fixed very quickly. Can support simulated sites

WP3 GridPP7 1/7/2003R-GMA Deployment16 Immediate Future Bringing it to the standards of the “performance document” and demonstrating it Security Increased resilience –Multiple registry instances –Multiple schema instances