RELATIONAL GRID MONITORING ARCHITECHTURE WP3 Data RELATIONAL GRID MONITORING ARCHITECHTURE An information and monitoring system for static and dynamic information about grid resources, applications, networks … Soft State Registration within the Servlet Based R-GMA Implementation Servlet aware of API during terminationInterval duration Existence of service instances in Registry up to terminationTime Producer API Producer instance Producer registration Registry API RDBMS HTTP Servlet Technology Implementation GMA Producer Servlet Producer Registry Servlet Sensor register The Grid environment needs to be aware of the existence of Producer and Consumer instances. The service result in the creation of a service instance at the servlet side. This state needs to be tidied up if the service disappears. Both Producers and Consumers are also registered within the Registry database. There are two categories of state management in our system, a servlet's awareness of an services communication with a service instance and the registering of the existence of service instances within the grid. Lifecycle management is modelled around the OGSA specification. The service has a termination interval associated with it. The servlet counts down a proportion of the termination interval and when it hears from the service within that time interval, it resets the termination time of the service instance to the current time + termination interval. If the servlet counts down the full termination interval it will remove all traces of the service instance and inform the registry to tidy itself up. In order to allow for network delays and reduce network load, the servlet resets termination times with the registry before the service termination interval expires, but not after every time the service contacts the service instance. Registry transfer data Hidden components Client side Consumer lookup DBProducer Servlet RDBMS DataBaseProducer API In the GMA of the Global Grid Forum(GGF) Producers register themselves with the Registry and describe the type and structure of information they want to make available to the Grid. Consumers can query the Registry to find out what type of information is available and locate relevant Producers. The Consumer can then contact the Producers directly to obtain the data. By specifying the Consumer/Producer protocol and the interfaces to the Registry one can build inter-operable services. LatestProducer API LatestProducer Servlet Registry API Sensor Registry API Registry Servlet RDBMS Consumer API LatestProducer API Schema API Registry API Application code Consumer Servlet RDBMS Registry/Schema Replication SchemaServlet Command R-GMA Registry1 Registry2 Producer1 Producer2 Data Info mastered by Registry1 Info mastered by Registry2 R-GMA is a relational implementation of the Grid Monitoring Architecture (GMA). Producers: A Producer is instantiated with the description of the information it has to offer specified by an SQL CREATE TABLE statement and a WHERE clause expressing a predicate that is true for the table. An instance of the producer is then created within the corresponding servlet and registered with the Registry/Schema. The Registry holds a table identifier for each table and a representation of the WHERE clause. The detailed information about the tables - what columns they have and of what type - is held in the Schema. To publish data, a method is invoked which takes the form of a normal SQL INSERT. Consumers: A Consumer handles a single query, expressed as an SQL SELECT statement. The table identifier and the WHERE clause of the Consumer query is stored in the Registry. Copy of info from Registry2 Copy of info from Registry1 A peer-to-peer policy for registry replication is adopted within R-GMA. Each registry is responsible for replicating its local data with other registries. Registry1 ensures Registry2 has a copy of all information it masters and similarly Registry2 ensures Registry1 has a copy of all information it masters. A checksum is sent between registries to validate replicated data. A failed checksum forces all replication data to be resent. Each registry maintains a list of the locations of other registries. Data that has been replicated is flagged to prevent duplication or re-replication. Deleted data is not sent for replication, lifetime management takes care of this. Any columns relating to time stamps are not replicated. Each type of Producer or Consumer API communicates with a corresponding servlet. A single servlet can serve many API instances. This avoids needing a large number of servers but offers great flexibility as we can always add more servers if necessary to balance the load. Access to servlets is via HTTP GET/POST using a thin client API. The response from the servlets is in the form of an XML string which conforms to an XML schema definition and contains either a custom ResultSet or an exception. LDAP Info Providers >> R-GMA >> LDAP Producer API From Servlets to Grid Services Producer Servlet Sensor Code Registry API Example producer ‘CPULoad' Country Site Facility Load Timestamp UK RAL CDF 0.3 19055711022002 ATLAS 1.6 19055611022002 GLA 0.4 19055811022002 ALICE 0.5 CH CERN 0.9 0.6 19055511022002 Work on an OGSA-compliant implementation of R-GMA is in progress. Producer and Consumer servlets will be replaced by Grid service factories which will then create the Producer and Consumer instances. Lifecycle management will be handled using OGSA mechanisms. The registry will include a HandleResolver. Consumers will use OGSA notification in order to keep track of the set of Producers that are of interest. Initially SQL will be used as Service Data Element query language, but XML based query languages (XPath, XQuery) can be adopted if needed. The first stages of this process are currently implemented and involved isolating logic from the servlet infrastructures, separating the core instance logic from the creation and tracking of these instances. Actions outstanding: Interoperability with GT3. Our Registry compatibility with OGSA registries. Replication of Registry/Schema services within OGSA. Additional Query Languages e.g. XQuery, XPath Use of Notification Mechanisms for information flow. Registry Servlet Schema API Producer API Registry API R-GMA RDBMS Producer API Archiver API Application Code Schema Servlet Consumer API Consumer Servlet ConsumerAPI LDAP Producer Factory Consumer Instance Schema Registry Sensor Producer API Application Code ConsumerAPI GIN GOUT LDAP InfoProvider By providing a mapping between LDAP ObjectClasses and relational tables it is possible to publish from MDS information provider scripts to R-GMA and publish from R-GMA to an LDAP server. LDAP multivalue attributes are mapped onto separate tables. Example consumer: SELECT * FROM cpuLoad WHERE country = ’UK’ AND site = ’RAL’ http://hepunx.rl.ac.uk/edg/wp3