1. RELATIONAL GRID MONITORING ARCHITECHTURE
WP3
Data
RELATIONAL GRID MONITORING ARCHITECHTURE
An information and monitoring system for static and dynamic information about grid resources, applications, networks …
Soft State Registration within the Servlet Based R-GMA Implementation
Servlet aware of API during terminationInterval duration
Existence of service instances in Registry up to terminationTime
Producer API
Producer
instance
Producer
registration
Registry
API
RDBMS
HTTP Servlet Technology Implementation
GMA
Producer Servlet
Producer
Registry
Servlet
Sensor
register
The Grid environment needs to be aware of the existence of Producer and Consumer instances. The service result in the creation of a service instance at the servlet side. This state needs to be tidied up if the service disappears. Both Producers and Consumers are also registered within the Registry database. There are two categories of state management in our system, a servlet's awareness of an services communication with a service instance and the registering of the existence of service instances within the grid. Lifecycle management is modelled around the OGSA specification. The service has a termination interval associated with it. The servlet counts down a proportion of the termination interval and when it hears from the service within that time interval, it resets the termination time of the service instance to the current time + termination interval. If the servlet counts down the full termination interval it will remove all traces of the service instance and inform the registry to tidy itself up. In order to allow for network delays and reduce network load, the servlet resets termination times with the registry before the service termination interval expires, but not after every time the service contacts the service instance.
Registry
transfer
data
Hidden components
Client side
Consumer
lookup
DBProducer Servlet
RDBMS
DataBaseProducer
API
In the GMA of the Global Grid Forum(GGF) Producers register themselves with the Registry and describe the type and structure of information they want to make available to the Grid. Consumers can query the Registry to find out what type of information is available and locate relevant Producers. The Consumer can then contact the Producers directly to obtain the data. By specifying the Consumer/Producer protocol and the interfaces to the Registry one can build inter-operable services.
LatestProducer
API
LatestProducer Servlet
Registry API
Sensor
Registry API
Registry Servlet
RDBMS
Consumer API
LatestProducer
API
Schema API
Registry API
Application code
Consumer Servlet
RDBMS
Registry/Schema Replication
SchemaServlet
Command
R-GMA
Registry1
Registry2
Producer1
Producer2
Data
Info mastered by Registry1
Info mastered by Registry2
R-GMA is a relational implementation of the Grid Monitoring Architecture (GMA).
Producers: A Producer is instantiated with the description of the information it has to offer specified by an SQL CREATE TABLE statement and a WHERE clause expressing a predicate that is true for the table. An instance of the producer is then created within the corresponding servlet and registered with the Registry/Schema. The Registry holds a table identifier for each table and a representation of the WHERE clause. The detailed information about the tables - what columns they have and of what type - is held in the Schema. To publish data, a method is invoked which takes the form of a normal SQL INSERT.
Consumers: A Consumer handles a single query, expressed as an SQL SELECT statement. The table identifier and the WHERE clause of the Consumer query is stored in the Registry.
Copy of info from Registry2
Copy of info from Registry1
A peer-to-peer policy for registry replication is adopted within R-GMA. Each registry is responsible for replicating its local data with other registries. Registry1 ensures Registry2 has a copy of all information it masters and similarly Registry2 ensures Registry1 has a copy of all information it masters. A checksum is sent between registries to validate replicated data. A failed checksum forces all replication data to be resent. Each registry maintains a list of the locations of other registries. Data that has been replicated is flagged to prevent duplication or re-replication. Deleted data is not sent for replication, lifetime management takes care of this. Any columns relating to time stamps are not replicated.
Each type of Producer or Consumer API communicates with a corresponding servlet. A single servlet can serve many API instances. This avoids needing a large number of servers but offers great flexibility as we can always add more servers if necessary to balance the load. Access to servlets is via HTTP GET/POST using a thin client API. The response from the servlets is in the form of an XML string which conforms to an XML schema definition and contains either a custom ResultSet or an exception.
LDAP Info Providers >> R-GMA >> LDAP
Producer
API
From Servlets to Grid Services
Producer
Servlet
Sensor
Code
Registry
API
Example producer ‘CPULoad'
Country
Site
Facility
Load
Timestamp UK
RAL
CDF
0.3
ATLAS
1.6
GLA
0.4
ALICE
0.5 CH
CERN
0.9
0.6
Work on an OGSA-compliant implementation of R-GMA is in progress. Producer and Consumer servlets will be replaced by Grid service factories which will then create the Producer and Consumer instances. Lifecycle management will be handled using OGSA mechanisms. The registry will include a HandleResolver. Consumers will use OGSA notification in order to keep track of the set of Producers that are of interest. Initially SQL will be used as Service Data Element query language, but XML based query languages (XPath, XQuery) can be adopted if needed.
The first stages of this process are currently implemented and involved isolating logic from the servlet infrastructures, separating the core instance logic from the creation and tracking of these instances.
Actions outstanding:
Interoperability with GT3.
Our Registry compatibility with OGSA registries.
Replication of Registry/Schema services within OGSA.
Additional Query Languages e.g. XQuery, XPath
Use of Notification Mechanisms for information flow.
Registry
Servlet
Schema
API
Producer
API
Registry
API
R-GMA
RDBMS
Producer API
Archiver API
Application
Code
Schema
Servlet
Consumer
API
Consumer
Servlet
ConsumerAPI
LDAP
Producer
Factory
Consumer
Instance
Schema
Registry
Sensor
Producer API
Application Code
ConsumerAPI
GIN
GOUT
LDAP
InfoProvider
By providing a mapping between LDAP ObjectClasses and relational tables it is possible to publish from MDS information provider scripts to R-GMA and publish from R-GMA to an LDAP server. LDAP multivalue attributes are mapped onto separate tables.
Example consumer:
SELECT * FROM cpuLoad WHERE country = ’UK’ AND site = ’RAL’