1 MSc Project Yin Chen Supervised by Dr Stuart Anderson 2003 Grid Services Monitor Long Term Monitoring of Grid Services Using Peer-to-Peer Techniques

2 Content ● ● Requirements ● ● Architecture ● ● Design & Implementation ● ● Implementation Outcome

3 Grid Services Monitor Requirements …

4 Requirements ● ● Monitoring grid services behaviours ● ● Providing long-term historical information ● ● Reporting to an end-user or a prediction model ● ● Helping make prediction of services behaviours

5 Requirements ● ● The system must be fault tolerant ● ● Should allow nodes to join and leave dynamically ● ● Must be able to scale with the grid ● ● Monitoring data must be distributed

6 Architecture … Grid Services Monitor

7 Architecture ● ● Peer-to-Peer architecture ● ● Why P2P ? - - Decentralized fashion - - Much scalability - - Better fault tolerance ● ● Traditional P2P system designed for file sharing and storing ● ● This project focuses on decentralized data querying and retrieval

8 System Architecture Monitor Sensor Data Store Local Collector Collector Monitor Sensor Data Store Local Collector Requester Listener AListener B

9 Topology Peer Group A Peer Group B Peer Group C

10 Pull Model ● ● Data delivered by Pull ● ● Push model: the listener sends out notifications to other peers ● Advantages : - - Less network traffic: data deliver only when necessary - - Has NO time synchronisation problem: collect data from resources at the same time - - The requester determines query conditions, data type etc., making data operation easier

11 Response Mode

12 Query Planning Send Union Receive Listener AListener N Requester Local Query Merge Query Receive …

13 Design & Implementation … Grid Services Monitor

14 Design & Implementation ● ● Monitor Sensor ● ● P2P Transport Mechanism ● ● Decentralized Data Storage and Querying

15 Monitor Sensor ● ● Implement GT3 ServiceLifecycleMonitor Create – when an instance of service is going to be created Destroy – when the instance is going to be destroyed PreCall – the service is going to be invoked PostCall – service invocation has finished PreSerializationCall – input parameters are going to be desterilized PostSerializationCall – input parameters have been desterilized ● ● Problem: can NOT report failure of the service instance ● ● Solution: use timeout

16 P2P Transport Mechanism ● ● Based on JXTA platform ● ● Queries are scoped to one peer group ● ● A requester multicasts a request message, all listeners response by sending back requested data ● ● Controlling the messages not to flood the network - - Data delivered by Pull - - Reduce the size of messages: Local collector precooked data - - Reduce the volume of messages ● ● Timeout: to finish a data collection process

17 P2P Transport Mechanism Monitor Sensor ListenerA Data Store ListenerC Requester Data Store Peer Group Local Collector Collector Output Pipe Input Pipe Output Pipe Input Pipe Output Pipe ListenerB

18 Data Storage & Querying ● ● Using XML file : light weight and platform independence ● ● Decentralized manner: Insert handled by Monitor Sensor of each Listener Retrieve by Local Collector of each Listener Data join by Collector of Requester ● ● Data operations: Insert : inserts a ChildNode at the root of log file Query : XPath Join : adds PeerID as a part of primary key to avoid primary key replication problems

19 Implementation Outcome … Grid Services Monitor

20 Generating Monitoring Data - Gt3 Server keeps tracing and monitoring Sample of log record A Grid Service having a monitor sensor on it

21 Collecting Monitoring Data ListenerA RequesterA> request Finish calculating data RequesterB> request Finish calculating data ListenerB RequesterA> request Finish calculating data RequesterB> request Finish calculating data ListenerC RequesterA> request Finish calculating data RequesterB> request Finish calculating data RequesterA ListenerA>I got it ListenerC>I got it ListenerB>I got it Finish collecting data Data collected from 3 peers RequesterB ListenerB>I got it ListenerC>I got it Finish collecting data Data collected from 2 peers

22 Displaying Data In Graphical Layout Service name Number of responding peers Key X Axis indicates the duration of the data Y Axis indicates successful or total running times Line of successful running times Line of total running times Marks on the line indicate the values

23 Displaying Data In Table Layout Service nameNumber of responding peers Success time The date of period Total running time

24 Comparing Different Services

25 Grid Services Monitor Thanks !