Download presentation
Presentation is loading. Please wait.
Published byClaude Horatio Owen Modified over 9 years ago
1
1 Distributed Monitoring of Peer-to-Peer Systems By Serge Abiteboul, Bogdan Marinoiu Docflow meeting, Bordeaux
2
2 2 Outline The Monitoring Problem & Approach A language for specifying monitoring tasks: P2PML P2PMonitor System ActiveXML Stream Algebra Architecture of P2PMonitor Monitoring Plan Generation & Query Rewriting Focus on Filtering Reusing running tasks Work in progress
3
3 3 The Monitoring Problem P2P systems are: a popular support for content sharing communities, distributed applications highly dynamic (intense communications, content changing rapidly, peers come/leave) and difficult to observe Observation is important ! error management & diagnosis statistics gathering & optimization issues : the « busiest » peer in a network business applications : billing & quality of service Web surveillance
4
4 4 Is it possible to observe & analyse a P2P system ? Difficult (if not impossible) in a centralized way Yes, in a distributed manner
5
5 5 XML strea m Approach Detect events at the (monitored) peer level Data changes, Web service calls -> alerters Each event is represented as an XML document XML Stream (distributed) XML stream processing system XML Streams are published
6
6 6 Outline The Monitoring Problem & Approach A language for specifying monitoring tasks: P2PML P2PMonitor System ActiveXML Stream Algebra Architecture of P2PMonitor Monitoring Plan Generation & Query Rewriting Focus on Filtering Reusing running tasks Work in progress
7
7 7 P2PML statement structure XQuery FLWR flavour For – maps streams to XML variables Let – assigns new XML variables Where – imposes conditions on events (filtering and join criteria) Return – generates reports / restructures XML By – specifies publication means : in channels for inside system publication e-mails, Web pages, RSS feeds for outside system publication
8
8 8 P2PML statement example for $c on local: outCOM let $timeCall := $c.call.time and $duration := $c.response.time - $timeCall where $c.call.method = “GetTemp” and $duration > 10 and $c.call.site = "http://meteofrance.fr" return {$timeCall} {$duration} by channel “QoS:Alerts”
9
9 9 Outline The Monitoring Problem & Approach A language for specifying monitoring tasks: P2PML P2PMonitor System ActiveXML Stream Algebra Monitoring Plans Architecture of P2PMonitor Focus on Filtering Reusing running tasks Work in progress
10
10 ActiveXML Stream Algebra is the support for monitoring plan representation and the basis for the its optimization : Distribute the work among the peers Try to place computation close to data if possible Try to reduce redundancy
11
11 Scenario P2PMLQueries XML streams
12
12 Monitoring Plans
13
13 Architecture of P2PMonitor(1) Subscription Manager Alerters (WS Alerter, Database Alerter, RSS Alerter) Stream Processors Without « storage »: Filter, Restructure, Union With « storage »: Join, Group-By, Duplicate Removal Publishers E-mail, WebPage, RSS Channel Publisher : a user or another peer may subscribe to it
14
14 Architecture of P2P Monitor(2)
15
15 Outline The Monitoring Problem & Approach A language for specifying monitoring tasks: P2PML P2PMonitor System ActiveXML Stream Algebra Monitoring Plans Architecture of P2PMonitor Focus on Filtering Reusing running tasks Work in progress
16
16 Focus on Filtering Filtering is a crucial operator in stream processing ! E.g., Many users might be interested in events coming from the same source / alerter : bottleneck hazard Our approach to the problem : two-stage filtering Reasons: Attributes of XML document’s root reflect the most important properties of an event The event’s details can be given intentionnally (ActiveXML style)
17
17 Two-stage filtering A subscription is viewed as a conjunction of simple conditions (e.g., « attribute » = « value ») and « more difficult » XPath queries 1st data structure regroups the (ordered) simple conditions of all the subscriptions by commonalities (Atomic Event Set structure) 2nd data structure regroups XPath queries of all the subscriptions (path – based indexing YFilter style – using NFA) On a XML document: 1st stage: read the root, evaluate AES, detect the « difficult » XPath queries that remain to be evaluated 2nd stage (if needed): adapt the second structure and evaluate the set of XPath queries on the body of the XML document (if necessary execute Web service calls). The output is the set of the subscriptions « hit » by the XML document
18
18 Outline The Monitoring Problem & Approach A language for specifying monitoring tasks: P2PML P2PMonitor System ActiveXML Stream Algebra Monitoring Plans Architecture of P2PMonitor Focus on Filtering Reusing streams / running tasks Work in progress
19
19 Reusing running tasks Optimization by trying to avoid redundancy Before building new operators (and streams), try to discover useful ones Stream representation in XML: Stream Definition Database – description of available streams Distributed, not centralized (avoid bottlenecks) Implemented using KadoP – index and repository system over a DHT
20
20 Stream replication and equivalence(1) Streams can be replicated between peers With two similar operators on two replicas of the same stream, we obtain two equivalent streams Replication can be represented in the Stream Definition Database Stream Equivalence is difficult to detect
21
21 Algorithm for discovering useful streams It uses XPath queries on the Stream Definition Database: E.g. for identifying the output stream of alerter inCOM : /Stream[@PeerId=$p1][Operator/inCOM]->(S 1, P 1 ) It goes bottom-up on the query tree E.g., Join P (σ F (inCOM@P 1 ), outCOM@P 2 ) (S 5, P 1 )
22
22 Outline The Monitoring Problem & Approach A language for specifying monitoring tasks: P2PML P2PMonitor System ActiveXML Stream Algebra Monitoring Plans Architecture of P2PMonitor Focus on Filtering Reusing streams / running tasks Work in progress
23
23 Work in progress (1) Link with Incremental View Maintenance Defining a monitoring task by a tree-pattern query on an active document with streams - powerful way of expressing complex monitoring tasks (difficult to express directly in P2PML)
24
24 Work in progress (2) Introducing explictly the « time » in P2PML -possible impact on P2PMonitor performance (reactivity) and resource consumption (needed storage) E.g. for $e1 on P1:inCOM, $e2 on P2:outCOM where $e1.timeEvent > $e2.timeEvent +25 … Queries on traces obtained by P2PMonitor – diagnosis, detecting patterns of evolution for the monitored system E.g. Trace = I 1,I 2 …I n - instances of a document For each new order detected in instance I k, there is a payment present in one of the following instances
25
25 Thank you very much!
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.