1. 1 DocFlow - kick off - 2007 - Monitoring 1 Distributed Monitoring in P2P Systems Serge Abiteboul, Bogdan Marinoiu INRIA-Futurs and Univ. Paris 11

2. 2 DocFlow - kick off - 2007 - Monitoring 2 Organization Motivation An algebra over XML streams The P2PMonitor system Optimization Conclusion

3. 3 Motivation

4. 4 DocFlow - kick off - 2007 - Monitoring 4 P2P systems Peer-to-peer: A large and varying number of computers cooperate to solve some particular task without any centralized authority seti@home; kazaa; cabal Interactivity between the peers (Web 2.0) Content sharing communities Distributed applications (e.g., games) New services

5. 5 DocFlow - kick off - 2007 - Monitoring 5 P2P systems Benefit from technologies like XML and Web services Complex and hard to control systems Many peers Peers are distributed Peers are autonomous Peers are often unreliable and selfish Peers come and leave Intense communications between peers Content changes rapidly Goal: monitor such systems

6. 6 DocFlow - kick off - 2007 - Monitoring 6 Why monitor P2P systems? P2P Monitoring, e.g., supply chain or manufacturing system Web Intelligence: e.g., business intelligence and surveillance of competing companies P2P Security: e.g., against intrusion, spamming P2P optimization and tuning: e.g., gather statistics, control indexing P2P system administration: e.g., choose where to log or control replication P2P error management: e.g., error detection, error, diagnosis Business: e.g., billing for using resources, procurement

7. 7 An algebra over XML streams

8. 8 DocFlow - kick off - 2007 - Monitoring 8 XML strea m Monitoring one peer - Alerters Detect events at the peer level Data changes Database updates Events in RSS feed Web page changes Web service calls Out call In call Each event is represented as an XML document XML Stream

9. 9 DocFlow - kick off - 2007 - Monitoring 9 An algebra over streams Efficient Filter (can provide many selections simultaneously) Binary operators: union, join, etc. Operator with memory: duplicate elimination Same plans meaningful for query processing or monitoring  = outer join &  =  s=“Lhasa”  s1 s2s3s4    s5 

10. 10 DocFlow - kick off - 2007 - Monitoring 10 ActiveXML algebra in short Used both for query and monitoring plans Local computation - Algebra for queries over trees: Pick your favorite Has to work on streams Manage replication Generic data and services Glue Send, receive, eval Essential Include service calls – typically query calls Exchange streams of ActiveXML Algebraic Expressions

11. 11 DocFlow - kick off - 2007 - Monitoring 11 AXML strea m AXML stream operator r1 t s t s t s r r1@p1 Music@p3 Music@p2 a Peer p1 Q “Easy case” Monotone in-stream Monotone query Stream of ActiveXML

12. 12 P2P Monitor system

13. 13 DocFlow - kick off - 2007 - Monitoring 13 Stream specification language for $c on local: outCOM let $timeCall := $c.call.time and $duration := $c.response.time - $timeCall where $c.call.method = "GetTemp“and $duration > 300 and $c.call.site = "http://meteofrance.fr" return {$timeCall} {$duration}

14. 14 DocFlow - kick off - 2007 - Monitoring 14 Compiled into a monitoring plan Compile the specification into a monitoring plan Some alerters produce input streams Stream operators Evaluate it The monitoring plan is distributed

15. 15 DocFlow - kick off - 2007 - Monitoring 15 What to do with the result streams Publish a stream RSS, email, web site Turn a stream into a channel Users can subscribe to it Store stream – distributed tracing Provide traces of the computation Basis for querying a run Basis for analyzing a run: diagnosis

16. 16 DocFlow - kick off - 2007 - Monitoring 16 Example: monitoring involving many peers For $com1 on http://peer.a.com :outCOM | http://peer.b.com :outCOM,http://peer.a.comhttp://peer.b.com $com2 on http://meteo.fr : inCOMhttp://meteo.fr Let $duration := $com1.response.timestamp - $com1.call.timestamp Where $duration > 10 and $com1.call.method = « getTemp » and Filter $com1.call.site = http://meteo.fr andhttp://meteo.fr $com1.call.id = $com2.call.id Join Return Report($com1, $com2) By email to service@meteo.fr; A B M

17. 17 DocFlow - kick off - 2007 - Monitoring 17 Monitoring Task Global Plan alerter@A Filter alerter@B Filter ∪ Join alerter@M Reporter Publisher

18. 18 DocFlow - kick off - 2007 - Monitoring 18 Distributed Monitoring Task alerter@A Filter@A alerter@B Filter@B ∪ @B Join@M alerter@M Reporter@M Publisher@M Publisher@A Publisher@B Peer A: For $c on local:outCOM Let $duration = … Where. Return $c By publish as channel X Peer B: For $c on local:outCOM | X Let $duration = … Where …. Return $c By publish as channel Y Peer M: For $c1 on local:inCOM, $c2 on Y Where $c1.call.id = $c2.call.id Return Report($c1, $c2) By email to service@meteo.fr Y X

19. 19 Optimization

20. 20 DocFlow - kick off - 2007 - Monitoring 20 Optimize the plan Distribute the work among the peers Try to place computation close to data if possible Try to reduce redundancy

21. 21 DocFlow - kick off - 2007 - Monitoring 21  = outer join &  =  s=“Lhasa”  r1[t,s] r2[s,at]r3[t,s]r4[t,s]    r1[t,s] r3[t,s]r4[t,s]     r1[t,s] r3[t,s]r4[t,s]     r5[t,s]       monitoring plan (a) monitoring plan (b) monitoring plan (c) Optimization based on ActiveXML algebra

22. 22 Conclusion

23. 23 DocFlow - kick off - 2007 - Monitoring 23 The plan streams Specify monitoring Generate data streams Store traces Query traces Mine traces (e.g., diagnosis)

24. 24 DocFlow - kick off - 2007 - Monitoring 24 Merci