1 Serge Abiteboul - Monitoring 1 Monitoring of distributed applications (in P2P) Serge Abiteboul, Pierre Bourhis, Bogdan Marinoiu, INRIA Saclay and Université Paris 11

2 Serge Abiteboul - Monitoring 2 Organization The context: the AXML program Motivation for monitoring The architecture Axlog Example Conclusion

3 The context: the AXML program Serge Abiteboul - Monitoring 3

4 Distributed data management with AXML Active XML: XML document with embedded function calls Intentional & dynamic information In pull or push mode Object & Active data AXML store AXML peer/V2 persistence, activation of local data AXML query processor and optimizerOptimax Distributed query plans AXML Business artifacts: verification AXML monitorP2Pmonitor Distributed monitoring Serge Abiteboul - Monitoring 4

5 Motivation

6 Serge Abiteboul - Monitoring 6 With the Web, more and more distribution Often distributed applications are very dynamic Content change rapidly Intense communications Complex and hard to control systems Many peers Peers are distributed Peers are autonomous Peers are sometimes unreliable and selfish Peers sometimes come and leave Goal: monitor such systems & support active features ala active databases

7 Example: Dell supply chain Serge Abiteboul - Monitoring 7 Shipping Co. Bank Customer Webstore Bank Plant Revolver Shipping Co. Suppliers Monitor Global supervisor Monitoring of distributed information manufacturing system

8 Serge Abiteboul - Monitoring 8 More examples Business applications Gather information for billing, bug tracing… Procurement Web Intelligence: e.g., business intelligence and surveillance of competing companies Security surveillance against intrusion, spamming P2P applications Optimization and tuning: e.g., gather statistics, control indexing Error management, detection, diagnosis

9 The architecture

10 Architecture Serge Abiteboul - Monitoring 10 publishers Alerters Streams Stream processors actions RSS

11 Serge Abiteboul - Monitoring 11 Alerter: monitor a single system Detect events at the peer level Depend on the peer that is monitored Monitor what? Database updates Events in RSS feed Web page changes WS calls (in/out) Each event is represented as an XML document Active stream Active stream

12 Active streams Monotone case Stream of XML documents (possibly Active XML) Nonmonotone case Think of the stream as defining a forest We can send updates to the previous data, typically deletions Insert [12,jean,1300], [11,marie,1555] Insert [3,zoe,1333] Delete [Id=12]; Insert [5,noé,1111] In the style of RSS feeds Streams are published as channel where it is possible to subscribe They are implement as Web service calls Serge Abiteboul - Monitoring 12

13 Stream processors Efficient Filter - can support a large number of selections simultaneously Binary operators: union, join, etc. Operator with memory: duplicate elimination Aggregation operator Publisher XML page, RSS, , Web page Novelty: axlog (axlog subscription & axlog engine) Serge Abiteboul - Monitoring 13

14 Serge Abiteboul - Monitoring 14 Algebraic (monitoring) plans ActiveXML algebra: same algebra as for distributed query processing Algebra over streams of (A)XML documents To send just some tree T: T, eos From ActiveXML: XML + embedded service calls A service may be any query The algebraic glue: send, receive, eval

15 Axlog (axml & datalog)

16 Axlog principle = Doc + query Incoming streams The outgoing stream is defined by a query Q (tree pattern + join) Each time an incoming message arrives, it modifies the document Possibly the query result An outgoing message is generated Incremental view maintenance Serge Abiteboul - Monitoring 16 Q Active XML document

17 Axlog - continued We have implemented an axlog engine We use datalog to benefit from 1. Incremental view maintenance in datalog – Δ technique 2. Query optimization in datalog – Magic set We have developed specific techniques 1. 3-valued based (true, false, possibleInFuture) 2. Avoid some derivation that Magic would do – preprocessing 3. Precompute in an optimistic manner 4. Garbage collection Serge Abiteboul - Monitoring 17

18 Example Serge Abiteboul - Monitoring 18

19 Example Some process (e.g.,, BPEL workflow mailOrder) may be initiated at a webStore in US or France The last operation of the process (e.g., shipping) occurs on Shipping – we are interested in shipping from UK We want to detect when one mailOrder took more that 5 days to process when the customer is a Premium customer Easy case Monotone in-stream Monotone query No time constraint No aggregation Serge Abiteboul - Monitoring 19

20 Local alerter specifications On webStore US and webStore FR for $call in(WS-outCall) where $call/method = “mailOrder“ return $call/ID $call/time On shipping UK for $call in(WS-outCall) where$call/method = “shipping“ return $call/ID $call/time Serge Abiteboul - Monitoring 20

21 Axlog specification AXMLmonitor.axml Queryfrom supervisor in India For $s in doc(“monitor.axml”)/starts/start, $e in doc(“monitor.axml”)/ends/end Where $s/Id = $e/Id and $s//Premium and ($e/time - $s/time) > 1800 and Return $s, $e Serge Abiteboul - Monitoring 21

22 Serge Abiteboul - Monitoring 22 Monitoring task plan σ σ Join Reporter Publisher ∪ σ Emai lto India

23 Serge Abiteboul - Monitoring 23 Monitoring task plan – localisation wsAlerter σ wsAlerter σ Join Reporter Publisher Publish Site US Site FR Site UK channel Publish ∪ σ wsAlerter to India

24 Conclusion Prelim paper in International Workshop on Web Information and Data Management 2007 Demo in ICDE08 On going Temporal queries Negation in queries Deletion Related work on Mashups with Tel Aviv Univ. & IBM Haifa Purely relational Serge Abiteboul - Monitoring 24

25 Serge Abiteboul - Monitoring 25 Merci