Querying and Monitoring Distributed Business Processes Daniel Deutch, Tova Milo Tel-Aviv University ERP HR System eComm CRM Logistics Customer Bank Supplier

Querying and Monitoring Distributed BPs Data model Design Query language Streams ... Select… From… Where… Students XML SOAP WSDL ... Optimization Indexing Transactions Files organization Distribution ... Students Takes sid=sid sname name=“Mary” cid=cid Courses Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to Business Processes Querying Monitoring Summary & Research Directions Outline Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to Business Processes Querying Monitoring Summary & Research Directions Outline Querying and Monitoring Distributed BPs

What is a Business Process? Introduction to BPs What is a Business Process? Logically related activities that, when combined in a flow, achieve a business goal. Activities may either be local or remote Operates in a cross-organization, distributed environment Abstract representation, independent of implementation Standards facilitate design, deployment, and execution Querying and Monitoring Distributed BPs

Programming In the large Introduction to BPs Programming In the large BPs are designed by Non-programmers Specify combination of functionalities and flow thereof, to solve a complex problem Example: process an order The operations/functions are implemented by programmers (programming in the small) Example: fetch order document Querying and Monitoring Distributed BPs

Abstract Process Executable Process Abstract vs. Executable process Introduction to BPs Abstract vs. Executable process Abstract Process Defines the publicly visible behavior of services that an executable process offers Executable Process Contain the partner’s business logic behind an external protocol Querying and Monitoring Distributed BPs

Orchestration vs.Choreography Introduction to BPs Orchestration vs.Choreography Orchestration: Executable process, message exchange sequences are controlled by the orchestration designer. Choreography: Non-executable protocol for interactions. E.g., the legal sequences of messages exchanged, guaranteeing interoperability BPEL is an Orchestration language, not a choreography language (see Web Service Choreography).The primary difference between orchestration and choreography is executability and control. An orchestration specifies an executable process that involves message exchanges with other systems, such that the message exchange sequences are controlled by the orchestration designer. A choreography specifies a protocol for peer-to-peer interactions, defining, e.g., the legal sequences of messages exchanged with the purpose of guaranteeing interoperability. Such a protocol is not directly executable, as it allows many different realizations (processes that comply with it). A choreography can be realized by writing an orchestration (e.g. in the form of a BPEL process) for each peer involved in it. The orchestration and the choreography distinctions are based on analogies: orchestration refers to the central control (by the conductor) of the behavior of a distributed system (the orchestra consisting of many players), while choreography refers to a distributed system (the dancing team) without centralized control. Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to BPs Orchestration Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to BPs Web Services Meet BPs Local to company A Web Service 1 Web Service 4 Web Service 2 Web Service 5 Web Service 3 Web Service n At company B On the Web Company A business process Querying and Monitoring Distributed BPs

Web-Based Travel Agency BP Introduction to BPs Web-Based Travel Agency BP Travel request responses Airlines Websites queries Consolidate Results Confirmation responses queries Travel Service Hotels Websites Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to BPs BP Management Planning Modeling & Design Development & Deployment Execution, Interacting & Monitoring Analysis and Optimization We discuss mainly modeling, analysis, monitoring Querying and Monitoring Distributed BPs Source: Microsoft BPM Description

Querying and Monitoring Distributed BPs Introduction to BPs Modeling Challenges Complex Systems Distributed Settings Interoperability issues Robustness Scale Web interface Legacy systems Querying and Monitoring Distributed BPs

BP market (BPTrends survey) Introduction to BPs BP market (BPTrends survey) Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to BPs BP Management buzz Over 20 million hits in google for “business process” Over 5 million hits for "business process management" Vast interest by analysts (e.g. Gartner) Rapidly growing interest in industry New Standards Querying and Monitoring Distributed BPs

BPEL4WS 1.0 (IBM, Microsoft) Introduction to BPs Standards History BPML (Intallio et al) BPSS (ebXML) WSCI (Sun et al) WS-Choreography (W3C) BPEL4 PEOPLE (Oracle et. Al) 2000/05 2001/03 2001/05 2001/06 2002/03 2002/06 2002/08 2003/01 2003/04 2007/04 2007/06 XLang (Micorsoft) WSFL (IBM) WSCL (HP) BPEL4WS 1.0 (IBM, Microsoft) BPEL4WS 1.1 (OASIS) WSBPEL 2.0 (OASIS) BPEL Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to BPs Standards stack Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to BPs BPEL in a nutshell Language for specifying BP behavior based on Web Services (WS) Define BPs as coordinated sets of Web service interactions Define both abstract and executable processes Specifies Web services Composition Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to BPs BPEL constructs <receive> <reply> <invoke> <assign> <throw> <wait> <sequence> <flow> (parallel) <if> <while> <repeatUntil> … Communication-related constructs Flow-related constructs The emphasis at first was on the communication-related constrcuts. The flow-realtd constrcuts development is growing (towards programming-in-the-small abilities) Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to BPs BPEL as XML BPEL is XML-based In general we could use XML editors for specification design Infeasible in practice Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to BPs BPEL as XML (cont.) <process> <!– Definition and roles of process participants --> <partnerLinks> ... </partnerLinks> <!- Data/state used within the process --> <variables> ... </variables> <!- Properties that enable conversations --> <correlationSets> ... </correlationSets> <!- Exception handling --> <faultHandlers> ... </faultHandlers> <!- Error recovery – undoing actions --> <compensationHandlers>...</compensationHandlers> <!- Concurrent events with process itself --> <eventHandlers> ... </eventHandlers> <!- Business process flow --> (activities)* </process> Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to BPs BPEL as XML (cont.) <if> <condition> bpel:getVariableProperty('shipRequest', 'props:shipComplete') </condition> <sequence> <assign> <copy> <from variable="shipRequest" property="props:shipOrderID" /> <to variable="shipNotice" </copy> property="props:itemsCount" /> </assign> If (shipRequest = shipComplete) { shipNotice.OrderId = shipRequest.OrderId; shipNotice.itemsCnt = shipRequest.itemsCnt; } Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to BPs Basic activities Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to BPs Flow activities Turing-complete? Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to BPs BP specification Querying and Monitoring Distributed BPs

Travel Agency Process Flow Introduction to BPs Travel Agency Process Flow Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to BPs Zoom In Querying and Monitoring Distributed BPs

Example Editor (eclipse) Introduction to BPs Example Editor (eclipse) Querying and Monitoring Distributed BPs

Example editor (Oracle) Introduction to BPs Example editor (Oracle) Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to BPs Example editor (IBM) Querying and Monitoring Distributed BPs

Example editor (Microsoft VS) Introduction to BPs Example editor (Microsoft VS) Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to BPs Challenges So far: challenges & solutions for modeling BPs are hard to analyze, debug, and optimize Again, due to scale, distributed settings, legacy systems,….. Good modeling simplifies process specification But further analysis tools are required Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to BPs Analysis Types Static Analysis “What kind of credit services are used (in)directly?” “How can I buy a plane ticket?” “Can one get a price quote without giving first credit card info?” Monitoring “Notify me when a user hacks the system and get a price quote without giving his credit card info” Log Analysis “Find all logs where a user bought a plane ticket” Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to Business Processes Querying Monitoring Summary & Research Directions Outline Querying and Monitoring Distributed BPs

Querying Business Processes Querying BPs Querying Business Processes Statically analyze a Business Process “Find all ways in which a user can buy a plane ticket” Analysis needs Control flow analysis (Reachability, Cycle Detection, Temporal properties,…) Structural analysis Data analysis Messages Validation, Pointer analysis, Array bounds analysis,… Combined (data and flow) analysis Database approach Treat BPs as data Design a query language Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Querying BPs Why not XQuery? Recall that BPEL is XML-based, then… Why not use XQuery?! Similar arguments to BPs / XML Need to handle complex technical constructs Complex, unintuitive queries (many joins & recursion) No abstraction No robustness to changes in BPEL standard Querying and Monitoring Distributed BPs

An Ideal querying/analysis tool Querying BPs An Ideal querying/analysis tool Uniform Graphical Scalable Flexible Similar to the specification design Can handle partial information and uncertainty Querying and Monitoring Distributed BPs

Models & Query Languages Querying BPs Models & Query Languages Monadic Second Order Logic Context Free Graph Grammars Temporal Logic Recursive State Machine Mu-calculus CTL* First Order Logic Finite State Machine LTL CTL Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Querying BPs First Try Finite State Machines (FSM) for software specification Temporal Logic (TL) [Emerson,’90] for querying all possible behaviors Very common in software (and hardware) verification Typically Linear time evaluation (data complexity) Exponential time (query complexity) Querying and Monitoring Distributed BPs

Finite State Machines (FSM) Querying BPs Finite State Machines (FSM) States and transition function (typically no “accepting” state) The system configuration is encoded within the states Interested in properties of possible traversal over the states Temporal Logics express such properties Finite State Machines (FSM) Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Querying BPs FSM (example) “Flat” No functions No recursion Cycles allowed Execution path by traversal Login Search Reserve Payment Cash Credit Confirm Cancel Querying and Monitoring Distributed BPs

Temporal Logic - Ingredients Querying BPs Temporal Logic - Ingredients Predicates x=0? Was a reservation made? Logical operators (and, or, not) Queries Can a reservation be made without relaying a credit card number? Must one eventually login if he makes a trip search? Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Querying BPs Temporal Operators Quantifiers over execution paths A φ - All: φ holds on all paths starting from the current state. E φ - Exists: φ holds in at least one path. Path-specific quantifiers X φ - Next: φ holds at the next state. G φ - Globally: φ has to hold on the entire subsequent path. F φ - Finally: φ eventually has to hold (somewhere). φ U ψ - Until: φ has to hold until at some position ψ holds, and ψ must hold eventually. φ W ψ - Weak until: φ has to hold until ψ holds. Querying and Monitoring Distributed BPs

(Monadic) Second Order Querying BPs Temporal Logic Queries Can a reservation be made without relaying a credit card number? E(F(Reserve) and not F(Credit)) Must one eventually login if he makes a trip search? A (F(login) or not F(search)) Logics Linear time Logic (LTL) No path quantifiers CTL* Allows path quantifiers Mu-calculus Introduces fix-point operators First Order (Monadic) Second Order Querying and Monitoring Distributed BPs

Features & Limitations Querying BPs Features & Limitations Scale 2. Expressive Power (Specification) 3. Expressive Power (Query language) 4. (Un)Intuitive Formulation Querying and Monitoring Distributed BPs

1. Scale Evaluation is linear in FSM size, but… Features and Limitations 1. Scale Evaluation is linear in FSM size, but… FSM size is huge for real-life specifications Call stack Data Unfeasible Evaluation Login Search Reserve Payment Cash Credit Confirm Cancel Querying and Monitoring Distributed BPs

Solutions Bounded Model Checking Summarization Compact representation Features and Limitations 1. Scale Solutions Bounded Model Checking [Biere et. Al ’99,’03] ,[Clarke et. Al ’04], … Summarization [Reps et. Al ’98], [Sagiv et. Al ‘05],… Compact representation StateCharts [Harel ’87] BDD [Bryant 86’, Lam et. Al ’05] Querying and Monitoring Distributed BPs

Features and Limitations 1. Scale StateCharts Graphical tool for designing state machine based specifications Extend basic FSM concepts (super-states) Simplifies design UML standard Querying and Monitoring Distributed BPs

Features and Limitations 1. Scale BDD Data structure that allows compact representation of data with high similarities (b1^ b2) V (not(b1) ^ b3) b1 b2 b3 למשל, נכון שכל קונטקס הוא אחר, אבל הפרטמטרים שמועברים הם למשל זהים, קוד שמתבצע זהה וכו 1 Querying and Monitoring Distributed BPs

Store commands as db relations Features and Limitations 1. Scale Relational DB approach [Lam ’05] Store commands as db relations command c is written at location l “Open” all possible contexts Exploit similarities and represent compactly by Binary Decision Diagrams Datalog queries Querying and Monitoring Distributed BPs

FSMs have limited expressive power Features and Limitations 2. Expressive Power (Specification) FSMs have limited expressive power May yield inaccurate approximation of real-life specification Querying and Monitoring Distributed BPs

Models & Query Languages Expressive Power (Specification) Models & Query Languages Monadic Second Order Logic Context Free Graph Grammars Temporal Logic Recursive State Machine First Order Logic Finite State Machine Querying and Monitoring Distributed BPs

Each with multiple entries & exits Expressive Power (Specification) Recursive State Machines (RSMs) A collection of FSMs Each with multiple entries & exits Some states represent calls to other state machines (or to self) An expansion is replacing a call state by a possible implementation An execution is a sequence of expansions Querying and Monitoring Distributed BPs

RSM (example) Call nodes may specify several implementations Expressive Power (Specification) RSM (example) pay search credit Start Payment confirm payment reserve cash HomePage Call nodes may specify several implementations Call nodes may specify entry/exit nodes for connection Querying and Monitoring Distributed BPs

Evaluation (Temporal Logic) Expressive Power (Specification) Evaluation (Temporal Logic) Single Exit RSMs CTL* : Linear data complexity [Benedikt et. Al ‘05] Mu-calculus: also [Alur et. Al ’07] Multiple Exit RSMs LTL: PTIME CTL,CTL*: EXPTIME Mu-calculus: EXPTIME Querying and Monitoring Distributed BPs

Models & Query Languages Expressive Power (Specification) Models & Query Languages Monadic Second Order Logic Context Free Graph Grammars Temporal Logic Recursive State Machine First Order Logic Finite State Machine Querying and Monitoring Distributed BPs

Context Free Graph Grammars Expressive Power (Specification) Context Free Graph Grammars Extensions of string grammars to graphs Labels over graph nodes (VR) or edges (HR) Terminal and non-terminal labels Derivation rules for non-terminal labels No start and end nodes! Connection rules, by labels, for the derived sub-graph Querying and Monitoring Distributed BPs

Expressive Power (Specification) Context Free Graph Grammars Example pay search credit Start Payment confirm payment reserve cash HomePage Homepage payment Connection relation : StartPayment connects only to Search Confirm connects to cash, but not to credit Querying and Monitoring Distributed BPs

Depends greatly on the allowed connection relation Expressive Power (Specification) Context Free Graph Grammars Evaluation Depends greatly on the allowed connection relation A restricted model defines entries and exits for graphs and is equivalent to RSMs Typically Monadic Second Order Queries We’ll revisit it later.. Querying and Monitoring Distributed BPs

Active XML AXML [Abiteboul, M et. Al ’04-’08] Expressive Power (Specification) Active XML AXML [Abiteboul, M et. Al ’04-’08] Extension of XML to include embedded Web-Services calls Very useful for modeling web-sites Formally – a restriction of context free graph grammars to trees XML query languages XPath,XQuery,.. Practically efficient evaluation Querying and Monitoring Distributed BPs

Temporal logics have limited expressive power Features and Limitations 3. Expressive Power (Queries) Temporal logics have limited expressive power Basically – consider only executions Good for behavioral properties Can’t capture structural properties Bisimulation-invariant Example Querying and Monitoring Distributed BPs

Behavioral vs. Structural Analysis Expressive Power (Queries) Behavioral vs. Structural Analysis Querying and Monitoring Distributed BPs

Solution? Expressive Power (Queries) Monadic Second Order Logic Context Free Graph Grammars Temporal Logic Recursive State Machine First Order Logic Finite State Machine Querying and Monitoring Distributed BPs

Studied extensively for Context Free Graph Grammars Expressive Power (Queries) Infeasible! Studied extensively for Context Free Graph Grammars Linear in grammar size But unfortunately… Non-elementary in the query (formula) size 2^2^2^…..2 (tower size depends on query size) Infeasible for even the smallest queries Querying and Monitoring Distributed BPs

4. (Un) intuitive Formulation Features and Limitations 4. (Un)intuitive Formulation Very difficult to express properties of interest in FO or MSO Long and error-prone formulas Temporal Logic is more intuitive Still, textual and complex, especially for large-scaled analysis Existing tools provide inadequate interface 4. (Un) intuitive Formulation Querying and Monitoring Distributed BPs

r[n,z](v) = (c[n](v) & r[n,x1](v)? (Un)intuitive formulation First Order Formulas TVLA (Sagiv et. Al [04]) syntax r[n,z](v) = (c[n](v) & r[n,x1](v)? z(v) | E(v_1) z(v_1) & TC (v_1, v) (v_3, v_4) (n(v_3, v_4) & !x1(v_3)) : r[n,z](v) & ! (E(v_1) r[n,z](v_1) & x1(v_1) & r[n,x1](v) & !x1(v))) Rechability & Pointer Analysis Querying and Monitoring Distributed BPs

Temporal Logic SMV (Clarke et. Al [04]) syntax (Un)intuitive formulation Temporal Logic SMV (Clarke et. Al [04]) syntax PReqFullfilledDef1: assert (P_BUTTON_PRESSED -> (~P_REQ_FULFILLED U P_state)); PReqFullfilledDef2: assert (P_state -> (P_REQ_FULFILLED U P_BUTTON_PRESSED)); EnterTStateDef: assert ((~T_state & X(T_state)) -> X(ENTER_T_STATE)); EnterPStateDef: assert ((~P_state & X(P_state)) -> X(ENTER_P_STATE)); MoveToPPrevDef: assert ((~move_to_p & X(move_to_p)) -> MOVE_TO_P_WAS_SET_TRUE_IN_PREV_SEC); MoveToItoTPrevDef: assert( (~move_to_i_to_t & X(move_to_i_to_t)) -> X(X(MOVE_TO_I_TO_T_WAS_SET_TRUE_IN_PREV_SEC))); Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Querying BPs Mid-section Summary Model problems expressive power scale Pretty much solved by models we’ve seen Query Language problems unintuitive formulation Not solved yet BPQL to the rescue Querying and Monitoring Distributed BPs

Possible Solution Querying BPs BPQL Monadic Second Order Logic Context Free Graph Grammars Temporal Logic Recursive State Machine First Order Logic BPQL Finite State Machine Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Querying BPs BPQL Queries BP patterns (like tree patterns for XML) Single/double-headed edges (Xpath’s / and //) edges paths of arbitrary length Single/double-bounded activities: simple zoom-in unbounded zoom-in Querying and Monitoring Distributed BPs

Q1: used credit card services? Querying BPs Q1: used credit card services? local Querying and Monitoring Distributed BPs

Q2: search without login? Querying BPs Q2: search without login? Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Querying BPs Q3: data flow Querying and Monitoring Distributed BPs

Queries and their semantics Querying BPs Queries and their semantics Query: A BP pattern with some transitive nodes & edges transitive An embedding: a mapping from: query graphs to: [possible flows defined by the] BP graphs A result: image of query graph under an embedding Answer: all results Querying and Monitoring Distributed BPs

Structural vs. Behavioral Semantics Querying BPs Structural vs. Behavioral Semantics Sub-graph homomorphism vs. bisimulation Both are supported in BPQL Querying and Monitoring Distributed BPs

Query with an infinite answer? Querying BPs Query with an infinite answer? Querying and Monitoring Distributed BPs

Finite representation Querying BPs Finite representation Querying and Monitoring Distributed BPs

Query Evaluation Algorithm Querying BPs Query Evaluation Algorithm Systems and queries are essentially Context Free Graph Grammars (Recursive state machines) We basically compute their intersection Bad news:  These are not closed in general under intersection Good news:  Our systems and queries are sufficiently simple: PSIZE representation (as a BP) can be computed in PTIME (data complexity!) Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Querying BPs And what about data? So far we have mainly considered flow Data is important as well Data: variable values, message exchange,… Especially interesting in context of Web Some representative works Web-services analysis Pointer Analysis Querying and Monitoring Distributed BPs

Web-services analysis Querying BPs Web-services analysis [Deutsch et. Al ‘06] Query language combines LTL and FO LTL for temporal relationships FO for data snapshots Polynomial Data Complexity (restricted versions) [Fu et. Al ’04] Guarded automaton for flow query XPath “guards” relate to data Reduces problem into “conventional” model checking Undecidability for general case Polynomial Data Complexity for bounded message number Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Querying BPs Pointer Analysis [Lam et. Al ’05] Pointer analysis for sql injections, buffer overflow,… Analyzes all possible call stack contexts BDD-based optimization techniques capture similarities Efficient run-time, practical system [Sagiv et. Al ’06] Shape analysis Summarization techniques Querying and Monitoring Distributed BPs

Context Free Processes Querying BPs Some Systems SMV, NuSMV [Clarke et. Al, ’92] bddbddb [Lam et. Al, ’04] TVLA [Sagiv et. Al, ’99] SPIN [Bell Labs,’91] SLAM [Ball et. Al. ’00] Moped [Schwoon ’02] Mops [Chen et. Al ’02] BPQL [Beeri, M, D et. Al ’05] Finite State Machine Context Free Processes Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to Business Processes Querying Monitoring Summary & Research Directions Outline Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Monitoring BPs What is Monitoring? The aggregation, analysis, and presentation of real time information about activities inside organizations and involving customers and partners (Gartner) Provide real-time information on executions Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Monitoring BPs Why Monitoring? Imagine you run an auction service… Guarantee fair play: notify on too many cancels Maintain SLA: monitor response time Promotions: prizes for the x10,000 transaction Illegal access: notify on buyers attempt to confirm bids without registering first Monitoring is crucial for enforcing business policies and meeting efficiency & reliability goals Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Monitoring BPs BPEL XML events <actionData> <header> <processName> auctionHouse </processName> <instanceId> 517 </instanceId> <sensorTarget> notify_winner </sensorTarget> <timestamp> 2006-05-31T11:32:46.510+00:00 </> </header> … <activityData> <activityType>invoke </activityType> <evalPoint> completion </evalPoint> … Querying and Monitoring Distributed BPs

Monitoring Systems Layers Monitoring BPs Monitoring Systems Layers Absorb the stream of events coming from the BP execution engine Process and filter events, selects relevant events data and automatically triggers actions A dashboard that allows users to follow the processes progress, view custom reports, perform analysis,… Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Monitoring BPs Existing Approaches XML streams management Complex Event Processing (CEP) Commercial tools) BAM( BP-Mon Querying and Monitoring Distributed BPs

XML streams management Monitoring BPs XML streams management Many works on xml streaming Query Optimization [Koch et. Al ’04, Viglas et. Al ’02,…] Security [Altinel, Franklin ’00, Benedikt et. Al, ’08,…] Updates & Concurrency [Grabs et. Al ’02, Nicola et. Al ’07,…] Automata-based techniques vastly used BPEL processes emits XML messages Use XML streaming engines for monitoring? Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Monitoring BPs Why not ? Each XML element describes an individual event Fairly complex XQuery queries (lots of joins) Difficult to handle by existing streaming engines… XML stream engines manage tree-shaped data (vs. DAGs) XML stream engines expect to receive elements in document order (but we have here parallel flow). Querying and Monitoring Distributed BPs

Complex Event Processing Monitoring BPs Complex Event Processing Processing of multiple events to identify semantically meaningful combinations. Studied extensively for Active Databases [Widom ’96], [Payton ’99], [Wolski ’98],… Recent works in context of BPs [Wu ’06], [Jobst ‘07] Identify meaningful activities combinations that form a business logic Still, somewhat low-level Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Monitoring BPs BP-Mon BAM tools provide solid solutions But do not incorporate static analysis BP-Mon is an integrated framework Uses a query language similar to BPQL Graphical & Intuitive Semantics: evaluated over run-time executions (vs. potential executions in BP-QL) Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Monitoring BPs Running Example Querying and Monitoring Distributed BPs

Unfair play (too many cancellations) Monitoring BPs Query Example (1) Unfair play (too many cancellations) As before New or rep Report/ Report* Querying and Monitoring Distributed BPs

Monitor response time of data-store Monitoring BPs Query Example (2) Monitor response time of data-store Sliding window Time based Instance based Output Xquery like Group by, having Querying and Monitoring Distributed BPs

Illegal bidding (mix of static and run-time analysis) Monitoring BPs Query Example (3) Illegal bidding (mix of static and run-time analysis) Querying and Monitoring Distributed BPs

Some Nice BP-Mon Features Monitoring BPs Some Nice BP-Mon Features Intuitive monitoring (looks like BPEL) Easy deployment (implemented as BPEL) Greedy embedding Automata-based evaluation algorithm Type-based optimization Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Monitoring BPs Evaluation algorithm Incrementally extends a greedy matching to one of a larger prefix Automaton with pattern nodes as states Tries to concurrently match the all concrete patterns of a given pattern Attempts to match events as early as possible On failure: backtracks & retries Complexity: polynomial in the size of the execution log (with the exponent determined by the size of the pattern) Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Monitoring BPs Easy Deployment The automaton is compiled into BPEL May be used on any application server Guarantees portability Querying and Monitoring Distributed BPs

Type-Based Optimization Monitoring BPs Type-Based Optimization Exploit knowledge of specification Infer Irrelevancy & inconsistency using BP-QL Querying and Monitoring Distributed BPs

Querying and Monitoring Distributed BPs Introduction to Business Processes Querying Monitoring Summary & Research Directions Outline Querying and Monitoring Distributed BPs

Summary & Research Directions Almost done  Summary & Research Directions Important aspects of Business Processes Design Analysis Monitoring Plenty of work on each subject, in many different fields Still missing a real sinergy Programming languages, model checking, DB technology XML Streaming, CEP/BAM All together… BPQ: integrated, high-level, intuitive framework for all. Querying and Monitoring Distributed BPs

Topics for research: Summary & Research Directions Almost done  Summary & Research Directions Topics for research: Missing information (partial specifications, logs,…) Probabilistic Processes Data values Interactions Log mining Enhanced query language features Optimizations Further applications and more… Querying and Monitoring Distributed BPs