MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Enabling BPM for Clouds Hans-Arno Jacobsen Bell University Laboratory Chair University of Toronto Summer School on Service Research, July 19 th, 2010 “BPM in Cloud Architectures: Business Process Management with SLAs and Events“BPM in Cloud Architectures: Business Process Management with SLAs and Events”, joint work with Vinod Muthusamy (extended abstract) An eQoSystem for declarative distributed applications with SLAs via Events & SLAs

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen c001 end Service Summer, July 19th, 2010, KIT, Germany > 0.7 else Business Process Example Check score Credit check Credit check 2 Check score 2 Reject Approve Send to officer < 0.3 Store in DB … … Loan Application Processing < 0.5 service RTT < 100ms uptime > % P id =c401 gid=c001 service time < 2s service cost < $0.02 P id =c1 P id =c001 P id =c3301

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Service Summer, July 19th, 2010, KIT, Germany Large-scale Business Processes Vendor Sale Manufactory Finance Dispatch B Out-stock B Pick-up goods Packaging Marketing Design Out-stock B Target price Prototype OutTake Control Assign Confirm Determinate plan Check stock Raw materials Audit Raw Determinate plan Execute plan Process control Monitor Process Pay Check Signature Print receipt Warehouse Delivery FedEx Pick up Monitoring Statistic Chart Strategy Design Marketing Order Manufactory Payment Requirement collection Feature selection Goods selection Confirm features Material Make plan Feedback Check order Fill order Check dealerCheck credit Approval Validate Affirm order Sale prediction Sign Contract CCC administrate Goods delivery Fill dispatch bill Fill out-stock bill Credit card Case Study (Chinese Electronics Manufacturer): Department-level processes with 26 to 47 activities Global processes that compose departmental ones Thousands of concurrent instances Hundreds of collaborating partners Geographically distributed Administrative boundaries

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Service Summer, July 19th, 2010, KIT, Germany What Support is Required ? De-coupling and loose coupling Fine-grained event filtering In-network event processing Composite event detection Event correlation

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Service Summer, July 19th, 2010, KIT, Germany Agenda Enabler The PADRES approach Event-driven BPM with PADRES SLA-driven BPM with PADRES

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen What Abstractions Enable BPM? Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen What Abstractions Do Not Work? Databases  Great for managing historic data  But what about future data (e.g., events) Data streams  Great for managing structured streams of tuples  But what about un-structured, multi-typed, sporadic, un- ordered events from many sources Rule-based expert systems  Great for inference and reasoning  But what about managing large numbers of fined-grained filters in distributed environments Cum grani salis Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Service Summer, July 19th, 2010, KIT, Germany What Abstractions Enable BPM? It is our opinion that the afore-mentioned requirements can best be addressed by  The content-based publish/subscribe paradigm  Realized by content-based message routing Events represent state transitions in the environment.  Conveyed as publications to the pub/sub system Event filtering and correlation is based on  Subscriptions managed by the pub/sub system

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen amount > $500K Input queue Output queue d2 Output queue d3 subscriptiondest Matching Engine Routing Table & Class = Loan d2 Service RTT < 2s d3 B B B B B Content-based Publish/Subscribe Publisher Subscriber Publications Subscriptions Notification Service RTT < 2s Service RTT < 150ms Uptime > 99.99% class = Loan, Service = credit check status = approved, 1. Advertise 2. Subscribe 9 Event-Based Decoupled Flexible Responsive Content Routing Declarative Service Summer, July 19th, 2010, KIT, Germany Content-based Routing 3. Publish

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Application Modeling Advertisements (schema or types)  [class=Loan], [action=*], [customerID=*], [amount<$100K], [region=East] Publications & events (data)  [class, Loan], [action, request], [customerID, ], [amount, $50K] Subscriptions (query)  [class=Loan], [amount>$500K], [region=*] Application semantics is expressed via advertisements (data sources), publications (data sources), and subscriptions (data sinks) Service Summer, July 19th, 2010, KIT, Germany A S P

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Service Summer, July 19th, 2010, KIT, Germany Benefits of Publish/Subscribe Simplifies IT development and maintenance by decoupling enterprise components Supports sophisticated interactions among components using expressive subscription languages Supports fine-grained subscriptions for event management Achieves scalability with in-network filtering and processing

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Service Summer, July 19th, 2010, KIT, Germany Many Applications are Event-based & Benefit from Publish/Subscribe A BC D E F RFID and sensor networksService oriented architectures Workflows, business processes and job scheduling Supply chain and logistics Event-Based Light Callback RFID: Razor SKU Transform Fault! Temperature Invoke Loan Ordered Delivered In flight Job A done Triggered

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Web start & open source padres.msrg.org Implemented in Java Service Summer, July 19th, 2010, KIT, Germany Our PADRES ESB for Event-driven BPM First generation of students, when I looked away   Peng Alex David aRno Eli Serge PADRES is Publish/subscribe Applied to Distributed Resource Scheduling PAdres is Distributed REsource Scheduling Enterprise Services Bus (Events & Services Bus) 13 Acknowledgements (2004-present):

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Service Summer, July 19th, 2010, KIT, Germany Our PADRES ESB Stack & Vision

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen B B B B P S S input queue output queue dest2 output queue dest3 dest1 dest2 dest3 P S = publisher = subscriber subscriptiondest Matching Engine Routing Table + service time < 3s dest2 service time < 2s dest3 Publications service time = 2.5s service time = 1s service time = 3s PADRES ESB is an Overlay of P/S Brokers output queue dest1 Service Summer, July 19th, 2010, KIT, Germany Try out and download at:

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Service Summer, July 19th, 2010, KIT, Germany Innovative PADRES Features A BC D E F Composite Events Historic Access Management Robustness Load Balancing Security ACM Middleware’2004 IEEE ICDCS’2005 IEEE ICDCS’2009 ACM Middleware’2008 IEEE ICDCS’2010 ACM Middleware’2006 ACM Middleware’2007 CSRG TR 2009 ACM DEBS’2007

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Event-driven BPM with PADRES Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen BC D A BC D E F Example: D executes, if B and C have completed (D depends on B and C). Service Summer, July 19th, 2010, KIT, Germany Modeling Business Processes Dependency in processes and more complex process patterns require event correlation Event correlation is enabled by the detection of composite events Composite events are expressed via composite subscriptions Composite subscription consists of atomic subscriptions Subscription language features for BPM modeling  E.g., AND, OR, and variables ($x) A BC D E F Exception

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Composite Subscription Examples Expresses a structural property of a process [class = Activity Status], [cmd. = Archived],[Process ID = $X ] AND [class = Activity Status], [cmd. = Signed Off],[Process ID = $X ] Expresses a performance property of a process [cmd. = Credit check request], [Process ID = $X ] AND [status = Approved], [Process ID = $X ] Service Summer, July 19th, 2010, KIT, Germany Check score Credit check Process Approve Singed Off … Loan Process Reject Archive > $50K

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Service Summer, July 19th, 2010, KIT, Germany Business Process Management Transformation of process into pub/sub language Deployment of transformed process Execution of process by triggering instances Monitor process & instance execution Manage, i.e., control, version,… A BC D trigger E F trigger multiple instances concurrently Exception & compensation

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Business Process Execution Receive PADRES ESB BPEL Receive Assign Flow InvokeWait Reply Wait Reply Invoke 2 Assign WS Gateway Agent 6 END WS client Web Service http/soappub/sub Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Sub4: [class,eq,ACTIVITY_STATUS], [process,eq,’Process5’], [activityName,eq,’activity2’], [IID,eq,$X], [status,eq,’SUCCESS’] && [class,eq,LINK_STATUS], [process,eq,’Process5’], [activityName,eq,’activity2’], [IID,eq,$X], [status,isPresent,any] BPEL Transformation Example Process 5 activity1 flow1 activity2activity5 activity3activity6 activity4activity7 activity8 Sub1: [class,eq,ACTIVITY_STATUS], [process,eq,’Process5’], [activityName,eq,’activity1’], [IID,isPresent,any], [status,eq,’SUCCESS’] Pub1: [class, ACTIVITY_STATUS], [process,’Process5’], [activityName,’flow1’], [IID,’g001’], [status,’STARTED’] Sub2: [class,eq,ACTIVITY_STATUS], [process,eq,’Process5’], [activityName,eq,’flow1’], [IID,isPresent,any], [status,eq,’STARTED’] Pub2: [class, ACTIVITY_STATUS], [process,’Process5’], [activityName,’actitiy2’], [IID,’g001’], [status,’SUCCESS’] Pub3: [class,LINK_STATUS], [process,’Process5’], [activityName,’actitiy2’], [IID,’g001’], [status,’POSITIVE’] Pub4: [class, ACTIVITY_STATUS], [process,’Process5’], [activityName,’actitiy6’], [IID,’g001’], [status,’SUCCESS’] Sub5: [class,eq,ACTIVITY_STATUS], [process,eq,’Process5’], [activityName,eq,’activity4’], [IID,isPresent,any], [status,eq,’SUCCESS’] && [class,eq,ACTIVITY_STATUS], [process,eq,’Process5’], [activityName,eq,’activity7’], [IID,isPresent,any], [status,eq,”SUCCESS”] Pub5: [class, ACTIVITY_STATUS], [process,’Process5’], [activityName,’actitiy7’], [IID,’g001’], [status,’SUCCESS’] Service Summer, July 19th, 2010, KIT, Germany Cf. our ACM Trans Web’2010 for full BPEL mapping Sub1 (flow agent): [class = ACTIVITY_STATUS], [process = ’Process5’], [activityName = ’activity1’], [IID isPresent any], [status = ’SUCCESS’] Sub1 (flow agent): [class = ACTIVITY_STATUS], [process = ’Process5’], [activityName = ’activity1’], [IID isPresent any], [status = ’SUCCESS’] Sub4 (actvity6 agent): ( [class = ACTIVITY_STATUS], [process = ’Process5’], [activityName = ’activity5’], [IID = $X], [status = ’SUCCESS’] ) AND ( [class = LINK_STATUS], [process = ’Process5’], [activityName = ’activity2’], [IID = $X], [status isPresent any] )

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Evaluation: Changing Request Rate P/S PADRES ESB Clustered Centralized Distributed Service Summer, July 19th, 2010, KIT, Germany 20 servers

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen SLA-driven BPM Service Summer, July 19th, 2010, KIT, Germany An eQoSystem for declarative distributed applications with SLAs

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Currently, business goals must be manually considered at every stage of the business process development cycle N Y Far? Get destination Validate request Find flight Find train cost < $0.02 service time < 3s Only trusted partners Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Service Summer, July 19th, 2010, KIT, Germany LayerSLA MetricExample Business Process CostCost of search service < $0.02 per use Fidelity, quality, utilityMap resolution > 300x300 Trust, reputationOnly use trusted payment services Deployment & Operations Service timeExecution time < 3s ThroughputThroughput > 100/min AvailabilityUptime > 99.9% Load balanceServer utilization delta < 10% NetworkLatencyService RTT < 100ms BandwidthMax bandwidth < 3Mbps JitterDelay jitter < 10ms Service Level Agreements (SLAs) SLAs are contracts between service consumers and providers that specify the expected behavior of each party and the penalties of violating the contract. SLAs specify business goals declaratively.

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Runtime Uses of SLAs ABCD p q Web serviceExecution engine 2 1b service time < 1s M ESB broker topology A,B D 1a C service time < 2s Dynamic service discovery Discover services with capabilities that satisfy goals. Monitoring Only monitor the business events related to goals. Feed back measurements to support runtime adaptations. Distributed execution Fine-grained allocation of process to available resources. Move portions of process to strategic locations. ESB adaptation Reconfigure the ESB topology to satisfy goals. M Monitor ESB node (PADRES broker) Process Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Agenda Distributed process execution  Architecture  Components Execution & optimization algorithms  Design issues  Evaluation Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Process Execution Architectures Centralized One execution engine May not scale Central point of failure Agent-based Distributed execution engine In-network processing Lower bandwidth and latency Fine-grained use of resources Clustered Replicated execution engines Centralized control and data High bandwidth and latency Still may not scale & administrative limitations ABCDABCD ABCDABCD ABCDABCD D C A,B Problem: How to deploy activities in a distributed manner to satisfy SLAs? Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Large-scale Business Processes Vendor Sale Manufacture Finance Dispatch B Out-stock B Pick-up goods Packaging Marketing Design Out-stock B Target price Prototype OutTake Control Assign Confirm Determinate plan Check stock Raw materials Audit Raw Determinate plan Execute plan Process control Monitor Process Pay Check Signature Print receipt Warehouse Delivery FedEx Pick up Monitoring Statistic Chart Strategy Design Marketing Order Manufacture Payment Requirement collection Feature selection Goods selection Confirm features Material Make plan Feedback Check order Fill order Check dealerCheck credit Approval Validate Affirm order Sale prediction Sign Contract CCC administrate Goods delivery Fill dispatch bill Fill out-stock bill Credit card Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen 31 Distributed Process Execution: Architecture & Components Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Execution Engine Atomic Redeployer (IEEE ICDCS’2009) Candidate Engine Discovery (ACM DEBS’2009)ACM DEBS’2009 PADRES messaging layer Activity Manager Activities (ACM TWEB’2010) Activity Profiler Engine Profiler Latency Bandwidth Engine resource Redeployment Manager (CASCON)CASCON SLAs, cost models Estimators Ranking algorithms SLA Management Stack Instance states Input, output queues Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Activity Profiler Profiles execution of local activities Maintains profiles for various metric types  Message hops, disk I/O, energy usage, etc. Profile for message hops metric ActivityPred/ Succ Msg rate TP40/min TS10/min 147 Execution engineBroker P T S PTS Process ESB broker topology Profile for execution time metric ActivityExec time T0.3 s Profile for memory usage metric ActivityMem T30 MB Activity Service Summer, July 19th, 2010, KIT, Germany Example:

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Activity Profiler Summary Service Summer, July 19th, 2010, KIT, Germany AP k activity profiler for metric type M k AP k activity profiler for metric type M k activity lifecycle events aiai s k (a i )

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Engine Profiler (e.g., distance) Discover paths  e 2 →e 5, e 5 →e 7 Probe paths  e 5 →e 4 (for candidate C) Compute paths  e 2 →e 4, e 4 →e 7  Cases e 4 in e 2 →e 5 e 4 in e 5 →e 7 otherwise 147 Execution engineBroker P S C PTS Probed paths 5411:34pm Discovered paths 2531:48pm 5721:49pm Computed paths 2422:02pm 4712:04pm Computes and caches information about candidate engines Cf. DEBS’2009 for our resource discovery algorithms to identify candidatesDEBS’2009 T Process Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Engine Profiler Summary Service Summer, July 19th, 2010, KIT, Germany EP k engine profiler for metric type M k EP k engine profiler for metric type M k set of candidate engines ejej p k (e j )

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Redeployment Manager Estimator: Computes an estimate of the metric cost c k (a i,e j ) of hosting an activity a i at engine e j Cost model: Computes an estimate of the cost c(a i,e j ) of hosting activity a i on engine e j Check deployment: Determines what to do with an activity a i  Determine best engine e  Compute benefit: c(a i ) – c(a i,e)  Compute resident time at current engine  If resident long enough If benefit is large enough move a i to e Otherwise, apply pressure to other activities Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Redeployment Manager Summary Compute the cost of deploying local activities a i at candidate engines e j Given F(a i ): Cost function P(a i ): Predecessors S(a i ): Successors Measurements C(a i ): Cost at local engine E(P(a i )): Location of predecessors E(S(a i )): Location of successors Compute C(a i, e j ) for every a i, e j : Estimated cost of deploying activity a i at candidate engine e j Complexity Memory: O( |a i | |e j | ) Computation: O( |a i | | e j | |N avg (a i )| ) Cost Model Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Redeployment Manager Summary Service Summer, July 19th, 2010, KIT, Germany E k estimator for metric type M k E k estimator for metric type M k aiejaiej c k (a i,e j ) AP k EP k CM cost model CM cost model aiejfaiejf c(a i,e j ) EkEk EkEk EkEk EkEk Check Deployment aiai ejej redeploy {e j } pressure RM Redeployment Manager RM Redeployment Manager {a i } { (a i,e j ) } redeploy { (a i,{e j })} pressure

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Atomic Redeployment Traditional pub/sub client movement protocols are expensive and do not offer transactional properties Transactional movement  Formalized movement properties similar to ACID properties  Efficient and guaranteed routing reconfiguration For example, guarantee that no messages are lost, if an activity is re-deployed See IEEE ICDCS’2009IEEE ICDCS’2009 Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Execution Engine Atomic Redeployer (IEEE ICDCS’2009) Candidate Engine Discovery (ACM DEBS’2009)ACM DEBS’2009 PADRES messaging layer Activity Manager Activities (ACM TWEB’2010) Activity Profiler Engine Profiler Latency Bandwidth Engine resource Redeployment Manager (CASCON)CASCON SLAs, cost models Estimators Ranking algorithms SLA Management Stack Summary Instance states Input, output queues Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen 1. Distribution cost C dist = f(C d3, C d1, C d2 ) C d1 Message rate C d2 Message size C d3 Latency 2. Engine cost C eng = f(C e1, C e2,C e3 ) C e1 Load (number of instances) C e2 Resources (CPU, memory, etc.) C e3 Activity complexity 3. Service cost C serv = f(C s1, C s2, C s3 ) C s1 Latency of external service C s2 Execution time of external service C s3 Marshalling/unmarshalling Cost(activity) = f(w i C i ) Cost(process) = ∑cost(activity) The cost of a process is based on metrics These metrics can be weighted to achieve different objectives Optimize time w d1 = w d3 = w e3 = w serv = 1, other w i = 0 Optimize network overhead w d2 = w d3 = 1, other w i = 0 Various optimization criteria can be specified Threshold criteria: ∑w i C i > x E.g., Report SLA violations within 3 s. Minimized criteria: min( ∑wiCi ) E.g., Minimize distribution overhead Cost Model Components Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Examples of SLAs & Cost Functions Minimize message hops  f()= msg_hops_rate = msg_rate * engine_distance Minimize bandwidth cost  f()= msg_rate * link_cost $$$ Limit CPU & network energy usage  f()= 0.3* cpu_energy * link_energy < X f() = 0.3* (invocation rate * engine_unit_energy) * (msg_rate * link_unit_energy) < X Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Evaluation and design issues Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Red activities are pinned to brokers Process Hotspot – Illustration Execution engineBroker Process ESB broker topology SLA Minimize traffic Metric Message hops DEF p = 90%q = 10 % B C AI G H Service Summer, July 19th, 2010, KIT, Germany D F E AB C GI H

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Process Hotspot – Results Traffic with redeployment is 47% of the static case  Post-redeployment traffic is 10%  Redeployment triggered in about 30 sec 47 10% of static Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Varying Hotspot – Illustration DEF pq Execution engineBroker Process ESB broker topology B C AI G H Service Summer, July 19th, 2010, KIT, Germany D F E AB C GI H Red activities are pinned to brokers

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Varying Hotspot – Results Dynamic redeployment suffers temporarily after process hotspot moves Traffic with redeployment is 42% of the static case Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Summary on SLA-driven BPM Distributed execution engine has qualitative and quantitative advantages Redeployment algorithm can optimize SLA in many cases Challenge 1: Local optima  Techniques to selectively expand knowledge can work Widen candidate radius Redeploy sets of activities Challenge 2: Coordination  Independent decisions can destabilize system  Potential problem has not manifest in evaluations so far Challenge 3: SLA granularity (more an engineering issue)  Can’t specify SLA on portions of a process  Can’t specify SLA on particular instances of a process (e.g., VIP user) Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Service Summer, July 19th, 2010, KIT, Germany Benefits of Content-based Publish/Subscribe for BPM Naturally enables centralized and distributed business process coordination Coordination can span administrative domains and physically distributed resources Supports process orchestration and choreography Monitoring & control is integral part of paradigm Agile on the fly process adaptation and versioning Correlation of application events with low-level infrastructure events

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Service Summer, July 19th, 2010, KIT, Germany Summary: The PADRES Stack Download

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Outlook Dynamic BPM for flexible service value chains Ad hoc business processes Business impact assessment Business entity concept Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Service Summer, July 19th, 2010, KIT, Germany Conclusions Many applications are inherently event-driven. Effective BPM requires capable event processing abstractions. Content-based publish/subscribe is a powerful event processing abstraction and paradigm. PADRES is based on the pub/sub paradigm. PADRES is an ESB targeted at event-based BPM. PADRES enables real-time business analytics and business activity monitoring.

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Acknowledgements: Current PADRES Team Chen Chen Alex Cheung Alton Chiu Amer Farroukh Patrick Lee Guoli Li Bala Maniymaran Vinod Muthusamy Reza Sherafat Naweed Tajuddin Chunyang Ye Young Yoon Service Summer, July 19th, 2010, KIT, Germany Countless alumni (see our web site)

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Service Summer, July 19th, 2010, KIT, Germany Questions & Discussion? A D R E S P

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Selected Literature about PADRES All our papers are available from  & hyperlinked belowhttp://msrg.org/tags/padres A Distributed Service Oriented Architecture for Business Process Execution. ACM Trans. on the Web, A Distributed Service Oriented Architecture for Business Process Execution. Publisher Placement Algorithms in Content-based Publish/Subscribe. IEEE ICDCS, Publisher Placement Algorithms in Content-based Publish/Subscribe Transactional Mobility in Distributed Content-Based Publish/Subscribe Systems. IEEE ICDCS, Transactional Mobility in Distributed Content-Based Publish/Subscribe Systems. Distributed Automatic Service Composition in Large-Scale Systems. ACM DEBS, Distributed Automatic Service Composition in Large-Scale Systems Adaptive Content-based Routing in General Overlay Topologies. ACM Middleware, Adaptive Content-based Routing in General Overlay Topologies. Historic Data Access in Publish/Subscribe. ACM DEBS, Historic Data Access in Publish/Subscribe Papers on managing SLAs with PADRES  Service Summer, July 19th, 2010, KIT, Germany

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Service Summer, July 19th, 2010, KIT, Germany References The PADRES ESB project home  An eQoSystem for declarative distributed applications with SLAs  The Micro-ToPSS event processing middleware for sensor networks (RFID)  Mobile-ToPSS – publish/subscribe for mobile and location-based applications  ToPSS - the Toronto Publish/Subscribe System Family  (coming soon ) Quantifying events in software to increase modularity & customization in C- based systems and software-based product lines  (ACC - the AspeCt-oriented C compiler) The Middleware Systems Research Group  My web site  the University of Toronto DEBS Conference July IBM in New York, US

MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG © 2010 Hans-Arno Jacobsen Service Summer, July 19th, 2010, KIT, Germany