1. CS 493/693: Distributed Systems Programming V. “Juggy” Jagannathan CSEE, West Virginia University April 11, 2005

2. Project Review Details Total time – 5 minutes  Introduction & Use case diagrams – 1 minute  Sequence Diagrams – 2 minutes  Screen shots – 2 minutes Email class presentation before class (Monday, April 25 th ) or bring it on a USB thumb drive (no floppies please – I don’t have a floppy drive) Email complete project source code, updated design documents and screen shots before end of the day April 25 th (Monday)

3. Review of Chapters 3, 4, 7 & 8 Web Services: Concepts, Architectures and Applications G. Alonso et. al. Springer Verlag

4. Chapter 3: Enterprise Application Integration Web Services: Concepts, Architectures and Applications G. Alonso et. al. Springer Verlag

5. From Middleware to Enterprise Application Integration (EAI) Middleware and 3-tier architectures addressed the need to integrate multiple resources (typically databases whose interface was standardized using ODBC or JDBC type interfaces) EAI refers to the integration faced when dealing with multiple middleware systems to provide enterprise-wide functionality

6. Copyright Springer Verlag Berlin Heidelberg 2004 EAI: Application Example Manual process of implementing a supply-chain system

7. dispatcher inventory management ERPshipping message-oriented middleware month-end closing new PO Copyright Springer Verlag Berlin Heidelberg 2004 Need for message brokers With RPC or message-based interoperability, applications need to be Changed if they need to interoperate with a new system (dashed)

8. message broker core senderreceiver message broker with message brokers, custom message routing logic can be defined at the message broker level or at the queue level in basic MOM it is the sender who specifies the identity of the receivers Copyright Springer Verlag Berlin Heidelberg 2004 Extending basic Message-Oriented Middleware (MOM) Message brokers enable users to define custom message routing logic

9. dispatcher (publisher) inventory management (subscriber) ERP (subscriber) shipping (subscriber) message broker month-end closing (subscriber) new PO Copyright Springer Verlag Berlin Heidelberg 2004 Publish/Subscribe Interaction Model

10. Workflow Management Systems

11. Historical background on workflow systems WfMS – work flow management systems Evolved from supporting “office automation” Types of workflows  Administrative workflows (travel reimbursement processing)  Production workflows Commercial platforms  WebSphere MQ workflow (IBM)  Vitria Businessware  Tibco Business Process Modeling (BPM)  BEA Web Logic  Microsoft BizTalk Orchestration Standardization effort – Workflow Management Coalition (WfMC)

12. Workflow definitions Business Process – a collection of activities that are performed by humans or software applications Work node – work that needs to be done Routing node – define the order in which work can be carried out Start and completion nodes – start and end points of workflows Workflow instance – an instance of a specific workflow process

13. workflow engine resource broker completed work items inbound queue resource 1 resource 2 resource n 1 2 4 5 3 workflow definitions outbound queues workflow designer Copyright Springer Verlag Berlin Heidelberg 2004 Scheduling and managing workflows

14. Recovering from failures Forward recovery Backward recovery  Sagas or compensating activity Exception handling languages Deadlines and deadline expiration

15. Workflow summary Expensive and complicated to implement Useful in applications that are highly repetitive But the same applications can be done better with conventional middleware Nevertheless, they constitute a generic approach to a lot of commercial workflow problems.

16. Chapter 4: Web Technologies Web Services: Concepts, Architectures and Applications G. Alonso et. al. Springer Verlag

17. HTTP client wide area network (Internet) HTTP proxy HTTP server HTTP gateway firewall HTTP tunnel Copyright Springer Verlag Berlin Heidelberg 2004 HTTP requests processed through intermediaries

18. middleware Web server client browser java virtual machine applet wide area network (Internet) firewall server (resource manager) Copyright Springer Verlag Berlin Heidelberg 2004 Applets as a way to implement remote clients

19. middleware Web server browser wide area network (Internet) firewall HTTP GET request client CGI program server (resource manager) Copyright Springer Verlag Berlin Heidelberg 2004 Common Gateway Interface (CGI)

20. middleware browser wide area network (Internet) firewall HTTP GET request client Web server Java server process Java thread server (resource manager) Copyright Springer Verlag Berlin Heidelberg 2004 Servlets replaces CGI

21. connection to resource mgmt layer presentation layer resource management layer application logic layer client application server Web server wide area network (Internet) firewall HTTP browser other protocols other servers (email, SOAP,..) Copyright Springer Verlag Berlin Heidelberg 2004 Application servers

22. support for communication and presentation Servlets JavaServer Pages (JSP) Enterprise Java Beans (EJB) Java DataBase Connectivity (JDBC) Java Naming and Directory Interface (JNDI) support for the application integration Java 2 Connector Architecture (J2CA) Java Message Service (JMS) Java transaction API (JTA) Java API for XML Processing (JAXP) JavaMail Java Authentication and Authorization Service (JAAS) support for access to resource managers Copyright Springer Verlag Berlin Heidelberg 2004 J2EE Platform

23. EJB Beans  Session Beans – can be stateful (e.g. shopping cart) or stateless  Entity beans – persistent objects – typically saved in databases  Message-driven beans Container

24. Chapter 7: Service Coordination Protocols Web Services: Concepts, Architectures and Applications G. Alonso et. al. Springer Verlag

25. 1: requestQuote 2: orderGoods 3: makePayment customer (client) supplier (Web service) The internal business logic of clients and Web services must support the conversation, and maintain the state across different operation invocations belonging to the same conversation. The interaction between clients and services is often formed by a set of operation invocations (i.e., it is a conversation). A service provider may support some conversations while disallowing others. Copyright Springer Verlag Berlin Heidelberg 2004 Sample Conversation

26. quote requested goods ordered requestQuote cancelOrder order canceled order completed orderGoods makePayments Copyright Springer Verlag Berlin Heidelberg 2004 Depicting conversation using state machine model

27. supplier customer 1:requestQuote 2:orderGoods 4:makePayment 3:confirmOrder Copyright Springer Verlag Berlin Heidelberg 2004 Conversation with two web services

28. supplier customer 1:requestQuote 2:orderGoods 5:makePayment warehouse 3:checkShipAvailable 7:getShipmentDetail 8:confirmShipment 9:confirmShipment 6:orderShipment 4:confirmOrder Copyright Springer Verlag Berlin Heidelberg 2004 Procurement Protocol Modeled using State Diagram

29. requestQuote orderGoods confirmOrder getShipmentDetail confirmShipment supplier customer warehouse checkShipAvailable makePayment orderShipment confirmShipment Copyright Springer Verlag Berlin Heidelberg 2004 Procurement Protocol Modeled using Sequence Diagram

30. requestQuote (to supplier) checkShipAvailable (to warehouse) confirmOrder (to customer) orderGoods (to supplier) cancelOrder (to customer) makePayment (to supplier) orderShipment (to warehouse) getShipmentDetails (to customer) confirmShipment (to warehouse) confirmShipment (to supplier) supplierwarehousecustomer warehouse confirms warehous e cancels Copyright Springer Verlag Berlin Heidelberg 2004 Procurement Protocol Modeled using Activity Diagram

31. manufacturinghealth care…telecomfinance support for protocols such as transactionality, reliability, security,… Web services executing vertical protocols. The main focus of standards such as RosettaNet, xCBL, and part of ebXML is to describe protocols at this level. other Web services middleware (e.g., SOAP routers) service provider middleware for horizontal protocols provides properties and guarantees to the execution vertical protocols. Standards such as WS-Coordination, WS-Transaction, and part of ebXML fit here. SOAP messages Copyright Springer Verlag Berlin Heidelberg 2004 Horizontal & Vertical Protocols for Web Services

32. Infrastructure for Coordination Protocols Conversation controllers  Conversation routing  Protocol compliance verification Horizontal/vertical protocols

33. P1P1 P 2, P 3 P 4, P 5 conversation controller object for P 1 object for P 2 object for P 3 object for P 4 object for P 5 P1P1 P2P2 P3P3 P4P4 P5P5 service requestor service provider clients invoke operations at the same address the controller dispatches messages to the appropriate implementation object Copyright Springer Verlag Berlin Heidelberg 2004 Conversation controller routes messages

34. CreateCoordinationContext -... - coordination type - current context CreateCoordinationContextResponse -... - coordination context - identifier - coordination type - registration service -... ActivationCoordinatorPortType coordinator ActivationRequestorPortType Web service Copyright Springer Verlag Berlin Heidelberg 2004

35. protocol-specific messages from participant to coordinator protocol-specific messages from coordinator to participant XCoordinatorPortType coordinator XParticipantPortType Web service Copyright Springer Verlag Berlin Heidelberg 2004

36. atomic transaction coordinator CompletionCoordinatorPortType CompletionWithAckCoordinatorPortType PhaseZeroCoordinatorPortType 2PCCoordinatorPortType OutcomeNptificationCoordinatorPortType CompletionParticipantPortType CompletionWithAckParticipantrPortType PhaseZeroParticipantrPortType 2PCParticipantPortType OutcomeNptificationParticipantPortType ActivationCoordinatorPortType RegistrationCoordinatorPortType RegistrationParticipantPortType WS-Coordination interfaces needed for chaining WS-Transaction interfaces WS-Transaction interfaces needed for chaining Copyright Springer Verlag Berlin Heidelberg 2004

37. Web service AWeb service BWeb service CCoordinator R create CC A1 operational message register for BusinessAgreement BusinessAgreement coordinator operational message register for BusinessAgreement BusinessAgreement coordinator completed faulted compensate forget Copyright Springer Verlag Berlin Heidelberg 2004

38. WS-coordination Goals  A framework for supporting coordination protocols  A meta-specification  Supports mechanisms for communicating: unique identifiers – using a structure called “coordination context” the ports used in web services for participating in various conversations the role that a participant plays in a given conversation using “action” interface

39. Components of WS-coordination

40. Web service coordinator Web service coordinator (a) central coordination (b) distributed coordination Copyright Springer Verlag Berlin Heidelberg 2004 Centralized and distributed Coordination

41. Components of WS-Coordination Abstractions used  Coordination protocol – a set of rules governing conversations  Coordination type – grouping of coordination protocols  Coordination context – data structure that tracks instance identifier and pertinent information related to a conversation Interaction forms  Activation – participant initiates a new context and creates a new conversation  Registration – a web service registers as a participant in a conversation  Protocol-specific interactions – messages exchanged in the context of executing a particular conversation

42. Central Coordination

43. Types of messages exchanged Operational messages – e.g. OrderGoods, RequestQuote WS-coordination messages – e.g. activation, registration Protocol-specific messages – to mark the start and end of various actions/activities associated with any conversation WS-Coordination only provides standardization to WS- coordination type messages.

44. Web service A activation participant registration participant protocol participant coordinator C activation coordinator registration coordinator protocol coordinator Web service B activation participant registration participant protocol participant 1. create CC 2. X1 3. register 4. protocol coordinator 5. operational message 6. register 7. protocol coordinator 8. protocol-specific message 9. protocol-specific message Web service implementation Copyright Springer Verlag Berlin Heidelberg 2004 Sample types of messages exchanged

45. WS-Coordination summary WS-coordination defines:  SOAP extensions for supporting coordination  Meta-protocols for creating coordination contexts, binding participants to coordinators  Mechanisms to support both central and distributed coordination Mechanisms defined are similar to the ones defined by CORBA Object Transaction Service (OTS).

46. WS-Transaction

47. Builds on top of WS-Coordination Initially proposed by IBM, Microsoft & BEA in August 2002 Transactions in web services  Will have to deal with lack of a homogeneous middleware  Transactions likely to be long-running  Not always possible to impose ACID properties  Concepts which are precise in database operations may have no relevance in web services Approach used in web services to deal with transactions  Notion of “compensation” as a way of rolling back “business activities”  Short duration transactions are handled similar to middleware and are called “atomic transactions”

48. RosettaNet

49. Goals and scope of RosettaNet  Global, non-profit, business consortium of 400 vendors  eCommerce interfaces to align IT supply chain  Standardization efforts in the following areas Business processes RosettaNet Partner Interface Processes Defines business documents, vocabulary and choreography of messages PIPs are published by RosettaNet in a PIP Directory Data format RosettaNet Business Dictionary Rosettanet Technical Dictionary Messaging services Rosettanet Implementation Framework (RNIF)

50. Partner Interface Process (PIP) Specifications PIP specification includes  A technical dictionary  Message guideline Two categories of PIP messages  Business action messages – e.g. Purchase Order; Billing statement  Business signal messages – e.g. acknowledgement messages related to business action messages Process of developing a PIP specification  Model “as-is” business process and partner types identification  Re-engineer this model to support web-based interactions  Create a PIP Blueprint document of the re-engineered process showing partner types and interactions  Create a PIP protocol that can be implemented by system developers

51. > request price and availability :customer :product supplier start process price and availability request > price and availability request > price and availability response [ fail ] failed [ success ] end Copyright Springer Verlag Berlin Heidelberg 2004 RosettaNet PIP 3A2 (Request Price and Availability) Business Process Flow Diagram

52. Other standards related to Coordination Protocols XML Common Business Library (xCBL)  CommerceOne developed this specification  For example, develops an EDI for order management, package, transport and invoicing using XML  Includes notions of roles – buyer, seller Electronic Business Using eXtensible Markup Language (ebXML)  Sponsored by UN/CEFACT – UN that is responsible for EDI  Specifications for how electronic commerce should be specified, documented, and conducted. Web service choreography interface (WSCI)

53. Chapter 8: Service Composition Web Services: Concepts, Architectures and Applications G. Alonso et. al. Springer Verlag

54. Basics of Service Composition Composition as a way to master complexity

55. requestQuote orderGoods makePayment customer (client) supplier (Web service) The internal business logic of clients and Web services is quite sophisticated, as it must support the execution of different conversations so that each party can properly interact with every other party. A client engages in different conversations with several Web services. In general, these conversations may be regulated by different protocols, and each invoked Web service may not be aware that the client is invoking other Web services. approval (Web service) another supplier (Web service) requestQuote notifyPayment Copyright Springer Verlag Berlin Heidelberg 2004 A client engaged in multiple conversations with different web services

56. supplier procurement approval another supplier supply chain accounting inventory planning …… Customer Copyright Springer Verlag Berlin Heidelberg 2004 Composite services are themselves web services

57. development environment composite service execution data schema definitions service composition model and language (usually characterized by a graphical and a textual representation) run-time environment (composition engine) schema designer the run-time environment executes the Web service business logic by invoking other services (through SOAP and HTTP modules) Web service composition middleware other Web services middleware (e.g., SOAP engine and conversation controller) supplier services offered by other providers warehouse accounting a service provider Copyright Springer Verlag Berlin Heidelberg 2004 Fundamental elements of a web service composition middleware

58. Web service (Composite) Web service Other tiers Service composition support (modeling and execution) Web services middleware The composite service is directly implemented at the middleware level, by the composition engine. Company A Company B Company C Company D Copyright Springer Verlag Berlin Heidelberg 2004 By using web service composition middleware, the implementation of a composite Web service resides in the Web Services middleware

59. Composition versus Coordination Middleware

60. supplier customer 1:requestQuote 2:orderGoods 4:makePayment 3:confirmOrder Conversation controller composition engine the procurement business protocol executed among Web services another Web service, possibly offered by another company yet another Web service if the supplier is implemented by means of composition technologies, then its business logic is defined by a composition schema and its execution is driven by a composition engine. depending on the implementation of the (composite) service, the supplier may contact other Web services. The customer is unaware of these interactions, that may occur according to other protocols. Copyright Springer Verlag Berlin Heidelberg 2004 Composition and coordination protocols have a different scope: external interaction versus internal implementation.

61. Web Service Composition Limitations of conventional middleware in supporting composition  Not standardized – different middleware, different solutions  Complex Web services potential  Simpler approach  Standardized interfaces that have broader acceptance

62. Service composition models Dimensions of a web service composition model  Component model Defines the nature of components  Orchestration model Abstractions and languages used to specify the order in which services needs to be invoked.  Data and data access model Deals with data definitions and communication protocols  Service selection model Nature of binding – static or dynamic  Transactions  Exception handling

63. invoke checkLocalStock invoke checkShipAvailable send confirmOrder inStock=false send cancelOrder inStock=true shippingAvail=true shippingAvail=false receive orderGoods supplier customer warehouse orderGoods confirmOrder cancelOrder checkShipAvailable local service offered by the supplier checkLocalStock Copyright Springer Verlag Berlin Heidelberg 2004 Supplier web service modeled as an activity diagram Orchestration modeled as an activity diagram

64. π-calculus Descendant of Communicating Sequential Processes (CSP) An attempt to develop a formal theory for process specification – similar to what relational algebra is to databases Offers constructs for composing activities that are parallel, sequential or conditional execution. Sample Notations  A.B  activity A happens before activity B  A|B  activity A and B happen in parallel  A+B  either A or B happens  [var = value] A  conditional execution of A

65. Orchestration modeled by π-calculus A = receiveOrderGoods.invokeCheckLocalStock B= [shippingAvail=false]sendCancelOrder + [shippingAvail=true]sendConfirmOrder C=invokeCheckShipAvailable.B Procurement=A.( ([inStock=false]C) + ([inStock=true]sendConfirmOrder) )

66. ON receive orderGoods IF true THEN invoke checkLocalStock; ON complete(checkLocalStock) IF (inStock==true) THEN send confirmOrder; ON complete(checkLocalStock) IF (inStock==false) THEN invoke checkShipAvailable; ON complete(checkShipAvailable) IF (shippingAvail ==true) THEN send confirmOrder; ON complete(checkShipAvailable) IF (shippingAvail ==true) THEN send cancelOrder; Copyright Springer Verlag Berlin Heidelberg 2004 Orchestration modeled by rules

67. Data and data transfer model

68. Data types Two general categories of data  Application specific  Control flow Control flow data also referred to as “process variables” tends to be simple Application data can be quite complex. XML is the natural choice though not necessarily the efficient one in some cases Binary attachments to XML are being standardized and this would make it possible to treat XML documents as containers for all sorts of data

69. Data Transfer Blackboard approach Explicit data flow

70. Service Selection

71. Approaches to service selection Static binding Dynamic binding by reference Dynamic binding by lookup Dynamic operation selection

72. UDDI Registry Newly added node that accesses a (statically specified) UDDI registry to determine which warehouse should be contacted for the product being ordered. The result is stored into the warehouse process variable. Note that in practice several invocations of the UDDI API may be needed to get the desired information Variables: warehouse: URI inStock, shippingAvail: bool customer: String … Subsequent nodes can use the reference approach and determine the URI based on the value of the warehouse variable invoke checkLocalStock invoke checkShipAvailable send confirmOrder inStock=false send cancelOrder inStock=true shippingAvail=true shippingAvail=false receive orderGoods supplier invoke get_bindingDetail Copyright Springer Verlag Berlin Heidelberg 2004 Combining service selection by reference with access to service directory

73. Transactions

74. Subtransaction S 1 Subtransaction S 2 … Subtransaction S n Long-lived transaction T (saga) Compensating Subtransaction CS 1 Compensating Subtransaction CS 2 … Compensating Subtransaction CS n Forward execution Compensation flow Copyright Springer Verlag Berlin Heidelberg 2004

75. Exception Handling

76. Variables: warehouse: URI inStock, shippingAvail: bool customer: String … invoke checkLocalStock invoke checkShipAvailable send confirmOrder inStock=false AND no errors returned send cancelOrder inStock=true AND no errors returned shippingAvail=true shippingAvail=false receive orderGoods supplier Some exception- handling logic here an error is returned or a timeout expired repeat the execution, if the exception handling logic so requires Copyright Springer Verlag Berlin Heidelberg 2004 Flow-based approach to exception handling

77. Other approaches to exception handling Try-catch-throw approaches  Clear separation of normal and exceptional logic  Exception logic can be associated to different levels of abstraction of the composition logic  Provides a mechanism to specify “continuation” logic i.e. what to do after the exception is handled Rule-based approaches

78. Conversation Controllers and Composition Engines

79. another Web service conversation controller composition engine service provider object (Web service implementation) object (Web service implementation) Instance of a composition schema instance of a composition schema messages related to protocols implemented by basic Web services (or anyway services implemented by means of conventional programming languages) messages related to protocols implemented through service composition technologies the engine has to match messages with instances, just like the conversation controller has to match messages with objects Copyright Springer Verlag Berlin Heidelberg 2004

80. BPEL: Business Process Execution Language for Web Services

81. supplier invoke checkLocalStock invoke checkShipAvailable invoke confirmOrderinvoke cancelOrder receive orderGoods customer warehouse local service offered by the supplier port types Abstract and/or executable process orchestration, variables and data transfers, exception handling, correlation information (for instance routing) Variables: warehouse: URI inStock, shippingAvail: bool customer: String … roles Copyright Springer Verlag Berlin Heidelberg 2004 Scope of the BPEL Specifications

82. BPEL: Component Model Consists of activities that are basic or structured Structured activities are used to specify orchestration Basic activities are the real components  Invoke  Receive  Reply  Assign  Wait

83. BPEL – Orchestration Model Sequence  Execute a sequence of activities Switch  Similar to switch command in “C” Pick  Association of a task with an event While  Execute task while condition holds Flow  Orchestrating parallel set of tasks

84. receive orderGoods invoke checkShipAvailable invoke checkLocalStock inStock=false processOrder sequence searchExternal sequence chooseLocal switch invoke confirmOrder chooseExternal switch invoke cancelOrderinvoke confirmOrder shippingAvail=trueshippingAvail=false inStock=true Copyright Springer Verlag Berlin Heidelberg 2004 Orchestration in BPEL

85. supplier customer warehouse local service offered by the supplier partner link definition : it further qualifies the interactions occurring through a partner link type. Its definition refers to a partner link type and specifies the role played by the composite service as well as the one played by the other partner <partnerLink name="customerP" partnerLinkType=“orderLT" myRole=“supplier” partnerRole=“customer”> partner link type orderLT port type supplierPT Copyright Springer Verlag Berlin Heidelberg 2004 Definition of partners in BPEL

86. receive orderGoods invoke checkShipAvailable invoke checkLocalStock inStock=false processOrder sequence searchExternal sequence chooseLocal switch invoke confirmOrder chooseExternal switch invoke cancelOrderinvoke confirmOrder shippingAvail=trueshippingAvail=false inStock=true scope of the searchExternal activity due to the behavior of the default handler, implicitly associated with each activity, a fault F occurring in activity send confirmOrder would propagate up until activity searchExternal, where the handler resides includes fault handler for fault F Copyright Springer Verlag Berlin Heidelberg 2004 Exception Handling in BPEL

87. supplier warehouse message checkAvailability orderID requestedDeliveryDate deliveryLocation … message availability orderID shippingAvail the orderID can be used for correlating the two messages Copyright Springer Verlag Berlin Heidelberg 2004 Correlating IDs to associate messages sent and received