Managing Process Integrity (Chapter 8) B. Ramamurthy 1/12/2019

Topics for discussion Review of last lectures: Services layers diagram (Ch. 1-7) Webservices (design and deployment) Relate these two in the context of design, development, deployment cycle of an SOA. Lets understand WSDL http://www.w3.org/TR/wsdl#_wsdl http://www.w3schools.com/wsdl/wsdl_documents.asp http://webservices.amazon.com/AWSECommerceService/AWSECommerceService.wsdl? Lets look at the components of WSDL and look at Amazon.com ECS as example Review for midterm Process Integrity 1/12/2019

Anatomy of a WSDL document 1/12/2019

Sample WSDL http://www.w3.org/2001/04/wsws-proceedings/uche/wsdl.html http://webservices.amazon.com/AWSECommerceService/AWSECommerceService.wsdl? 1/12/2019

Significance of WSDL WSDL for a web service (WS) when specified in an IDE (such as Netbeans) exposes the operations and the signature for the operations. This can be used access the operations of the WS. When designing business systems using BPEL, WSDL represents the requirements of a process in a BPEL. Business BPEL WSLD  services (SOA) Thus WSDL is the link that aligns a business process with an IT service. 1/12/2019

Data integrity Consistency Accuracy Correctness of data Entity integrity: each row in relational table must be uniquely identified Domain integrity: the values must fall within a specified value range Referential integrity: specifies the validity of relationship between different data sets. 1/12/2019

Process integrity Business processes span multiple IT systems. There are times when a process state may be inconsistent. For example, a cancelled order takes some time to register with order system. Technical failures such as network failure: system-wide solution Business exceptions must be handled at the process level: WSDL fault provisions Special cases: overbooking of a flight (cannot be handled by domain integrity.) Must be handled at code level. Process ownership is another problem. For example, consider a workflow application. 1/12/2019

Process integrity solutions Logging and tracing ACID transactions Atomicity, consistency, isolation, durability Transaction monitors 2PC: two-phase commit protocol is used to ensure ACID properties for transactions that span more than a single resource manager. Phase 1: all participating resource managers acquire locks on shared resources Depending on phase transaction coordinator informs participants whether to commit or rollback Phase 2: participants either commit or rollback to previous state depending on instruction from coordinator Issues: performance, lack of support for long-lived transactions, integration of legacy systems, not suited for discontinuous network, use of multi-level and nested transactions 1/12/2019

Process integrity solutions: transactional steps Combination of transactional steps and persistent queue can improve reliability of complex processes. Transactional steps are a set of closely related activities executed in the context of a single complex transaction. Transactional steps facilitate the decomposition of complex, long-lived business processes into individual steps with short execution times and high transactional integrity. Transactional steps dramatically increase the robustness and flexibility of distributed systems. ~ provide a great way of ensuring the integrity of individual process steps. 1/12/2019

Process integrity solutions transaction chain & compensation ACID properties are too strong for complex transactions Chained transactions with compensating transactions offer a viable means of dealing with process integrity. Compensating transactions undo previously executed steps if a problem is encountered during the execution of a particular step in a transaction chain. http://www.serviceoriented.org/ws-transaction.html 1/12/2019

Sagas Sagas are formal workflow models that build on the concept of chained transactions (steps). A Saga describes a workflow wherein each step associated with a compensating transaction. How do you compensate “out of funds” failure? 1/12/2019

BPM and process integrity BPM (Business Process Management) platforms provides features that enable business analysts and developers to design, execute and manage instances of complex business process. Introduction of BPM will help the process integrity aspects of an enterprise application. Explicit modeling separates technical integrity from process integrity BPM engine provides mechanisms for monitoring processes at runtime BPM supports compensating transactions which is key concept for managing the rollback of partially executed processes after a failure. 1/12/2019

Related WS-Standards See fig. 8-4 A number of different industry alliances and standardization bodies are working on WS-based transaction protocols. OASIS Business Transaction Processing (BTP) WS Flow language WS-BPEL BPEL4WS XLANG WS-Composite Application Framework WS-coordination WS-Transaction … 1/12/2019

Process Integrity Spectrum 2PC/TPM in Heterogeneous env. 2PC/TPM in homogeneous env. Level of integrity Sagas with complex compensation BPM engine with Simple compensation Transactional Steps Custom-built log infrastructure DB log analyzer Eyes closed & Praying Implementation cost 1/12/2019

Mid-term review Date: 11/29 Duration: 1 hour 30min Topics: 1-7 Chapters Closed book Topics for questions: Fundamental concepts: synchrony, loose coupling etc., interface vs. payload semantics Service types Design of an SOA for a application: layered architecture 1/12/2019

Process Integrity in SOA Case Study: Travel Itinerary Management Analyze the problem Choose concurrency control: optimistic vs pessimistic Use idempotent update operations Avoid distributed 2PC Build transactional steps Design simple and flexible compensations Combine SOA, MOA and BPM for flexibility 1/12/2019

Problem Statement & Analysis (8.3.1 & Fig.8-5) Typical sites enable you to book individual flights. The new travel itinerary mgt. system should create complete itineraries for trips including car, hotel and flights. Reuse existing IT subsystems esp. customer management and billing systems. Two different front-ends: simple html front and a complex front-ends for call center (VB GUI) functionality Three backend systems: customer management, billing, and other complex processes such as customer complaints etc.) A number of partner systems need to be integrated; example: partner airlines, hotel and car reservation systems. Key transactions are: confirm itinerary (typically irreversible) and create invoice; these transactions cross organizational boundaries cross several technical boundaries involve different databases involve different departments 1/12/2019

Architectural model 1/12/2019 Enterprise Reservation systems (cars, hotels, other airlines) Enterprise Reservation systems (cars, hotels, other airlines) Customer Customer & Itinerary DB Web front-end IncidentManager Process state DB Billing Callcenter front-end Invoice DB 1/12/2019

Concurrency Control (8.3.2, figs.8-6, 8-7) This specifies how to deal with concurrent access to shared data in an SOA. Optimistic and pessimistic Pessimistic control gives exclusive access to data through acquisition of locks; locks typically are held for short duration. Optimistic control: no locks are acquired during the transaction execution, only when data needs to be updated at the end of the transaction. Optimistic control is the choice for long running transactions. 1/12/2019

Implementing Optimistic Concurrency Control In order to determine write conflicts, optimistic concurrency model must, at the end of each transaction, determine whether a competing transaction has changed the data after it was initially checked out. Different ways: timestamps, version counts, state comparisons For the first two an extra column is added in the relational table Version number or timestamp is added to the primary key used in the WHERE clause of UPDATE operation State comparison is implemented in Microsoft’s ADO.NET; advantage: does not require extra column in the relational table Version numbers are associated with a service at root level If a client is a trusted party then version number is preferable solution; else state comparison 1/12/2019

Example Customer profile Customer updates using web || call center updates using telephone Customer and call center read profile version #645 Customer updates, saves, version #646 Call center looses its updates, needs redo due version conflict Alternatively, we could allow merge operation; possible with state comparison If nothing works resort to pessimistic control, such as in an incident management service 1/12/2019

Optimistic concurrency control Read customer profile update customer Version conflict Commit changes Version inc. no yes Merge possible yes Notify user yes merge changes 1/12/2019

Pessimistic concurrency control Get a lock on Incident object Lock Avail? no Notify user yes Work on Incident yes Possibility for discard changes as well as supervisor notification no Commit changes Release lock 1/12/2019

Make updates Idempotent Effect of a N (N > 1) of the same requests is same as one request. Use sequence numbers interface Itinerary { SQN getSeqeunceNumber(); void addBooking (in SQN s, in booking b); } Can be used in error handling 1/12/2019