1. ARCH-13 Transactions in an SOA World
Mike Ormerod
Product Manager & Applied Architect

2. Agenda Definition Through the ages Distributed Database Transactions
SOA Transactions
Compensation
Coordinator
Managed vs. Un-Managed
Summary
ARCH-13: Transactions in an SOA World

3. Definition : Transaction
“A transaction is a set of related changes to the database that the database either completes in its entirety or discards, leaving no modification to the database.”
OpenEdge Development: ABL Handbook
ARCH-13: Transactions in an SOA World

4. ACID Properties Atomic – All or nothing
Consistent – Start & End in a Consistent state
Isolated – transactions appear isolated from all other operations
Durable – once notified, the transaction will persist
Atomicity states that database modifications must follow an “all or nothing” rule. Each transaction is said to be “atomic.” If one part of the transaction fails, the entire transaction fails. It is critical that the database management system maintain the atomic nature of transactions in spite of any DBMS, operating system or hardware failure.
Consistency states that only valid data will be written to the database. If, for some reason, a transaction is executed that violates the database’s consistency rules, the entire transaction will be rolled back and the database will be restored to a state consistent with those rules. On the other hand, if a transaction successfully executes, it will take the database from one state that is consistent with the rules to another state that is also consistent with the rules.
Isolation requires that multiple transactions occurring at the same time not impact each other’s execution. For example, if Joe issues a transaction against a database at the same time that Mary issues a different transaction, both transactions should operate on the database in an isolated manner. The database should either perform Joe’s entire transaction before executing Mary’s or vice-versa. This prevents Joe’s transaction from reading intermediate data produced as a side effect of part of Mary’s transaction that will not eventually be committed to the database. Note that the isolation property does not ensure which transaction will execute first, merely that they will not interfere with each other.
Durability ensures that any transaction committed to the database will not be lost. Durability is ensured through the use of database backups and transaction logs that facilitate the restoration of committed transactions in spite of any subsequent software or hardware failures.
ARCH-13: Transactions in an SOA World

5. Agenda Definition Through the ages Distributed Database Transactions
SOA Transactions
Compensation
Coordinator
Managed vs. Un-Managed
Summary
ARCH-13: Transactions in an SOA World

6. Update or Read EXCLUSIVE-LOCK..
Host Based
Application
DO Transaction..
FOR EACH..
REPEAT..
DO ON ERROR..
Update or Read EXCLUSIVE-LOCK..

7. Update or Read EXCLUSIVE-LOCK..
Client-Server
Host Based
Application
Client
DO Transaction..
FOR EACH..
REPEAT..
DO ON ERROR..
Update or Read EXCLUSIVE-LOCK..
Server

8. Update or Read EXCLUSIVE-LOCK..
Client-Server
n-Tier / Stateless
Client
DO Transaction..
DO ON ERROR..
REPEAT..
FOR EACH..
AppServer
Server
Update or Read EXCLUSIVE-LOCK..
ARCH-13: Transactions in an SOA World

9. Common Infrastructure Update or Read EXCLUSIVE-LOCK..
n-Tier / Stateless
OERA
Presentation
Business Components
Data Access
Data Sources
Common Infrastructure
Enterprise Services
Client
AppServer
DO Transaction..
FOR EACH..
REPEAT..
DO ON ERROR..
Update or Read EXCLUSIVE-LOCK..
All the above work due to a locked state which is possible in a closed environment
ARCH-13: Transactions in an SOA World

10. OERA Transactions No Transaction Only State Transaction Start
Service Adapter
Enterprise Services
Business Entity
Business Task
Data Access Object
Data Source Object
Business Workflow
No Transaction
Only State
Transaction Start
Sub-Transaction
Start
Transaction Start
ARCH-13: Transactions in an SOA World

11. Agenda Definition Through the ages Distributed Database Transactions
SOA Transactions
Compensation
Coordinator
Managed vs. Un-Managed
Summary
ARCH-13: Transactions in an SOA World

12. Definition : Distributed Transaction
“A distributed transaction is a single transaction that updates two or more databases.”
OpenEdge Data Management: Database Admin
ARCH-13: Transactions in an SOA World

13. Distributed Transaction
Withdrawal
Banking
Application
Deposit X
A bank has two accounts, one on database acct1 and another on database acct2. The bank runs
an application that starts a transaction to withdraw a sum of money from acct1 and deposit it
into acct2. To keep the accounts in balance, it is critical that both operations—the withdrawal
and the deposit—succeed, or that they both fail. For example, if acct1 commits its part of the
transaction and acct2 does not, there is an inconsistency in the data.
ARCH-13: Transactions in an SOA World

14. Distributed Transaction: 2 Phase Commit
Ensures transaction consistency across all databases
All commit or none
Commit in 2 Phases
Complete or Roll Back
ARCH-13: Transactions in an SOA World

15. Distributed Transaction (2PC)
Commit
Commit
Withdrawal
Banking
Application
Deposit
Two-phase commit ensures that distributed transactions occur consistently across all databases.
Two-phase commit protects against inconsistency by making sure that all databases commit the
transaction, or that none commit. To ensure database integrity across all involved databases, the
database engine commits database updates in two distinct phases. During the first phase, the
database engine checks each database involved in a transaction to verify that it is ready to
commit the transaction. During the second phase, the database engine directs the databases to
commit the transaction and then verifies that they committed it properly. If there is an
inconsistency, the database engine displays error messages and allows you to complete or roll
back the inconsistent transaction to return the data to a consistent state.
Again possible due to closed environment and roll-back
ARCH-13: Transactions in an SOA World

16. Distributed Transaction – Limbo Transaction
Commit
Commit X
Withdrawal
Banking
Application
Deposit
A limbo transaction (also known as an in-doubt transaction) occurs if the coordinator database
commits or aborts a distributed transaction, but a hardware or software failure prevents other
databases from doing likewise. This is called a limbo transaction because the processing of the
transaction is temporarily suspended. A limbo transaction might occur for a variety of reasons;
for example, as a result of a power outage.
When a limbo transaction occurs, you must resolve the transaction to re-establish data
Consistency. Once the coordinator database establishes the status of a distributed transaction, it writes the
status to its BI and TL (transaction log) files. The TL file tracks the status of all distributed
transactions that affect the coordinator database.
Since the database engine continually overwrites the contents of the BI file, the TL file is
necessary to permanently record the status of a transaction. If you must resolve limbo
transactions, the transaction log file ensures that the coordinator has a reliable record of the
transaction.
To resolve limbo transactions, you must consult the coordinator database of each
transaction to see if the coordinator committed the transaction. If the coordinator database
committed the transaction, you must also commit the transaction on the database where
the limbo transaction occurred. If the coordinator did not commit the transaction, you must
terminate the transaction on the database where the limbo transaction occurred.
Again possible due to closed environment and roll-back
ARCH-13: Transactions in an SOA World

17. Agenda Definition Through the ages Distributed Database Transactions
SOA Transactions
Compensation
Coordinator
Managed vs. Un-Managed
Summary
ARCH-13: Transactions in an SOA World

18. SOA Transactions Functionality separated into Services
Application comprises many Services
Services as a separate platform
System Independent Entities
ARCH-13: Transactions in an SOA World

19. SOA Transaction Issues
Multiple services resulting in multiple endpoints
Loosely coupled systems
Maintaining Txn’s only possible in closely coupled systems
Services based on any platform
Resources can’t be kept in a locked state
Alternate Recover methods
Let's consider a scenario where the transaction involves the participation of three different services - each performing a particular operation. Only if all three operations are successful would the transaction be deemed a success. Any other outcome would result in the transaction being marked as a failure. Then, if and when the transaction fails, appropriate recovery measures have to be implemented. And to top it off, we can lock a resource only for the time when the service local to the resource is operating.
Let's look into the problems encountered with long-running transactions in SOA. They can be referred to as failure cases:
1) The participation of multiple services results in multiple endpoints being invoked during one cycle of the transaction. Any of these services can be down at the instance when the transaction is in process.
2) SOA boasts of loosely coupled systems. Maintaining transactions is only possible in closely coupled systems.
3) The services involved can be based on any platform. Because of the disparity among the underlying implementation of the services, a context can't be deployed across the services to manage the transactions.
4) The current status in the flow of the transaction can't be known at a given instance.
5) If asynchronous services are involved in the transaction they can't be reached back, unless the service information is explicitly passed on.
6) Resources can't be kept in a locked state for long periods of time. To free a resource once the service is done with it, it must release it. Doing this can cause a problem later on if the service fails and a rollback is issued throughout all the services.
7) Alternate methods need to be devised to perform the appropriate recovery operations. In most cases these methods are either platform-specific or too dependent on the underlying business process.
ARCH-13: Transactions in an SOA World

20. What’s needed Uniquely identify transaction across services
Guarantee that data is delivered and notifications sent
Some form of compensation for when it goes wrong
Handle errors in asynchronous services
ARCH-13: Transactions in an SOA World

21. Possible Solutions Compensation Transaction Coordinator
ARCH-13: Transactions in an SOA World

22. Agenda Definition Through the ages Distributed Database Transactions
SOA Transactions
Compensation
Coordinator
Managed vs. Un-Managed
Summary
ARCH-13: Transactions in an SOA World

23. Definition : Compensation
“Compensation is an action taken when something goes wrong or when there is a change of plan.”
IBM Systems Journal– April 2002
ARCH-13: Transactions in an SOA World

24. Compensation (5) Error/Exception thrown (7) Call Compensation
(1) Calls
(3) Calls
Service 1
Service 2
Service 3
(9) Return Failure
(6) Return error
Let's consider a scenario where the transaction involves the participation of three different services - each performing a particular operation. Only if all three operations are successful would the transaction be deemed a success. Any other outcome would result in the transaction being marked as a failure. Then, if and when the transaction fails, appropriate recovery measures have to be implemented. And to top it off, we can lock a resource only for the time when the service local to the resource is operating.
if the transaction fails at the third service (the transaction is uniquely identified by an id throughout its lifetime), we need to perform a compensatory operation at the previous service to negate the effect caused by the transaction. So, if the second service sent out an announcing that it has implemented the changes, a compensatory operation would be to send another announcing the failure of the transaction and that the changes have been undone.
Ok for sync, but what about async ?
(2) Change DB
(8) Apply Compensation
(4) Fails
ARCH-13: Transactions in an SOA World

25. Compensation Issues Asynchronous services Where to store state details
ARCH-13: Transactions in an SOA World

26. Agenda Definition Through the ages Distributed Database Transactions
SOA Transactions
Compensation
Coordinator
Managed vs. Un-Managed
Conclusion
ARCH-13: Transactions in an SOA World

27. Definition : Transaction Coordination
“Orchestration of transactions through a transaction manager or process coordinator.”
SOA Systems – Feb 2007
ARCH-13: Transactions in an SOA World

28. Coordination Service Coordinator Service (11) Return Failure (1) Call
Compensation
(10) Return
Service 1
Service 2
Service 3
(3) Change DB
(9) Apply Compensation
(6) Fails
ARCH-13: Transactions in an SOA World

29. Coordination Advantages
Fixes Asynchronous issues of Compensation
Manages state & service information
Central management of transaction & compensation
ARCH-13: Transactions in an SOA World

30. (generates transaction id and maintains ACID properties)
Money Transfer
Log
Coordinator Service
(generates transaction id and maintains ACID properties)
(5) Call
(2) Call
(8) Return status
(12) Return
status
(3) Return id
(9) Call
(1) request
Client
Initiation
Service
Credit
Service
Debit
Service
The coordinator receives the input and generates an id to uniquely identify the transaction. An acknowledgement is sent to the initiation service as RECEIVED. The initiation service notifies the client about the start of the process and provides the unique transaction id. The client can use this transaction id to monitor and track the transaction. The initiation service is now ready to take further client input. The coordinator maintains a log to record each operation it performs. The log is created against the transaction id.
After generating the id for the transaction, the coordinator calls the external service of the bank, which accepts the money. This credit service takes the necessary input and starts updating the records in the database. Depending upon the style of the compensation, state information is saved before the update process initiates. Once the update takes place successfully, an acknowledgement to the coordinator is sent.
The coordinator then logs the changes and proceeds to call the debit service. The debit service makes the necessary changes to the local database to reflect the debit. The debit process follows the same pattern as the credit process. On successful operation, a DEBITED acknowledgement is sent to the coordinator. The coordinator notifies each service involved of successful individual transactions at each step by means enacts the 2PC execution. When there's a failure, the coordinator runs the compensation service for each activity.
(4) Return id
(7) Return status
(11) Return
status
(6) Update DB
(10) Update DB
ARCH-13: Transactions in an SOA World

31. Agenda Definition Through the ages Distributed Database Transactions
SOA Transactions
Compensation
Coordinator
Managed vs. Un-Managed
Summary
ARCH-13: Transactions in an SOA World

32. OpenEdge Reference Architecture
Presentation
Enterprise Services
Common Infrastructure
Business Components
Data Access
Open Edge Reference Architecture (OERA) provides a guideline of how a Web service can fit into the framework of an application.
A Web service definition is one type of physical implementation for the logical definition of a Service Oriented Architecture (SOA).
Data Sources
ARCH-13: Transactions in an SOA World

33. Definition : Managed Transaction
“A Managed Transaction is managed by the transaction manager that is built into the OpenEdge platform. A Managed Transaction can only access Managed Data Sources”
OERA Quick Reference Guide
ARCH-13: Transactions in an SOA World

34. Definition : Un-Managed Transaction
“An Un-Managed Transaction is a transaction that updates information in an Un-Managed Data Source or Enterprise Service. An Un-Managed transaction is not managed by the transaction manager of the OpenEdge platform.”
OERA Quick Reference Guide
ARCH-13: Transactions in an SOA World

35. Managed or Un-Managed Transaction?
Service Interface
Business Entity
Managed
Data Access Object
Data Source Object OE
ARCH-13: Transactions in an SOA World

36. Managed or Un-Managed Transaction?
Service Interface
Business Entity
Managed
Data Access Object
Data Source Object
Data Source Object OE OE
ARCH-13: Transactions in an SOA World

37. Managed or Un-Managed Transaction?
Service Interface
Business Entity
Un-Managed
Data Access Object
Data Source Object
Data Source Object OE
ARCH-13: Transactions in an SOA World

38. Managed or Un-Managed Transaction?
Service Interface
Business Task
Managed
Business Entity
Business Entity
Data Access Object
Data Access Object
Data Source Object
Data Source Object OE OE
ARCH-13: Transactions in an SOA World

39. Managed or Un-Managed Transaction?
Service Interface
Business Task
Un-Managed
Business Entity
Business Entity
Data Access Object
Data Access Object
Data Source Object
Data Source Object OE
ARCH-13: Transactions in an SOA World

40. Un-Managed Enterprise Services Enterprise Services Business Workflow
Service Adapter
Enterprise Services
Business Entity
Business Task
Data Access Object
Data Source Object
Business Workflow
Un-Managed
ARCH-13: Transactions in an SOA World

41. Common Infrastructure
Money Transfer
Common Infrastructure
Log
Coordinator Service
(generates transaction id and maintains ACID properties)
Business Workflow
(5) Call
(2) Call
(8) Return status
(12) Return
status
(3) Return id
(9) Call
(1) request
Business Components
Data Access
Business Components
Data Access
Client
Credit
Service
Business Components
Data Access
Initiation
Service
Debit
Service
The coordinator receives the input and generates an id to uniquely identify the transaction. An acknowledgement is sent to the initiation service as RECEIVED. The initiation service notifies the client about the start of the process and provides the unique transaction id. The client can use this transaction id to monitor and track the transaction. The initiation service is now ready to take further client input. The coordinator maintains a log to record each operation it performs. The log is created against the transaction id.
After generating the id for the transaction, the coordinator calls the external service of the bank, which accepts the money. This credit service takes the necessary input and starts updating the records in the database. Depending upon the style of the compensation, state information is saved before the update process initiates. Once the update takes place successfully, an acknowledgement to the coordinator is sent.
The coordinator then logs the changes and proceeds to call the debit service. The debit service makes the necessary changes to the local database to reflect the debit. The debit process follows the same pattern as the credit process. On successful operation, a DEBITED acknowledgement is sent to the coordinator. The coordinator notifies each service involved of successful individual transactions at each step by means enacts the 2PC execution. When there's a failure, the coordinator runs the compensation service for each activity.
(4) Return id
(7) Return status
(11) Return
status
(6) Update DB
(10) Update DB
ARCH-13: Transactions in an SOA World

42. Common Infrastructure Enterprise Service Bus
Money Transfer – ESB
Common Infrastructure
Dynamic Process Launcher
Log
Enterprise Service Bus
Business Workflow
Coordinator Service
(generates transaction id and maintains ACID properties)
(5) Call
(2) Call
(8) Return status
(12) Return
status
(3) Return id
(9) Call
(1) request
Business Components
Data Access
Business Components
Data Access
Business Components
Data Access
Client
Credit
Service
Initiation
Service
Debit
Service
Create an external repository and drive process definition through meta-data rather than manually creating processes in the bus will like this feature. What about fetching process information from an external web service and dynamically constructing a process based on the return of the web service?
The coordinator receives the input and generates an id to uniquely identify the transaction. An acknowledgement is sent to the initiation service as RECEIVED. The initiation service notifies the client about the start of the process and provides the unique transaction id. The client can use this transaction id to monitor and track the transaction. The initiation service is now ready to take further client input. The coordinator maintains a log to record each operation it performs. The log is created against the transaction id.
After generating the id for the transaction, the coordinator calls the external service of the bank, which accepts the money. This credit service takes the necessary input and starts updating the records in the database. Depending upon the style of the compensation, state information is saved before the update process initiates. Once the update takes place successfully, an acknowledgement to the coordinator is sent.
The coordinator then logs the changes and proceeds to call the debit service. The debit service makes the necessary changes to the local database to reflect the debit. The debit process follows the same pattern as the credit process. On successful operation, a DEBITED acknowledgement is sent to the coordinator. The coordinator notifies each service involved of successful individual transactions at each step by means enacts the 2PC execution. When there's a failure, the coordinator runs the compensation service for each activity.
(4) Return id
(7) Return status
(11) Return
status
(6) Update DB
(10) Update DB
ARCH-13: Transactions in an SOA World

43. Enterprise Service Bus
Money Transfer – BPEL
BPEL Service
Log
Enterprise Service Bus
Coordinator Service
(generates transaction id and maintains ACID properties)
Business Workflow
(5) Call
(2) Call
(8) Return status
(12) Return
status
(3) Return id
(9) Call
(1) request
Client
The coordinator receives the input and generates an id to uniquely identify the transaction. An acknowledgement is sent to the initiation service as RECEIVED. The initiation service notifies the client about the start of the process and provides the unique transaction id. The client can use this transaction id to monitor and track the transaction. The initiation service is now ready to take further client input. The coordinator maintains a log to record each operation it performs. The log is created against the transaction id.
After generating the id for the transaction, the coordinator calls the external service of the bank, which accepts the money. This credit service takes the necessary input and starts updating the records in the database. Depending upon the style of the compensation, state information is saved before the update process initiates. Once the update takes place successfully, an acknowledgement to the coordinator is sent.
The coordinator then logs the changes and proceeds to call the debit service. The debit service makes the necessary changes to the local database to reflect the debit. The debit process follows the same pattern as the credit process. On successful operation, a DEBITED acknowledgement is sent to the coordinator. The coordinator notifies each service involved of successful individual transactions at each step by means enacts the 2PC execution. When there's a failure, the coordinator runs the compensation service for each activity.
(4) Return id
(7) Return status
(11) Return
status
(6) Update DB
(10) Update DB
ARCH-13: Transactions in an SOA World

44. BPEL Compensation Handler
Fault Handler
Process (Scope 1)
Compensation Handler S2
Scope 2
Snapshot of Scope 2
Compensation Handler S3
Scope 3
Snapshot of Scope 3
When executing the process, the first scope P (the process itself) declares a variable V1 and initializes it to the value of 0. Scopes S2 and S3 are executed. At successful completion of S2 and S3, all variable values are set to 1 and are frozen into snapshots. Subsequently, a fault occurs within the process P (indicated by red dot in the picture), which gets caught by the fault handler FH(P) of the process P. When the fault handler of the process calls the compensation handler CH(S2) of scope S2,the snapshot of S2’s state is retrieved and used while compensating. The same applies when compensating scope S3.
Time
ARCH-13: Transactions in an SOA World

45. Agenda Definition Through the ages Distributed Database Transactions
SOA Transactions
Compensation
Coordinator
Managed vs. Un-Managed
Summary
ARCH-13: Transactions in an SOA World

46. In Summary ACID is good, taken in short doses!
Plan, Design & Architect for target implementation
Choose the appropriate transaction model for you
Consider ESB & BPEL
ARCH-13: Transactions in an SOA World

47. For More Information, go to…
PSDN
Service-oriented Architecture (SOA) (
Forums
Web Sites
Sonic Software
ARCH-13: Transactions in an SOA World

48. Relevant Exchange Sessions
Sonic-9: Introduction to BPEL
INT-9: Implementing ESB Processes with OpenEdge & Sonic
ARCH-13: Transactions in an SOA World

49. Questions?
ARCH-13: Transactions in an SOA World

50. Thank you for your time
ARCH-13: Transactions in an SOA World

51. ARCH-13: Transactions in an SOA World

52. Progress PPT Color Palette RGB Values
Default Background
Default Text
Default Shadows
Default Title Text
Default Fills
Default Accent
Default Accent
Default Accent
R-255 G-255 B-255
R-0 G-0 B-0
R-192 G-192 B-192
R-0 G-75 B-133
R-238 G-168 B-44
R-0 G-186 B-151
R-210 G-30 B-39
R-0 G-102 B-204
R-0 G-108 B-97
R-58 G-95 B-111
R-95 G-95 B-95
R-77 G-129 B-169
R-242 G-195 B-62
R-0 G-72 B-57
R-122 G-0 B-59
R-43 G-22 B-94
R-0 G-128 B-128
R-117 G-144 B-155
R-119 G-119 B-119
R-153 G-183 B-207
R-253 G-215 B-92
R-102 G-102 B-51
R-193 G-4 B-55
R-0 G-75 B-133
R-0 G-183 B-166
R-176 G-191 B-198
R-165 G-165 B-165
R-147 G-194 B-229
R-253 G-224 B-132
R-51 G-153 B-102
R-220 G-30 B-55
R-0 G-155 B-230
R-229 G-229 B-229
R-154 G-203 B-240
R-255 G-255 B-153
R-48 G-200 B-193
Session ID: Session Title