Distributed Transaction & Long-running transactions Rossen Zhivkov Freelance SharePoint Consultant January 19 th, 2008 Sofia, Bulgaria Krasimir Parushev Consultant, ITCE ltd.

What if… What if you have to update a SQL Server database and Oracle database in one transaction? What if you have to call two web services in one transaction? What if you have to implement transaction that lasts for days?

What is… What is distributed transaction What is 2-phase commit protocol What is transaction coordinator What is resource manager What is long-running transaction WS-Coordination, WS-AtomicTransaction, WS- Business Activity What is BPEL4WS

Roles in Transaction System Application Begins the transaction Makes changes to resources Can Commit or Abort the transaction Transaction Manager Transaction demarcation, participants, and state Prepare result is made durable Can Abort the transaction Resource Managers Controls changes to resources under the scope of a transaction Uses a (durable) log for recovery Can Abort the transaction Need to participate in 2PC (Atomicity) Provide Isolation of its resources Provide Durability of its resources

Local Transactions DBMS Application Application Activity Activity Activity ADOConn Client 1 Client 2 Client N Transaction Manager LockManager Data VB Object ADOConn ADOConn

Local Transactions All updates within a single DBMS Or other “resource manager” (RM) DBMS provides transaction demarcation Begin (start), end (commit), abort (rollback) DBMS provides transaction manager DBMS provide lock manager

Computer 3 RM3 - ORACLE Participating DTC Computer 2 RM2 -SQL Server Participating DTC Computer 1 The Coordinating DTC RM1 - MSMQ A Distributed Transaction Data Data Your Tx COM+ Application Queue RM Proxy

Distributed Transactions Transactions may span multiple RMs Databases, Queues, etc. Distributed Transaction Manager Provides common transaction demarcation Coordinates commit/abort across RMs 2 Phase Commit (2PC) Protocol RMs are “prepared” to either commit or abort RMs are notified of final outcome RM recovery Resource Managers Need to participate in 2PC (Atomicity) Provide Isolation of its resources Provide Durability of its resources

Two-Phase Commit Transaction Manager Dispenses new transactions to clients Tracks the resource managers participating in the transaction Coordinates the transaction outcome Resource Manager A subsystem that allows updates to transaction protected data SQLServer Transaction ManagerSQLServer FileSystem Resource Managers Resource Manager

Two-Phase Commit Transaction Manager Resource Manager Prepare Commit Prepare and Force write “Prepared” record to RM log Prepared Force write “Commit” record to TM log Lazy write “Committed” record to RM log and Release locks Committed Lazy write “Committed” record to TM log

Two Phase Commit – Failure Transaction Manager Resource Manager Prepare Commit Prepare & Write “Prepared” record to RM log Prepared Write “Commit” record to TM log Write “Committed” record to RM log & Release locks Committed Write “Committed” record to TM log Abort on Failure Commit Abort on Failure Indoubt on Failure Commit Will Abort Will Commit

Two Phase Commit Transaction manager coordinates updates made by resource managers Phase 1 Prepare to commit Phase 2 Commit Transaction manager always knows the state of the transaction

Phase 1 Transaction manager asks all RMs to prepare to commit. RMs can save their intended changes and then say ‘yes’. Any RM can say ‘no’. No RM actually commits yet! If all RMs said ‘yes’, go to Phase 2. If any RMs said ‘no’, tell everyone to abandon their intended changes.

Phase 2 Transaction manager asks all resource managers to go ahead and commit their changes. Can now recover from failure RM knows what transactions were questionable at point of failure TM knows whether transactions succeeded or failed

LIVE DEMO

MS-DTC Distributed Transaction Coordinator Windows NT Service Two Phase Commit (2PC) and Recovery Infrastructure COM Interfaces (a.k.a., DTC proxy) Used for App  TM, TM  RM communication Local TM, remote TM, TM failover Supported Resource Managers (RM) SQL Server, MSMQ, BizTalk, COM+ CRM, … Oracle, DB2, …

SQL Server And Dist Tx Client initiated distributed transaction Existing DTC transaction is passed to SQL Server using data access API ODBC SQL_ATTR_ENLIST_IN_DTC connection attribute OLE-DB ITransactionJoin::JoinTransaction ADO Only implicit via MTS/COM+ System.Data.SqlClient v1.1 SqlConnection.EnlistDistributedTransaction System.Transactions Later… DB-Library – dbenlisttrans COM+, System.EnterpriseServices, WCF

SQL Server And Dist. Tx… Server initiated BEGIN DISTRIBUTED TRANSACTION Linked Server call Extended Stored Procedures X-Transactions Partitioned Tables NT SQLServerSQLServer MS DTCApplication

Using distributed transactions in.NET System.Transactions System.Transactions = API for programming local & distributed transactions Enlisting resources in distributed transactions Ambient Transactions Transaction Promotion Transaction Flow

Transactions from.NET application

Custom Resource Managers Why to write own resource managers Incorporate custom resources in distributed transactions Implement logic for two-phase commit steps (Prepare, Commit, Rollback) Volatile resource managers Transactional variables Transactional collections Transactional files…. Sample

Transactional NTFS & Registry Kernel Transaction Memory Kernel objects in transaction (distributed / local) Allows atomic operations for: Many operations on single file Operations for multiple files Operations for many computers Registry modifications Sample

Windows Communication Foundation What is WCF? Web services + WSE + WS-specifications + MSMQ + EnterpriseServices(COM+) + Remoting Reliable messaging Securable services Protocols support New abstraction level Data Abstraction Communication Abstraction

WCF and Transactions WCF services & other resource managers can be executed within the scope of distributed transaction Transactable Services In the Internet – using WS-AtomicTransaction In the Intranet - DTC, OleTransaction protocol, System.Transaction MSDTC and WS-AtomicTransactions Transaction Flow

Incorporate web services in distributed transactions using WCF

Long-running transactions The business scenario We ”cannot” lock the service sessions (and the underlying database) for that long time Are ACID properties maintained? Is it OK to hold locks for days Compensation actions Logic to “undo” the effects from previously committed action

LIVE DEMO : Long-running transactions

WS-* specifications WS-Coordination Distribution of the transaction context WS-AtomicTransaction Pure ACID for short transactions with 2PC or similar Locking resources WS-Business Activity Long living transactions To coordinate large work implemented by several small WS- AtomicTransaction services Compensations SOAP “compensate” message

Business Process Execution Language For Web Services - BPEL4WS What is BPEL4WS: Orchestrating web services to implement business process Complex transactions modeled as … WORKFLOW Provides sequences, branches, iterations, etc Bound to local scope Use WS-BusinessActivity to incorporate distributed scopes

Comparison: Distributed vs. Long-running Transaction trees In distributed transactions: Parent NEVER commits before the child commits (2PC) Parent / Child hold locks till end of transaction In long-running transactions: Parent can commit independently of the child transaction If Child then rollbacks, parent executes compensation logic Parent / Child release locks immediately after commit

Workflow Foundation Business Processes orchestration Support for Distributed transactions via TransactionScope Support for Long-running transactions via Compensations Persistence and Tracking Integration with WCF & Web Services Published as WCF/web service Utilizes WCF/Web service

Implement long-running transactions in Windows Workflow Foundation

Summary Distributed Transactions 2-Phase Commit Protocol SQL Server 2005 & Distributed transactions.NET & Distributed Transactions Custom Resource Managers Distributed Transactions in Web Services Windows Communications Foundation & WS- AtomicTransaction Long-running transactions Workflow Foundation BPEL4WS & WS-BusinessActivity Next steps

Thank You for you time Rossen Zhivkov Krasimir Parushev