Synchronization outline Distributed transactions – part 1 Fowler’s locking patterns Implementing distributed transactions Logical clocks (aka distributed timestamps) Mutual exclusion across a network Other topics you may want to read about: Physical clocks (6.1.1) Clock synchronization (6.1.3) Election across a network (6.5) 4/26/2019 Distributed Systems - Comp 655

Distributed transactions – part 1 Why is it “part 1”? What is a transaction? ACID properties Implementing atomicity Implementing isolation 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Why “part 1”? Issues related to keeping replicated copies of a data store in synch will be covered in Chapter 7, “Consistency and Replication” Issues related to maintaining ACID properties in the presence of failure will be covered in Chapter 8, “Fault Tolerance” 4/26/2019 Distributed Systems - Comp 655

Distributed transactions – part 1 Why is it “part 1”? What is a transaction? ACID properties Implementing atomicity Implementing isolation 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 What is a transaction? A commitment to exchange goods and/or services and/or data and/or money In software systems, a commitment to change one or more (usually more) data items 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Transaction example Example: transfer $1,000 from savings to checking Implementation: Begin-transaction Read savings balance Reduce savings balance by $1,000 Read checking balance Increase checking balance by $1,000 End-transaction 4/26/2019 Distributed Systems - Comp 655

Transaction primitives 4/26/2019 Distributed Systems - Comp 655

Distributed transactions – part 1 Why is it “part 1”? What is a transaction? ACID properties Implementing atomicity Implementing isolation 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 ACID properties Atomicity – all or nothing Consistency – system integrity constraints are preserved Isolation (aka serializability) – externally-observable changes due to a set of transactions are consistent with some serial ordering Durability – changes are permanent 4/26/2019 Distributed Systems - Comp 655

The A of ACID: atomicity Example: transfer $1,000 from savings to checking Implementation: Begin-transaction Read savings balance Reduce savings balance by $1,000 Write updated savings balance Read checking balance Increase checking balance by $1,000 Write updated checking balance End-transaction If something fails here, it’s as if nothing happened at all 4/26/2019 Distributed Systems - Comp 655

The I of ACID: isolation The root problem we’re trying to solve: running transactions often block, waiting for I/O, so we want to increase throughput by interleaving the activities of multiple transactions. (in distributed systems, I/O wait times can be extra long, if network I/O is involved) The resulting problem we’re trying to solve: provide concurrency transparency by making sets of interleaved transactions look as if they ran one at a time. 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Quick quiz How many operations is this? x = x + 1 Answer: 3 Read x Add 1 Write x Whether the read and write go between memory and register or database and process memory, there’s a risk of data inconsistency … 4/26/2019 Distributed Systems - Comp 655

If you remember only one thing … 7 T x 1. T reads x 8 7 T x U 2. T sets x = 8 while U reads x 8 T x 3. T writes x, and U may do the wrong thing 7 U transaction database 4/26/2019 Distributed Systems - Comp 655

Isolation example: how interleaving could cost you money Transaction T: transfer $1,000 from savings to checking Transaction U: deposit $100 to checking Starting balances: $5,000 savings $1,000 checking Ending balances should be $4,000 savings $2,100 checking 4/26/2019 Distributed Systems - Comp 655

Isolation example, part 2: the steps of T and U T does this: Read savings Subtract 1,000 Write savings Read checking Add 1,000 Write checking U does this: Read checking Add 100 Write checking Remember: each transaction has private in-memory account balances that may be different from the public values 4/26/2019 Distributed Systems - Comp 655

Non-isolation can be bad … Imagine this sequence of interleaved operations: T reads savings = 5,000 T writes savings = 5,000 - 1,000 = 4,000 T reads checking = 1,000 U reads checking = 1,000 T writes checking = 1,000 + 1,000 = 2,000 U writes checking = 1,000 + 100 = 1,100 My $1,000 disappeared! 4/26/2019 Distributed Systems - Comp 655

Distributed transactions – part 1 Why is it “part 1”? What is a transaction? ACID properties Implementing atomicity Implementing isolation 4/26/2019 Distributed Systems - Comp 655

Context: general transaction management 4/26/2019 Distributed Systems - Comp 655

Context: managing distributed transactions 4/26/2019 Distributed Systems - Comp 655

Data manager implements atomicity 4/26/2019 Distributed Systems - Comp 655

Implementing atomicity Two main approaches: Private workspace Writeahead log with rollback 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Private workspace 4/26/2019 Distributed Systems - Comp 655

Writeahead log with rollback 4/26/2019 Distributed Systems - Comp 655

Distributed transactions – part 1 Why is it “part 1”? What is a transaction? ACID properties Implementing atomicity Implementing isolation 4/26/2019 Distributed Systems - Comp 655

Scheduler implements isolation 4/26/2019 Distributed Systems - Comp 655

Basic concept: a schedule A schedule for a set of transactions is an ordering of the steps of the transactions such that Each step of each transaction appears exactly once The steps of any given transaction appear in order 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Schedule examples Transaction T: T.1 T.2 T.3 T.4 Some schedules T.1 T.2 T.3 T.4 U.1 U.2 U.1 U.2 T.1 T.2 T.3 T.4 T.1 T.2 T.3 U.1 T.4 U.2 T.1 U.1 T.2 U.2 T.3 T.4 Transaction U: U.1 U.2 Quick exercise: How many schedules for T and U are there? 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Next basic concepts A serial schedule for a set of transactions is a schedule in which the steps of each transaction are consecutive A serializable schedule for a set of transactions and a set of initial data values is a schedule whose final data values are the same as those of some serial schedule 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Schedule examples Initial data: S 5,000 C 1,000 T.1 T.2 T.3 T.4 U.1 U.2 U.1 U.2 T.1 T.2 T.3 T.4 T.1 T.2 T.3 U.1 T.4 U.2 T.1 U.1 T.2 U.2 T.3 T.4 S 4,000 C 2,100 serial S 4,000 C 2,100 serial S 4,000 C 1,100 not serializable S 4,000 C 2,100 serializable 4/26/2019 Distributed Systems - Comp 655

Activity: serializability Decide whether a variety of schedules are serializable (see this week’s Agenda for details) 4/26/2019 Distributed Systems - Comp 655

Two ways to be serializable A schedule can be serializable thanks to Dumb luck that is, it just happens to produce the same result as some serial schedule Make it so that is, the schedule resulted from following principles that guarantee serializability Only the second is interesting for building systems … 4/26/2019 Distributed Systems - Comp 655

How to ensure serializability Ignore what is done to any data item The essential thing is that it is being read or written 4/26/2019 Distributed Systems - Comp 655

Transactions from the activity Initially, A=1, B=0, C=0 X.1 read A X.2 read C X.3 write B = C+1 X.4 write A = A-C Y.1 read B Y.2 read A Y.3 write C = A+B 4/26/2019 Distributed Systems - Comp 655

New view: conflicts only X.1 read A X.2 read C X.3 write B X.4 write A Y.1 read B Y.2 read A Y.3 write C 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Conflicts Order only matters when operations conflict Conflict occurs when Two operations operate on the same data item At least one of the operations is a write That makes two types of conflicts: Read-write Write-write 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Example: conflicts Transaction T: X = X + 1 Public data Transaction U: X = 7 X = 0 Read X makes TX = 0 UX = 7 read/write X = 7 X = X + 1 makes TX = 1 Write UX to X write/write X = 1 Write TX to X The read/write conflict forces T before U; the write/write conflict forces U before T; therefore this is not a serializable schedule. 4/26/2019 Distributed Systems - Comp 655

To guarantee serializability, you have to manage conflicts Implementation tools: Mutual exclusion (aka “locking”) Ordering via timestamps Implementation attitudes: Pessimistic Optimistic 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Three approaches Two-phase locking Pessimistic, based on mutual exclusion Pessimistic timestamp ordering (PTO) Optimistic timestamp ordering (OTO) We will cover locking in much more detail than timestamp ordering 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 The locking approach Allow transactions to ‘lock’ a data item Only the transaction that has a lock on a data item is allowed to use it A transaction has to wait until a data item is unlocked before it can lock it and use it If locks are managed properly, serializability can be guaranteed First try: two-phase locking Second try: strict two-phase locking 4/26/2019 Distributed Systems - Comp 655

Two-phase locking (2PL) rules When transaction T requests access to data item x, If x is not locked, T gets a lock on x If x is locked, T must wait until x becomes unlocked A lock is never released until the data operation is complete After transaction T releases a lock, it can never acquire another one 4/26/2019 Distributed Systems - Comp 655

Transaction lifecycle in 2PL 4/26/2019 Distributed Systems - Comp 655

2PL guarantees serializability The order in which transactions lock each data item defines a partial order on the set of transactions Pick any total order for the transactions that’s consistent with all of the partial orders 2PL guarantees that there will be at least one The schedule’s results on any set of initial data values will be the same as the results of running the transactions one at a time according to the total order 4/26/2019 Distributed Systems - Comp 655

Durability problem: cascaded abort Initial state: x = y = 0 T1 locks x T1: x = 1 T1 locks y T1: y = 1 T1 releases x T2 locks x T2: x = x + 1 T2 releases x At this point, x=2, y=1 T1 aborts Rolling back T1 requires rolling back T1’s writes AND any transaction (T2, for instance) that used its results 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Strict 2PL 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Problem: deadlock T2 requests x and waits for T1 T1 locks x x T1 T2 locks y T2 T1 requests y and waits for T2 y 4/26/2019 Distributed Systems - Comp 655

Approaches to deadlock Detection: look for cycles in the waits-for graph. When found, pick a member of the cycle and abort it. Prevention: Define an order for lockable resources Transactions always acquire resources in that order 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Deadlock prevention T2 requests x and waits for T1 T1 locks x x T1 T2 T1 locks y y When T1 completes, it releases x and y. T2 is granted a lock on x and resumes. It then locks y, does its work, and completes 4/26/2019 Distributed Systems - Comp 655

2PL across a network - options Centralized 2PL: one machine runs a lock manager for the whole system. Local transaction managers communicate with data managers. Primary 2PL: each resource has a designated primary machine. Each machine has a lock manager. Locks for a resource are granted by its primary machine’s lock manager. Distributed 2PL: schedulers handle communication with data managers. may rely on distributed mutual exclusion 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Another way Timestamp-based schedulers can guarantee serializability without locking Maintain timestamps on Transactions Each data item: timestamp of “youngest” reader Each data item: timestamp of “youngest” writer Serialize committed transactions in timestamp order Abort any transaction that tries to do something that would prevent serialization in timestamp order Pessimistic timestamp ordering: abort at the first bad read or write Optimistic timestamp ordering: check for bad reads/writes at commit time 4/26/2019 Distributed Systems - Comp 655

Locking vs timestamping When concurrency control based on timestamping encounters conflicts, it may abort a transaction. Normally, in a locking scheme, conflicts cause a process to wait (not abort) Timestamping is deadlock-free With locking, deadlocks have to be explicitly avoided or detected and broken Deadlock detection across a network can be prohibitively expensive 4/26/2019 Distributed Systems - Comp 655

Synchronization outline Distributed transactions – part 1 Fowler’s locking patterns Implementing distributed transactions Logical clocks (aka distributed timestamps) Mutual exclusion across a network 4/26/2019 Distributed Systems - Comp 655

Fowler’s offline concurrency patterns Optimistic offline lock What’s the recurring problem it solves? Both pessimistic and optimistic timestamp ordering are applications of this pattern Sequence diagram looks like PTO Text sounds like OTO Unit of work can act as the data manager Simplest implementation uses version numbers instead of timestamps 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Fowler’s locking, part 2 Pessimistic Offline Lock Basically, write your own lock manager Strict 2-phase locking is recommended on p 429, paragraph 5 But, commercial DBMS provide pessimistic lock management Under what conditions would it make sense to use this pattern? 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Fowler’s locking, part 3 Coarse-grained lock One lock, shared by a group of related objects Lock management can be done with any of the techniques we’ve talked about 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Fowler’s locking, part 4 Implicit lock Locks are obtained automatically by the framework Usually, in a Layer supertype Rejected lock requests can throw exceptions Developers don’t have to write code to acquire locks, but they do have to handle concurrency exceptions What’s the recurring problem here? 4/26/2019 Distributed Systems - Comp 655

Fowler’s locking, one last point Many concurrency issues are requirements questions For example, what are the requirements for when an address change should begin affecting tax calculations? (see example, top of p 419) 4/26/2019 Distributed Systems - Comp 655

Synchronization outline Distributed transactions – part 1 Fowler’s locking patterns Implementing distributed transactions Logical clocks (aka distributed timestamps) Mutual exclusion across a network 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Lamport timestamps 1 The happens-before relation: If a and b occur in the same process, and a occurs before b, a  b If a is sending a message, and b is receiving it, a  b (a and b may occur in different processes) Happens-before is transitive: a  b and b  c implies a  c a and b are concurrent if neither a  b nor b  a 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Lamport timestamps 2 In (b), all 8 events get timestamps that are globally agreeable 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Application: pessimistic timestamp ordering for distributed transactions T@50 starts U@51 starts x = 2 R@27 W@32 50 51 52 53 54 55 56 57 58 59 60 61 62 63 45 46 52 53 54 55 56 57 58 59 60 61 62 63 x? 51 52 53 54 55 56 57 58 59 60 61 62 63 64 y = 1 R@41 W@44 x? y? y? x=2 x = 2 R@51 W@32 x = 2 R@51 W@32 x=2 y = 1 R@50 W@44 y=1 y = 1 R@51 W@44 y=1 y:=3 x:=1 x = 1 R@51 W@51 ABORT T@50 because y was read at 51 x 2 2 -1 y 1 3 3 T U x 2 1 1 y 1 1 2 U T T: y = x + y U: x = x - y 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Application: optimistic timestamp ordering for distributed transactions T@50 starts U@51 starts x = 2 W@32 50 51 52 53 54 55 56 57 58 59 60 61 62 63 45 46 52 53 54 55 56 57 58 59 60 61 62 63 x? 51 52 53 54 55 56 57 58 59 60 61 62 63 64 y = 1 W@44 x? y? y? x=2 x = 2 W@32 U read x x = 2 W@32 T read x x=2 y = 1 W@44 T read x,y y=1 y = 1 W@44 U read x,y y=1 y:=3 x:=1 x = 1 W@51 ABORT T@50 because x was changed at 51 x 2 2 -1 y 1 3 3 T U x 2 1 1 y 1 1 2 U T T: y = x + y U: x = x - y 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Lamport timestamps 3 Using a logical clock that adjusts itself to message timestamps, plus tweaks to ensure that no two events will ever get the same timestamp, we can assign a logical clock time C(a) to each event a such that If a happens before b in the same process, C(a) < C(b) If a and b are sending and receiving a given message, C(a) < C(b) If a and b are distinguishable, C(a) ≠ C(b) 4/26/2019 Distributed Systems - Comp 655

Application: totally ordered multicast Assumes reliable transmission: no messages lost, none out of order Sender timestamps and multicasts a message to all processes, including itself Each receiver puts the message in a queue ordered by timestamp, then multicasts a (timestamped) acknowledgement Receivers de-queue and process messages only when they are at the head of the queue and acknowledged by all Result: messages processed in the same order in all processes Cost: n2 + n network messages per logical message 4/26/2019 Distributed Systems - Comp 655

How totally-ordered multicast works Process 1 Process 2 10 Queue: 8 Queue: Queue: 8 10 Queue: 8 10 * * * * msg ack process * 4/26/2019 Distributed Systems - Comp 655

Synchronization outline Distributed transactions – part 1 Fowler’s locking patterns Implementing distributed transactions Logical clocks (aka distributed timestamps) Mutual exclusion across a network 4/26/2019 Distributed Systems - Comp 655

Mutual exclusion across a network Centralized algorithm (uses a co-ordinator) Distributed algorithm Token ring algorithm Comparison 4/26/2019 Distributed Systems - Comp 655

Centralized mutual exclusion NOTE: coordinator will usually be elected. See book, section 6.3 4/26/2019 Distributed Systems - Comp 655

Distributed mutual exclusion Assumes reliable communication, global timestamps Process wanting to enter critical region sends message to all processes If receiver not in critical region and doesn’t want to be, replies OK If receiver in critical region, queues the request If receiver has also requested permission to enter critical region, it compares timestamps on the new request and its own If incoming timestamp smaller, sends OK If own timestamp smaller, queues request 4/26/2019 Distributed Systems - Comp 655

Distributed mutual exclusion 2 Process enters critical region when it has received OK from all other processes When process exits critical region, sends OK to all process with queued requests, and removes those requests from the queue 4/26/2019 Distributed Systems - Comp 655

Distributed mutual exclusion example 4/26/2019 Distributed Systems - Comp 655

Mutual exclusion by token ring 4/26/2019 Distributed Systems - Comp 655

Comparing network mutual exclusion algorithms 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 Bonus material Much of this bonus material was in the first edition of Tanenbaum and van Steen, but dropped for the second edition 4/26/2019 Distributed Systems - Comp 655

Pessimistic Timestamp Ordering - Fundamentals Relies on distributed timestamps Committed transactions can be serialized in timestamp order Timestamps: Transaction T: ts(T) is start time (also written as just T) Data item x: tsRD(x) is timestamp of last transaction to read x (also written as: Rx), tsWR(x) is timestamp of last transaction to write x (also written as: Wx) 4/26/2019 Distributed Systems - Comp 655

Pessimistic Timestamp Ordering - Rules Rules for T attempting to read x: If T < Wx , abort T If T  Wx , allow read, and set Rx = max(T, Rx) If Wx is timestamp of a running transaction, wait and read after it commits or aborts Rules for T attempting to write x: If T < max(Rx, Wx), abort T If T  max(Rx, Wx), allow tentative write, and set Wx = T 4/26/2019 Distributed Systems - Comp 655

Pessimistic Timestamp Ordering – tentative writes If T commits System makes its tentative writes permanent. If T aborts System discards its tentative writes, if any. Client can get a new timestamp and try again. 4/26/2019 Distributed Systems - Comp 655

Distributed Systems - Comp 655 See PTO examples 4/26/2019 Distributed Systems - Comp 655

Optimistic timestamp ordering Don’t worry, be working It actually works most of the time Check for conflicts at commit time, and abort transaction if conflict is detected Works best with a private-workspace approach to atomicity 4/26/2019 Distributed Systems - Comp 655