1. DBMS 2001Notes 8: Concurrency1 Principles of Database Management Systems 8: Concurrency Control Pekka Kilpeläinen (after Stanford CS245 slide originals by Hector Garcia-Molina, Jeff Ullman and Jennifer Widom)

2. DBMS 2001Notes 8: Concurrency2 Chapter 9Concurrency Control T1 T2… Tn DB (consistency constraints) How to prevent harmful interference btw transactions? => scheduling techniques based on - locks - timestamps and validation

3. DBMS 2001Notes 8: Concurrency3 Example: T1:Read(A)T2:Read(A) A  A+100A  A  2Write(A)Read(B) B  B+100B  B  2Write(B) Constraint: A=B

4. DBMS 2001Notes 8: Concurrency4 Correctness depends on scheduling of transactions A schedule - Chronological (possibly interleaving) order in which actions of transactions are executed - A correct schedule is equivalent to executing transactions one-at-a-time in some order

5. DBMS 2001Notes 8: Concurrency5 Schedule A T1T2 Read(A); A  A+100; Write(A); Read(B); B  B+100; Write(B); Read(A);A  A  2; Write(A); Read(B);B  B  2; Write(B); AB25 125 250 250 OK

6. DBMS 2001Notes 8: Concurrency6 Schedule B T1T2 Read(A);A  A  2; Write(A); Read(B);B  B  2; Write(B); Read(A); A  A+100; Write(A); Read(B); B  B+100; Write(B); AB25 50 150 150 OK

7. DBMS 2001Notes 8: Concurrency7 Schedule C T1T2 Read(A); A  A+100 Write(A); Read(A);A  A  2; Write(A); Read(B); B  B+100; Write(B); Read(B);B  B  2; Write(B); AB25 125 250 125 250250 OK

8. DBMS 2001Notes 8: Concurrency8 Schedule D T1T2 Read(A); A  A+100; Write(A); Read(A);A  A  2; Write(A); Read(B);B  B  2; Write(B); Read(B); B  B+100; Write(B); AB25 125 250 50 150 250150 Constraint violation!

9. DBMS 2001Notes 8: Concurrency9 Schedule E T1T2’ Read(A); A  A+100; Write(A); Read(A);A  A  1; Write(A); Read(B);B  B  1; Write(B); Read(B); B  B+100; Write(B); AB25 125 25 125125 Same as Schedule D but with new T2’ OK

10. DBMS 2001Notes 8: Concurrency10 Want schedules that are “good”,regardless of –initial state (  “good” in any DB state) and –transaction semantics Only look at order of READs and WRITEs –Note: transactions see values in buffers, not on disk => this time ignore INPUT/OUTPUTs Example: Sa (Schedule a) = r1(A)w1(A)r1(B)w1(B) r2(A)w2(A) r2(B)w2(B) T1 T2

11. DBMS 2001Notes 8: Concurrency11 A schedule is serial, if actions of transactions are not interleaved –e.g., (T1, T2) or (T2, T1) –A serial schedule obviously maintains consistency (assuming correctness of individual transactions) Could we reorder a schedule into an equivalent serial schedule? –Actions conflict, if swapping them may change the meaning of a schedule: any two actions of a single transaction two actions on a common DB element A, one of which is WRITE(A)

12. DBMS 2001Notes 8: Concurrency12 Sc’=r 1 (A)w 1 (A) r 1 (B)w 1 (B)r 2 (A)w 2 (A)r 2 (B)w 2 (B) T 1 T 2 Example: Sc=r 1 (A)w 1 (A)r 2 (A)w 2 (A)r 1 (B)w 1 (B)r 2 (B)w 2 (B) r 1 (B) w 2 (A) r 1 (B)r 2 (A) w 1 (B)w 2 (A) (of swapping non-conflicting actions)

13. DBMS 2001Notes 8: Concurrency13 However, for Sd: Sd=r 1 (A)w 1 (A)r 2 (A)w 2 (A)r 2 (B)w 2 (B)r 1 (B)w 1 (B) Sd cannot be rearranged into a serial schedule Sd is not “equivalent” to any serial schedule Sd is “bad”

14. DBMS 2001Notes 8: Concurrency14 Concepts Transaction: sequence of r i (x), w i (x) actions Conflicting actions: r h (A) w h (A) w h (A) w k (A) r k (A) w k (A) If schedule S contains conflicting actions …, p h (A), …, q k (A),... [i.e., one of p, q is w], transaction T h must precede T k in a corresponding serial schedule. Denote this by T h  T k

15. DBMS 2001Notes 8: Concurrency15 Returning to Sc Sc=r 1 (A)w 1 (A)r 2 (A)w 2 (A)r 1 (B)w 1 (B)r 2 (B)w 2 (B) T 1  T 2 T 1  T 2 No cycles  Sc is “equivalent” to a serial schedule (in this case T 1,T 2 )

16. DBMS 2001Notes 8: Concurrency16 Definition S 1, S 2 are conflict equivalent schedules if S 1 can be transformed into S 2 by a series of swaps on non-conflicting actions. (=> effect of both S1 and S2 on the DB is the same)

17. DBMS 2001Notes 8: Concurrency17 Definition A schedule is conflict serializable if it is conflict equivalent to some serial schedule. NB: Conflict serializability is a sufficient (but not a necessary) condition for serializability (equivalence to some serial schedule) Easier to enforce than serializability, therefore generally assured by commercial systems

18. DBMS 2001Notes 8: Concurrency18 Nodes: transactions T1, T2,... in S Arcs: Ti  Tj for i  j whenever - pi(A), qj(A) are conflicting actions in S, (same element A, at least one of actions is a write) - action pi(A) precedes qj(A) in S Precedence graph P(S) (S is schedule )

19. DBMS 2001Notes 8: Concurrency19 Exercise: What is P(S) for S = w 3 (A) w 2 (C) r 1 (A) w 1 (B) r 1 (C) w 2 (A) r 4 (A) w 4 (D) Is S serializable?

20. DBMS 2001Notes 8: Concurrency20 Lemma Let S 1, S 2 be schedules for the same set of transactions S 1, S 2 conflict equivalent  P(S 1 )=P(S 2 ) Proof: Assume P(S 1 )  P(S 2 )   T i : T i  T j in P(S 1 ) and not in P(S 2 )  S 1 = …p i (A)... q j (A)… p i, q j S 2 = …q j (A)…p i (A)... conflict  S 1, S 2 not conflict equivalent

21. DBMS 2001Notes 8: Concurrency21 Note: P(S 1 )=P(S 2 )  S 1, S 2 conflict equivalent Counter example: S 1 = w 1 (A) r 2 (A) w 2 (B) r 1 (B) S 2 = r 2 (A) w 1 (A) r 1 (B) w 2 (B)

22. DBMS 2001Notes 8: Concurrency22 Theorem I P(S 1 ) acyclic  S 1 conflict serializable (  ) Assume S 1 is conflict serializable   serial S s : S s, S 1 conflict equivalent  P(S s ) = P(S 1 ) [  Lemma]  P(S 1 ) acyclic since P(S s ) is acyclic

23. DBMS 2001Notes 8: Concurrency23 (  ) Assume P(S 1 ) is acyclic Transform S 1 as follows: (1) Take T 1 to be transaction with no incoming arcs (2) Move all T 1 actions to the front S 1 = …… q j (X) …… p 1 (A) … (3) we now have S 1 = (4) repeat above steps to serialize rest! Theorem (cont.) P(S 1 ) acyclic  S 1 conflict serializable T1T1 T3T3 T2T2 T4T4

24. DBMS 2001Notes 8: Concurrency24 How to enforce serializable schedules? Option 1: (Optimistic strategy) Run system, recording P(S); At end of day, check P(S) for cycles, and declare if execution was good

25. DBMS 2001Notes 8: Concurrency25 Option 2: (Pessimistic strategy) Prevent occurrence of cycles in P(S) T 1 T 2 …..T n Scheduler DB How to enforce serializable schedules?

26. DBMS 2001Notes 8: Concurrency26 A locking protocol Two new actions: lock (exclusive):l j (A) unlock:u j (A) scheduler T j T i lock table

27. DBMS 2001Notes 8: Concurrency27 Rule #1: Well-formed transactions T i : … l i (A) … p i (A) … u i (A)... Lock elements (A) before accessing them (pi is a read or a write) Eventually, release the locks (ui(A))

28. DBMS 2001Notes 8: Concurrency28 Rule #2 Legal scheduler S = …….. l i (A) ………... u i (A) ……... no l j (A) for i  j At most one transaction T i can hold a lock on any element A

29. DBMS 2001Notes 8: Concurrency29 What schedules are legal? What transactions are well-formed? S1 = l 1 (A)l 1 (B)r 1 (A)w 1 (B)l 2 (B)u 1 (A)u 1 (B) r 2 (B)w 2 (B)u 2 (B)l 3 (B)r 3 (B)u 3 (B) Exercise: S2 = l 1 (A)r 1 (A)w 1 (B)u 1 (A)u 1 (B) l 2 (B)r 2 (B)w 2 (B)l 3 (B)r 3 (B)u 3 (B) S3 = l 1 (A)r 1 (A)u 1 (A)l 1 (B)w 1 (B)u 1 (B) l 2 (B)r 2 (B)w 2 (B)u 2 (B)l 3 (B)r 3 (B)u 3 (B)

30. DBMS 2001Notes 8: Concurrency30 Schedule F (with simple locking) T1 T2 l 1 (A);Read(A) A:=A+100;Write(A);u 1 (A) l 2 (A);Read(A) A:=Ax2;Write(A); u 2 (A) l 2 (B); Read(B) B:=Bx2;Write(B); u 2 (B) l 1 (B); Read(B) B:=B+100;Write(B); u 1 (B) Constraint violation! A B 25 125 250 50 150 250 150

31. DBMS 2001Notes 8: Concurrency31 Simple-minded locking not sufficient to ensure serializability (i.e., correctness)!  More advanced protocol known as "two phase locking"

32. DBMS 2001Notes 8: Concurrency32 Rule #3 Two phase locking (2PL) for transactions T i = ……. l i (A) ………... u i (A) ……... no unlocks no locks All lock requests of a transaction have to precede its unlock requests

33. DBMS 2001Notes 8: Concurrency33 # locks held by Ti Time Growing Shrinking Phase Phase

34. DBMS 2001Notes 8: Concurrency34 Schedule G (with 2PL) delayed

35. DBMS 2001Notes 8: Concurrency35 Schedule H (T 2 reversed) delayed Neither proceeds: a deadlock –System must rollback (= abort & restart) at least one of T1, T2

36. DBMS 2001Notes 8: Concurrency36 Show that Rules #1,2,3  conflict (2PL) serializable schedule Next step:

37. DBMS 2001Notes 8: Concurrency37 # locks held by Ti To help in proof: Definition Shrink(Ti) = SH(Ti) = first unlock action of Ti T1 T2 Time SH(T2) SH(T1)

38. DBMS 2001Notes 8: Concurrency38 Lemma Let S be a 2PL schedule. Ti  Tj in P(S)  SH(Ti) < S SH(Tj) Proof of lemma: Ti  Tj means that S = … p i (A) … q j (A) …; p,q conflict By rules 1,2: S = … p i (A) … u i (A) … l j (A)... q j (A) … By rule 3: SH(Ti) SH(Tj) So, SH(Ti) < S SH(Tj)

39. DBMS 2001Notes 8: Concurrency39 Proof: Let S be a 2PL schedule. Assume P(S) has cycle T 1  T 2  …. T n  T 1 By Lemma: SH(T 1 ) < SH(T 2 ) <... < SH(T 1 ) Impossible, so P(S) acyclic  S is conflict serializable (by Th. I) Theorem II Rules #1,2,3 conflict (that is, 2PL)  serializable schedule

40. DBMS 2001Notes 8: Concurrency40 Beyond this simple 2PL protocol, it is all a matter of improving performance and allowing more concurrency…. –Shared locks –Multiple granularity –Inserts, deletes and phantoms –Other types of C.C. mechanisms Timestamping Validation

41. DBMS 2001Notes 8: Concurrency41 Shared locks So far only exclusive locks: S =...l 1 (A) r 1 (A) u 1 (A) … l 2 (A) r 2 (A) u 2 (A) … Do not conflict  locking unnecessary Instead, use shared locks (S) for reading: S=... ls1(A) r1(A) ls2(A) r2(A) …. u1(A)u2(A)

42. DBMS 2001Notes 8: Concurrency42 Write actions conflict  use exclusive (X) locks for writing Lock actions: l-m k (A): lock A in mode m (S or X) for T k u k (A): release (whatever) lock(s) held by transaction T k on element A

43. DBMS 2001Notes 8: Concurrency43 Rule #1 Well-formed transactions T i =... l-S 1 (A) … r 1 (A) … u 1 (A) … T i =... l-X 1 (A) … w 1 (A) … u 1 (A) … Request –an S-lock for reading –an X-lock for writing Release the locks eventually

44. DBMS 2001Notes 8: Concurrency44 What about transactions that first read and later write the same element? Option 1: Request exclusive lock T i =...l-X 1 (A) … r 1 (A)... w 1 (A)... u 1 (A) … Option 2: Upgrade (E.g., need to read, but don’t know if will write…) T i =... l-S 1 (A) … r 1 (A)... l-X 1 (A) …w 1 (A)...u(A)… Think as getting 2nd lock on A

45. DBMS 2001Notes 8: Concurrency45 Rule #2 Legal scheduler S =....l-S i (A) … … u i (A) … no l-X j (A) for j  i S =... l-X i (A) … … u i (A) … no l-X j (A) for j  i no l-S j (A) for j  i

46. DBMS 2001Notes 8: Concurrency46 A way to summarize Rule #2 Compatibility matrix: S X S true false Xfalse false Locks already held by some Ti Lock requested by some Tj True OK to give a new lock of requested kind

47. DBMS 2001Notes 8: Concurrency47 Rule # 3 (2PL) Only change to previous: Lock upgrades S(A)  {S(A), X(A)} or S(A)  X(A) are allowed only in the growing phase

48. DBMS 2001Notes 8: Concurrency48 Proof: Similar to the X locks case Theorem Rules 1,2,3  Conf.serializable for S/X locks schedules Lock types beyond S/X Examples: (1) update lock (2) increment lock (see the textbook)

49. DBMS 2001Notes 8: Concurrency49 Update locks

50. DBMS 2001Notes 8: Concurrency50 Solution If T i wants to read A and knows it may later want to write A, it requests an update lock (not shared)

51. DBMS 2001Notes 8: Concurrency51 CompSXU STFT XFFF U FFF New request Lock already held in As before

52. DBMS 2001Notes 8: Concurrency52 Note: object A may be locked in different modes at the same time... S 1 =...l-S 1 (A)…l-S 2 (A)…l-U 3 (A)… To grant a lock in mode m, mode m must be compatible with all currently held locks on object

53. DBMS 2001Notes 8: Concurrency53 How does locking work in practice? Every system is different (E.g., may not even provide CONFLICT-SERIALIZABLE schedules) Here is one (simplified) way...

54. DBMS 2001Notes 8: Concurrency54 (1) Don’t trust transactions to request/release locks (2) Do not release locks until transaction commits/aborts: # locks time Sample Locking System:

55. DBMS 2001Notes 8: Concurrency55 Ti Read(A),Write(B) l(A),Read(A), l(B),Write(B)… Read(A),Write(B) Scheduler, part I Scheduler, part II DB lock table Insert appropriate lock requests Execute or delay, based on existing locks

56. DBMS 2001Notes 8: Concurrency56 Lock table Conceptually A  B C ... Lock info for B Lock info for C If null, object is unlocked Every possible object

57. DBMS 2001Notes 8: Concurrency57 But use hash table: A If object not found in hash table, it is unlocked Lock info for A A... h

58. DBMS 2001Notes 8: Concurrency58 Lock info for A - example tran mode wait? Nxt T_link Element: A Group mode: U Waiting: yes List: T1 S no T2 U no T3X yes  To other lock table entries of transaction T3

59. DBMS 2001Notes 8: Concurrency59 What are the objects we lock? ? Relation A Relation B... Tuple A Tuple B Tuple C... Disk block A Disk block B... DB

60. DBMS 2001Notes 8: Concurrency60 Locking works in any case, but should we choose small or large objects? If we lock large objects (e.g., Relations) –Need few locks –Get low concurrency If we lock small objects (e.g., tuples,fields) –Need more locks (=> overhead higher) –Get more concurrency

61. DBMS 2001Notes 8: Concurrency61 We can have it both ways!! Ask any janitor to give you the solution... hall Stall 1Stall 2Stall 3Stall 4 restroom

62. DBMS 2001Notes 8: Concurrency62 Warning Protocol Hierarchically nesting elements (e.g. relation/block/tuple) can be locked with intention locks IS and IX Idea –start locking at the root (relation) level –to place an S or X lock on a subelement, first place a corresponding intention lock IS or IX the element itself Warns others: "I'll be reading/writing some subelement of this element"

63. DBMS 2001Notes 8: Concurrency63 Example (T 1 reads t 2, T 2 reads R1) R1 t1t1 t2t2 t3t3 t4t4 T 1 (IS) T 1 (S), T 2 (S)

64. DBMS 2001Notes 8: Concurrency64 Example (T 1 reads t 2, T 2 writes to t 4 ) R1 t1t1 t2t2 t3t3 t4t4 T 1 (IS) T 1 (S), T 2 (IX) T 2 (X)

65. DBMS 2001Notes 8: Concurrency65 Compatibility of multiple granularity locks Comp Requestor IS IX S X IS Holder IX S X TTTF FFFF FTFT FFTT As before

66. DBMS 2001Notes 8: Concurrency66 ParentChild can belocked in IS IX S X P C IS, S IS, S, IX, X [S, IS; not necessary] none

67. DBMS 2001Notes 8: Concurrency67 Rules (1) Follow multiple granularity comp function (2) Lock root of tree first, any mode (3) Node Q can be locked by Ti in S or IS only if parent(Q) can be locked by Ti in IX or IS (4) Node Q can be locked by Ti in X,IX only if parent(Q) locked by Ti in IX (5) Ti is two-phase (6) Ti can unlock node Q only if none of Q’s children are locked by Ti

68. DBMS 2001Notes 8: Concurrency68 Exercise: Can T 2 write element f 2.2 ? What locks will T 2 get? R1 t1t1 t2t2 t3t3 t4t4 T 1 (IX) f 2.1 f 2.2 f 3.1 f 3.2 T 1 (IX) T 1 (X)

69. DBMS 2001Notes 8: Concurrency69 Exercise: Can T 2 write element f 2.2 ? What locks will T 2 get? R1 t1t1 t2t2 t3t3 t4t4 T 1 (X) f 2.1 f 2.2 f 3.1 f 3.2 T 1 (IX)

70. DBMS 2001Notes 8: Concurrency70 Exercise: Can T 2 write element f 3.1 ? What locks will T 2 get? R1 t1t1 t2t2 t3t3 t4t4 T 1 (S) f 2.1 f 2.2 f 3.1 f 3.2 T 1 (IS)

71. DBMS 2001Notes 8: Concurrency71 Exercise: Can T 2 read element f 2.2 ? What locks will T 2 get? R1 t1t1 t2t2 t3t3 t4t4 T 1 (IX) f 2.1 f 2.2 f 3.1 f 3.2 T 1 (S,IX) T 1 (X)

72. DBMS 2001Notes 8: Concurrency72 Exercise: Can T 2 write element f 2.2 ? What locks will T 2 get? R1 t1t1 t2t2 t3t3 t4t4 T 1 (IX) f 2.1 f 2.2 f 3.1 f 3.2 T 1 (S,IX) T 1 (X)

73. DBMS 2001Notes 8: Concurrency73 Deletions similar to writes: Get an exclusive lock on A before deleting A Insertions more problematic: –possible to lock only existing elements Phantom tuples: –tuples that should have been locked, but did not exist when the locks were taken

74. DBMS 2001Notes 8: Concurrency74 Phantom tuples Example: relation R (E#,name,…) constraint: E# is key use tuple locking RE#NameFInit …. t155Bush G t275ClintonW

75. DBMS 2001Notes 8: Concurrency75 T 1 : Insert into R T 2 : Insert into R T 1 T 2 l-S 1 (t 1 ) l-S 2 (t 1 ) l- S 1 (t 2 ) l-S 2 (t 2 ) Check Constraint Insert [99,Gore,A,..] Insert [99,Bush,G,..]...

76. DBMS 2001Notes 8: Concurrency76 Solution Use multiple granularity tree Before insert of node Q, lock parent(Q) in X mode R1 t1t1 t2t2 t3t3

77. DBMS 2001Notes 8: Concurrency77 Back to example T 1 : Insert T 2 : Insert T 1 T 2 l-X 1 (R) Check constraint Insert u 1 (R) l-X 2 (R) Check constraint Oops! e# = 99 already in R! delayed

78. DBMS 2001Notes 8: Concurrency78 Validation Lock-based concurrency control is pessimistic: non-serializable schedules are prevented in advance Another, optimistic strategy: –allow transaction Ti access data without locks, but record elements read or written by Ti (in read and write sets RS(Ti) and WS(Ti)) –at the end validate that the actions correspond to some serial schedule

79. DBMS 2001Notes 8: Concurrency79 Validation Transactions have 3 phases: (1) Read –all accessed DB elements read –writes to temporary storage (no locking) (2) Validate –check if schedule so far is serializable; if yes, then... (3) Write –write updated elements to DB

80. DBMS 2001Notes 8: Concurrency80 Key idea Make validation an atomic operation –i.e., validate a single transaction at a time If T 1, T 2, T 3, … is validation order, then resulting schedule will be conflict equivalent to S s = T 1 T 2 T 3...

81. DBMS 2001Notes 8: Concurrency81 To implement validation, system maintains two sets of transactions: FIN = transactions that have finished phase 3 (writing, and are completed) VAL = transactions that have successfully finished phase 2 (validation), but not yet completed

82. DBMS 2001Notes 8: Concurrency82 Example of what validation must prevent: RS(T 2 )={B} RS(T 3 )={A,B} WS(T 2 )={B,D} WS(T 3 )={C} time T 2 start T 2 validated T 3 validating T 3 start     T2 may have written B after T3 read B, contradicting the assumed serial order (T2, T3)  T3 is rolled back

83. DBMS 2001Notes 8: Concurrency83 T 2 finish phase 3 Example of what validation must prevent: RS(T 2 )={B} RS(T 3 )={A, B} WS(T 2 )={B,D} WS(T 3 )={C} time T 2 start T 2 validated T 3 validated T 3 start allow T 3 start    

84. DBMS 2001Notes 8: Concurrency84 Another thing validation must prevent: RS(T 2 )={A} RS(T 3 )={A,B} WS(T 2 )={D,E} WS(T 3 )={C,D} time T 2 validated T 3 validating finish T 2 BAD: w 3 (D) w 2 (D)    

85. DBMS 2001Notes 8: Concurrency85 finish T 2 Another thing validation must prevent: RS(T 2 )={A} RS(T 3 )={A,B} WS(T 2 )={D,E} WS(T 3 )={C, D} time T 2 validated T 3 validated allow finish T 2    

86. DBMS 2001Notes 8: Concurrency86 Validation rules for T j : (1) When Tj starts phase 1 (reading DB): Ignore(Tj)  FIN; // Transactions that // do not affect the validation of Tj (2) At Validation of Tj: if Validates(Tj) then VAL  VAL U {Tj}; do the write phase; FIN  FIN U {Tj}; VAL  VAL - {Tj};

87. DBMS 2001Notes 8: Concurrency87 Validates(T j ): // returns True if T j validates for each U  VAL do if ( WS(U)  RS(Tj)   or WS(U)  WS(Tj)   ) then return False; end for; for each U  FIN - Ignore(Tj) do if ( WS(U)  RS(Tj)   ) then return False; end for; return True;

88. DBMS 2001Notes 8: Concurrency88 Exercise: Validation of U, T, V and W T: RS(T)={A,B} WS(T)={A,C} V: RS(V)={B} WS(V)={D,E} U: RS(U)={B} WS(U)={D} W: RS(W)={A,D} WS(W)={A,C} start validate finish 1. 4. 3. 2.

89. DBMS 2001Notes 8: Concurrency89 Validation is useful in some cases: - If interaction among transactions low  rollbacks rare - If system resources are plentiful - slightly more bookkeeping than for locking - If there are real-time constraints - causes no delays for transactions

90. DBMS 2001Notes 8: Concurrency90 Summary Have studied C.C. mechanisms used in practice - 2 PL - Multiple granularity - Validation