1. File Processing : Transaction Management 2015, Spring Pusan National University Ki-Joune Li

2. STEMPNU Basic Concepts of Transaction Transaction  A set of operations  Atomic : All or Nothing : Consistent State of Database Example : Flight Reservation Cf. Partially Done : Inconsistent State

3. STEMPNU Transaction States Active Partially Committed FailedAborted Committed the initial state; the transaction stays in this state while it is executing after the discovery that normal execution can no longer proceed. after the transaction has been rolled back and the database restored to its state prior to the start of the transaction. - restart the transaction or - kill the transaction ALL NOTHING

4. STEMPNU ACID Properties Atomicity.  All or Nothing  Not Partially Done  Example : Failure in Flight Reservation Consistency.  Execution of a transaction preserves the consistency of the database. State 1 State 2 All Nothing State 2’ Partially Done Consistent

5. STEMPNU ACID Properties Isolation.  Although multiple transactions may execute concurrently, each transaction must be unaware of other concurrently executing transactions.  Intermediate transaction results must be hidden from other concurrently executed transactions. Durability.  After a transaction completes successfully, the changes it has made to the database persist, even if there are system failures. DB Transation 1 Transation 2 No Effect

6. STEMPNU Example Transaction : Transfer $50 from account A to account B: 1.read(A) 2.A := A – 50 3.write(A) 4.read(B) 5.B := B + 50 6.write(B) Consistency requirement  the sum of A and B is unchanged after the transaction. Atomicity requirement Durability Isolation

7. STEMPNU Example : Concurrent Execution Two Transactions  T 1 : transfer $50 from A to B,  T 2 transfer 10% of the balance from A to B Serial ScheduleConcurrent Schedule

8. STEMPNU Serializability What happens after these transactions ?  Serial Schedule : Always Correct T1  T2 and T2  T1  Concurrent Schedule  Serializable if Result (T1 || T2) = Result(T1  T2) or Result(T2  T1)

9. STEMPNU Transaction Management  Guarantee ACID Properties of Transaction by Concurrency Control : Isolation and Consistency Recovery : Atomicity and Durability

10. File Processing : Concurrency Control

11. STEMPNU Serializability For given transactions T 1, T 2,.., T n,  Schedule (History) S is serializable if Result(S)  Result(S k ) where S k is a serial excution schedule.  Note that Result(S i ) may be different from Result(S j ) (i  j ) How to detect whether S is serializable  Conflict Graph

12. STEMPNU Conflict Graph T1T1 T2T2 r(a)r(a) w(a)w(a) affects r(b)r(b) w(b)w(b) S1S1 S1S1 T1T1 T2T2 r(a)r(a) w(a)w(a) r(b)r(b) w(b)w(b) S2S2 S2S2 Res(S 1 )  Res( (T 1, T 2 ) ) Res(S 1 )  Res( (T 1, T 2 ) ) Res(S 1 )  Res( (T 2, T 1 ) )

13. STEMPNU Detect Cycle in Conflict Graph T1T1 T2T2 r(a)r(a) w(a)w(a) affects r(b)r(b) w(b)w(b) T2T2 T2T2 T1T1 T1T1 If  Cycle in Conflict Graph  Then Not Serializable  Otherwise Serializable

14. STEMPNU How to make it serializable  Control the order of execution of operations in concurrent transactions. Two Approaches  Two Phase Locking Protocol Locking on each operation  Timestamping : Ordering by timestamp on each transaction and each operation

15. STEMPNU Lock-Based Protocols A lock  mechanism to control concurrent access to a data item Data items can be locked in two modes :  Exclusive (X) mode : Data item can be both read as well as written. X-lock is requested using lock-X instruction.  Shared (S) mode : Data item can only be read. S-lock is requested using lock-S instruction. Lock requests are made to concurrency-control manager. Transaction can proceed only after request is granted.

16. STEMPNU Lock-Based Protocols (Cont.) Lock-compatibility matrix A transaction may be granted a lock on an item  if the requested lock is compatible with locks already held  Any number of transactions can hold shared locks If a lock cannot be granted,  the requesting transaction is made to wait  till all incompatible locks held have been released.  the lock is then granted.

17. STEMPNU Lock-Based Protocols Example of a transaction performing locking: T 2 : lock-S(A); read (A); unlock(A); lock-S(B); read (B); unlock(B);

18. STEMPNU The Two-Phase Locking Protocol This is a protocol which ensures conflict-serializable schedules.  Phase 1: Growing Phase transaction may obtain locks transaction may not release locks  Phase 2: Shrinking Phase transaction may release locks transaction may not obtain locks The protocol assures serializability

19. STEMPNU Lock Conversions Two-phase locking with lock conversions:  First Phase: can acquire a lock-S on item can acquire a lock-X on item can convert a lock-S to a lock-X (upgrade)  Second Phase: can release a lock-S can release a lock-X can convert a lock-X to a lock-S (downgrade) This protocol assures serializability

20. STEMPNU Where to insert Lock related operations ? A transaction T i  issues the standard read/write instruction,  without explicit locking calls. Automatic Acquisition of Locks  The operation read(D) is processed as: if T i has a lock on D then read(D) else begin wait until no other transaction has a lock-X on D grant T i a lock-S on D; read(D) end

21. STEMPNU Automatic Acquisition of Locks write(D) is processed as: if T i has a lock-X on D then write(D) else begin if necessary wait until no other trans. has any lock on D, if T i has a lock-S on D then upgrade lock on D to lock-X else grant T i a lock-X on D write(D) end; All locks are released after commit or abort

22. STEMPNU Lock Manager Transaction  Transaction sends lock request to Lock Manager  Lock Manager determines whether to allow or not Lock Table  Keeps granted locks and pending locks for each item  In-memory Hash Table

23. STEMPNU Problem of Two Phase Locking Protocol Deadlock  Growing Phase and Shrinking Phase Prevention and Avoidance : Impossible Only Detection may be possible  When a deadlock occurs Detection of Deadlock : Wait-For-Graph Abort a transaction  How to choose a transaction to kill ?

24. STEMPNU Tree Lock : A Special Locking Mechanism Only exclusive locks are allowed.  The first lock by T may be on any data item.  Subsequently, a data Q can be locked by T only if the parent of Q is currently locked by T. Data items may be unlocked at any time. Pairwise lock Not Allowed

25. STEMPNU Timestamp-Based Protocols Each transaction is issued a timestamp when  TS(T i ) <TS(T j ) : old transaction T i and new transaction T j Each data Q, two timestamp :  W-timestamp(Q) : largest time-stamp for successful write(Q)  R-timestamp(Q) : largest time-stamp for successful read(Q)

26. STEMPNU Timestamp-Based Protocols : Read Transaction T i issues a read(Q)  If TS(T i )  W-timestamp(Q), then T i needs to read a value of Q that was already overwritten. Hence, the read operation is rejected, and T i is rolled back.  If TS(T i )  W-timestamp(Q), then the read operation is executed, and R-timestamp(Q) is set set

27. STEMPNU Timestamp-Based Protocols : Write Transaction T i issues write(Q).  If TS(T i ) < R-timestamp(Q), then the value of Q that T i is producing was needed previously, and the system assumed that that value would never be produced. Hence, the write operation is rejected, and T i is rolled back.  If TS(T i ) < W-timestamp(Q), then T i is attempting to write an obsolete value of Q. Hence, this write operation is rejected, and T i is rolled back.  Otherwise, the write operation is executed, and W-timestamp(Q) is reset

28. STEMPNU The timestamp-ordering protocol guarantees serializability since all the arcs in the precedence graph are of the form: Thus, there will be no cycles in the conflict graph Timestamp protocol : free from deadlock transaction with smaller timestamp transaction with larger timestamp Correctness of Timestamp-Ordering Protocol

29. STEMPNU Problem of Time-Stamping Protocol Rollback and Restarting Overhead  Rollback and Restarting relatively more frequent than deadlock Instead of Deadlock Detection  Cost ? p RB (C RB +C RStart ) > p DL C DL + C DLDetect

30. STEMPNU Long Duration Transaction Transaction  of Long Duration  with large number of operations Problem  Expensive rollback and restart  Degradation of concurrency Approach  Nested Transaction  Semantic Consistency rather than Serializability

31. File Processing : Recovery

32. STEMPNU Failure Classification Transaction failure :  Logical errors: internal error condition  System errors: system error condition (e.g., deadlock) System crash:  a power failure or other hardware or software failure  Fail-stop assumption: non-volatile storage contents are assumed to not be corrupted by system crash Database systems have numerous integrity checks to prevent corruption of disk data Disk failure

33. STEMPNU Recovery Algorithms Recovery algorithms : should ensure  database consistency  transaction atomicity and  durability despite failures Recovery algorithms have two parts 1.Preparing Information for Recovery : During normal transaction 2.Actions taken after a failure to recover the database

34. STEMPNU Storage Structure Volatile storage:  does not survive system crashes  examples: main memory, cache memory Nonvolatile storage:  survives system crashes  examples: disk, tape, flash memory, non-volatile (battery backed up) RAM Stable storage:  a mythical form of storage that survives all failures  approximated by maintaining multiple copies on distinct nonvolatile media

35. STEMPNU Recovery and Atomicity Modifying the database  must be committed  Otherwise it may leave the database in an inconsistent state. Example  Consider transaction T i that transfers $50 from account A to account B; goal is either to perform all database modifications made by T i or none at all. Several output operations may be required for T i  For example : output(A) and output(B).  A failure may occur after one of these modifications have been made but before all of them are made.

36. STEMPNU Recovery and Atomicity (Cont.) To ensure atomicity despite failures,  we first output information describing the modifications to stable storage without modifying the database itself. We study two approaches:  log-based recovery, and  shadow-paging

37. STEMPNU Log-Based Recovery A log : must be kept on stable storage. , and  Logging Method  When transaction T i starts, log record  When T i finishes, log record  Before T i executes write(X), log record  We assume for now that log records are written directly to stable storage Two approaches using logs  Deferred database modification  Immediate database modification

38. STEMPNU Deferred Database Modification The deferred database modification scheme  records all modifications to the log, writes them after commit. Log Scheme  Transaction starts by writing record to log.  write(X) : Note: old value is not needed for this scheme The write is not performed on X at this time, but is deferred.  When T i commits, is written to the log  Finally, executes the previously deferred writes.

39. STEMPNU Deferred Database Modification (Cont.) Recovering Method  During recovery after a crash, a transaction needs to be redone if and only if both and are there in the log. Redoing a transaction T i ( redoT i ) sets the value of all data items updated by the transaction to the new values.  Deletes T i such that exists but does not.

40. STEMPNU Deferred Database Modification : Example If log on stable storage at time of crash is as in case: (a) No redo actions need to be taken (b) redo(T 0 ) must be performed since is present (c) redo(T 0 ) must be performed followed by redo(T 1 ) since and are present T0: read (A)T1 : read (C) A: = A - 50 C:= C- 100 write (A) write (C) read (B) B:= B + 50 write (B)

41. STEMPNU Immediate Database Modification Immediate database modification scheme  Database updates of an uncommitted transaction  For undoing : both old value and new value Recovery procedure has two operations  undo(T i ) : restores the value of all data items updated by T i  redo(T i ) : sets the value of all data items updated by T i When recovering after failure:  Undo if the log, but not.  Redo if the log both the record and.

42. STEMPNU Immediate Database Modification : Example Recovery actions in each case above are: (a) undo (T 0 ): B is restored to 2000 and A to 1000. (b) undo (T 1 ) and redo (T 0 ): C is restored to 700, and then A and B are set to 950 and 2050 respectively. (c) redo (T 0 ) and redo (T 1 ): A and B are set to 950 and 2050 respectively. Then C is set to 600

43. STEMPNU Idempotent Operation Result (Op(x)) = Result (Op(Op(x))  Example Increment(x); : Not Idempotent x=a; write(x); : Idempotent Operations in Log Record  Must be Idempotent, otherwise Multple Executions (for redo) may cause incorrect results

44. STEMPNU Checkpoints Problems in recovery procedure by Log record scheme : 1.searching the entire log records : time-consuming 2.Discard unnecessary redo transactions already executed. Checkpoint Method  Marking Checkpoint  Recovery from Checkpoint Marking Checkpoint 1.Output all log records currently residing in main memory onto stable storage. 2.Output all modified buffer blocks to the disk. 3.Write a log record onto stable storage.

45. STEMPNU Checkpoints (Cont.) In case of failure  T 1 can be ignored  T 2 and T 3 redone.  T 4 undone TcTc TfTf T1T1 T2T2 T3T3 T4T4 checkpoint system failure

46. STEMPNU Shadow Paging Mechanism  maintain two page tables the current page table, and the shadow page table  shadow page table state of the database before transaction execution Shadow page table is never modified during execution To start with, both the page tables are identical.  Whenever any page is about to be written for the first time A copy of this page is made onto an unused page. The current page table is then made to point to the copy The update is performed on the copy

47. STEMPNU Shadow Page : Example Old State New State