1. 03/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Recovery

2. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Integrity or correctness of data Would like data to be “accurate” or “correct” at all times EMP Name White Green Gray Age 52 3421 1

3. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Integrity or consistency constraints Predicates data must satisfy Examples: - x is key of relation R - x  y holds in R - Domain(x) = {Red, Blue, Green}  is valid index for attribute x of R - no employee should make more than twice the average salary

4. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Definition: Consistent state: satisfies all constraints Consistent DB: DB in consistent state

5. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Observation: DB cannot be consistent always! Example: a 1 + a 2 +…. a n = TOT ( constraint ) Deposit $100 in a 2 : a 2  a 2 + 100 TOT  TOT + 100

6. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery a 2 TOT.... 50.... 1000.... 150.... 1000.... 150.... 1100 Example: a 1 + a 2 +…. a n = TOT ( constraint ) Deposit $100 in a 2 : a 2  a 2 + 100 TOT  TOT + 100

7. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Transaction: collection of actions that preserve consistency Consistent DBConsistent DB’ T

8. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery How can we prevent/fix violations? cc: due to data sharing only recovery: due to failures only

9. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Will not consider: How to write correct transactions How to write correct DBMS Constraint checking & repair That is, solutions studied here do not need to know constraints

10. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Storage hierarchy Memory Disk x x

11. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Operations: Input (x): block with x  memory Output (x): block with x  disk Read (x,t): do input(x) if necessary t  value of x in block Write (x,t): do input(x) if necessary value of x in block  t

12. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Key problem Unfinished transaction ExampleConstraint: A=B T 1 : A  A  2 B  B  2

13. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery T 1 :Read (A,t); t  t  2 Write (A,t); Read (B,t); t  t  2 Write (B,t); Output (A); Output (B); A: 8 B: 8 A: 8 B: 8 memory disk 16 failure!

14. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery T 1 :Read (A,t); t  t  2 A=B Write (A,t); Read (B,t); t  t  2 Write (B,t); Output (A); Output (B); A:8 B:8 A:8 B:8 memory disk log Undo logging 16 16 16

15. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery One “complication” Log is first written in memory Not written to disk on every action memory DB Log A: 8 16 B: 8 16 Log: A: 8 B: 8 16 BAD STATE # 1

16. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery One “complication” Log is first written in memory Not written to disk on every action memory DB Log A: 8 16 B: 8 16 Log: A: 8 B: 8 16 BAD STATE # 2...

17. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Undo logging rules (1) For every action generate undo log record (containing old value) (2) Before x is modified on disk, log records pertaining to x must be on disk (write ahead logging: WAL) (3) Before commit is flushed to log, all writes of transaction must be reflected on disk

18. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Recovery rules: Undo logging (1) Let S = set of transactions with in log, but no (or ) record in log (2) For each in log, in reverse order (latest  earliest) do: - if Ti  S then - write (X, v) - output (X) (3) For each Ti  S do - write to log

19. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery What if failure during recovery? No problem!  Undo idempotent

20. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery To discuss: Redo logging Undo/redo logging, why both? Checkpoints

21. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Redo logging (deferred modification) T 1: Read(A,t); t t  2; write (A,t); Read(B,t); t t  2; write (B,t); Output(A); Output(B) A: 8 B: 8 A: 8 B: 8 memory DB LOG 16 output 16

22. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Redo logging rules (1) For every action, generate redo log record (containing new value) (2) Before anything is modified on disk (DB), all log records for transaction that modify things (including commit) must be on disk

23. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery (1) Let S = set of transactions with in log (2) For each in log, in forward order (earliest  latest) do: - if Ti  S then Write(X, v) Output(X) Recovery rules: Redo logging

24. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Recovery is very, very SLOW ! Redo log: FirstT1 wrote A,BLast RecordCommitted a year agoRecord (1 year ago)--> STILL, Need to redo after crash!!... Crash

25. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Undo Recovery – Better? The first record without commit or abort should not be too far away from crash But immediate update itself is expensive... first record without commit or abort

26. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Solution: Checkpoint (simple version) Periodically: (1) Do not accept new transactions (2) Wait until all transactions finish (3) Flush all log records to disk (log) (4) Flush all buffers to disk (DB) (do not discard buffers) (5) Write “checkpoint” record on disk (log) (6) Resume transaction processing

27. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Example: what to do at recovery? Redo log (disk): Checkpoint Crash...

28. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Example: what to do at recovery? Undo log (disk): Checkpoint Crash...

29. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Nonquiescent Checkingpointing “Quiescent” checkpointing stalls DB “Nonquiescent” checkpointing admits new transactions while checkpointing

30. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Nonquiescent Checkpoint Rules - undo Write and flush a log record where T1,…,Tk are active transactions When all T1, …, Tk have completed, write and flush a log record

31. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Example: what to do at recovery? Undo log (disk): Crash... Crash after end of checkpoint Only need to undo all the incomplete transactions started after

32. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Example: what to do at recovery? Undo log (disk): Crash... … Crash before end of checkpoint Only need to undo all incomplete transactions started after and those in

33. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Nonquiescent Checkpoint Rules - redo Write and flush a log record <START CKPT(T1, …, Tk) where T1,…,Tk are active transactions Flush all the updates of the transactions committed before Write and flush a log record

34. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Example: what to do at recovery? Redo log (disk): Crash... Crash after end of checkpoint Only need to redo all the transactions committed after the latest

35. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Example: what to do at recovery? Redo log (disk): Crash... … Crash before end of checkpoint Only need to redo all the transactions committed after the latest successful

36. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Key drawbacks: Undo logging: need to update immediately, increasing I/O Redo logging: need to keep all modified blocks in memory until commit

37. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Solution: undo/redo logging! Update 

38. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Rules Page X can be flushed before or after Ti commit Log record flushed before corresponding updated page (WAL) Flush log at commit

39. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Non-quiesce checkpoint L O G for undo Start-ckpt active TR: Ti,T2,... end ckpt... Flush dirty buffers

40. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Examples what to do at recovery time? no T1 commit L O G T 1,- a... Ckpt T 1... Ckpt end... T1-bT1-b  Undo T1 (undo a,b)

41. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Example LOGLOG... T1aT1a T1bT1b T1cT1c T 1 cmt... Ckpt end ckpt-s T 1  Redo T1: (redo b,c)

42. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Examples what to do at recovery time? no T1 commit L O G T 1,- a... Ckpt T 1... T1-bT1-b  Undo T1 (all the actions)

43. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Example LOGLOG... T1aT1a T1bT1b T1cT1c T 1 cmt... ckpt-s T 1  Redo T1: (redo b,c, a and the actions after last successful ckp-s)

44. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Recovery process: Backwards pass (end of log -> latest checkpoint start)  construct set S of committed transactions  undo actions of transactions not in S Undo pending transactions  follow undo chains for transactions in (checkpoint active list) - S Forward pass (latest successful checkpoint start -> end of log)  redo actions of S transactions backward pass forward pass start check- point

45. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Recovery with Checkpoint Summary Logging still obey the logging rules  undo, redo, undo-redo Recovery still obey rules  Undo: undo all incomplete transactions  Redo: redo all completed transactions  Undo-redo: combined Checkpoint provides a way to limit the transactions needing undo or redo

46. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Summary undoredoUndo-redo commit commit means all updates on disk Commit means some updates may start to migrate to disk Commit does not signal disk update recovery Undo all incomplete transactions Redo all committed transactions Undo all incomplete transactions and redo all committed transactions Nonquiescent ckpt means every tran started before the matching is on disk means every tran committed before the matching is on disk means every update before the matching is on disk WAL

47. 13/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Summary Consistency of data One source of problems: failures - WAL