1. 1 Advanced Database Systems: DBS CB, 2 nd Edition Recovery Ch. 17

2. 2 Outline Background Failure Modes and Failure Model Storage Hierarchy Undo Logging Redo Logging Undo/Redo Logging Protecting Against Media Failures

3. 333 Background

4. 4 Integrity or correctness of data  Would like data to be “accurate” or “correct” at all times Integrity or consistency constraints  Predicates data must satisfy any constraints x is key of relation R x  y holds in R Domain(x) = {Red, Blue, Green}  is valid index for attribute x of R no employee should make more than twice the average salary Consistent state: satisfies all constraints Consistent DB: DB in consistent state Background

5. 5 Constraints ( as we use here) may not capture “full correctness” Example 1: Transaction constraints  When salary is updated, new salary > old salary  When account record is deleted, balance = 0  Note: could be “emulated” by simple constraints, e.g., Example 2: a 1 + a 2 +…. a n = TOT ( constraint ) Deposit $100 in a 2 : a2  a2 + 100 TOT  TOT + 100 Background Acct #….balancedeleted? Account

6. 6 Transaction: collection of actions that preserve consistency Background.... 50.... 1000.... 150.... 1000.... 150.... 1100 a2 TOT Constraint violation Consistent DBConsistent DB’ T

7. 7 Assumption: If T starts with consistent state + T executes in isolation  T leaves consistent state Correctness (informally)  If we stop running transactions, DB left consistent  Each transaction sees a consistent DB How can constraints be violated?  Transaction bug  DBMS bug  Hardware failure, e.g., disk crash alters balance of account  Data sharing, e.g.: T1: give 10% raise to programmers and T2: change programmers  systems analysts Background

8. 888 Failure Modes and Failure Model

9. 9 Events for DB needs to recover:  Wrong data entry  Disk failure  Fire / earthquake / ….  Systems crashes  Software errors  Power failures Failure Model: Failure Modes and Failure Model Events Desired Undesired Expected Unexpected

10. 10 Our failure Model: Desired Events: see product manual Undesired expected events: system crash, memory lost, CPU halt, resets, etc. Undesired unexpected events: everything else! such as disk data is lost, etc. Failure Modes and Failure Model CPU M D processor memory disk

11. 11 Is this model reasonable? Approach: Add low level checks + redundancy to increase the probability that the model holds E.g., Replicate disk storage (stable store) Memory parity CPU checks Failure Modes and Failure Model

12. 12 Storage Hierarchy

13. 13 Operations:  Input (x): block containing x  memory  Output (x): block containing x  disk  Read (x,t): do input(x) if necessary t  value of x in block  Write (x,t): do input(x) if necessary value of x in block  t Storage Hierarchy Memory Disk x x

14. 14 Constraint with unfinished transaction Example: Constraint: A=B T1: A  A  2 B  B  2 T1:Read (A,t); t  t  2 Write (A,t); Read (B,t); t  t  2 Write (B,t); Output (A); Output (B); Storage Hierarchy failure! A: 8 B: 8 A: 8 B: 8 memory disk 16

15. 15 Undo Logging

16. 16 Needs atomicity: execute all actions of a transaction or none at all One solution: Undo logging (immediate modification) Undo Logging T1:Read (A,t); t  t  2 A=B Write (A,t); Read (B,t); t  t  2 Write (B,t); Output (A); Output (B); memorydisk A:8 B:8 A:8 B:8 log 16 16 16

17. 17 Undo Logging – One “complication”  Log is first written in memory  Not written to disk on every action Undo Logging A: 8 16 B: 8 16 Log: A: 8 B: 8 16 BAD STATE # 1 (if memory is lost) memory DB Log

18. 18 Undo Logging – Write the commit record on the Log before commit Undo Logging A: 8 16 B: 8 16 Log: A: 8 B: 8 16 BAD STATE # 2 (tx committed but DB is not accurate)... memory DB Log

19. 19 Undo Logging rules For every action generate undo log record (containing old value) Before x is modified on disk, log records pertaining to x must be on disk (Write Ahead Logging: WAL) Before commit is flushed to log, all writes of transaction must be reflected on disk Undo Logging

20. 20 Undo Logging – Recovery rules For every Ti with in log:  If or in log, do nothing  Else For all in log: write (X, v) output (X ) Write to log Undo Logging  IS THIS CORRECT??

21. 21 Undo Logging – Recovery rules Let S = set of transactions with in log, but no (or ) record in log For each in log, in reverse order (latest  earliest) do:  if Ti  S then - write (X, v) - output (X) For each Ti  S do - write to log Undo Logging

22. 22 Undo Logging – What if failure happens during recovery?  No problem!  Undo is idempotent Undo Logging

23. 23 Undo Logging – Checkpoint  In principal, once tx has its COMMIT log record written to disk, the undo log records of that tx are not needed during recovery  However, other txs executing in parallel at that time; we can’t ignore log records prior to a COMMIT  solution is a checkpoint: Stop accepting new transactions Wait till all current active txs commit or abort and have written a COMMIT or ABORT record in the log Flush the log to disk, and flush all DB buffers to disk as well Write a log record, and flush the log again Resume accepting new transactions Undo Logging

24. 24 Undo Logging – Nonquiescent Checkpoint  A problem with regular checkpoint is it effectively halts the system from processing new transaction for a while!  A more advanced approach is Nonquiescent checkpoint: Write a log record, and flush to the log. The (T 1, …,T k ) transactions are the current active transactions that did not commit/abort yet Wait till (T 1, …,T k ) commit or abort, but do not prohibit new transactions from starting When all (T 1, …,T k ) have completed, write a log record and flush the log Undo Logging

25. 25 Redo Logging

26. Redo Logging (deferred modification) T1: Read(A,t); t t  2; write (A,t); Read(B,t); t t  2; write (B,t); Output(A); Output(B) LOG 26 Redo Logging A: 8 B: 8 A: 8 B: 8 memory DB output 16

27. 27 Redo Logging Rules 1. For every action, generate redo log record (containing new value) 2. Before X is modified on disk (DB), all log records for transaction that modified X (including commit) must be on disk 3. Flush log at commit 4. Write END record after DB updates are flushed to disk Redo Logging

28. 28 Redo Logging – Recovery Rules For every Ti with in log:  For all in log: Write(X, v) Output(X) Redo Logging  IS THIS CORRECT??

29. 29 Redo Logging – Recovery Rules 1. Let S = set of transactions with (and no ) in log 2. For each in log, in forward order (earliest  latest) do:  if Ti  S then Write(X, v) Output(X) 3. For each Ti  S, write Redo Logging

30. 30 Redo Logging - Combining Records Want to delay DB flushes for hot objects Redo Logging Say X is branch balance: T1:... update X... T2:... update X... T3:... update X... T4:... update X... Actions: write X output X write X output X write X output X write X output X combined (checkpoint)

31. 31 Redo Logging - Checkpoint  Unique problem is DB changes made to a committed tx can be copied to disk much later than the commit time  We can’t limit our concerns to active transactions when we decide to create a CKPT (regardless if it is quiescent or nonquiescent);  Between start and end CKPT we need to write to disk all DB elements (dirty buffers) that has been modified by committed transactions  On the other hand, we can complete the CKPT w/o waiting for the active txs to commit or abort Redo Logging

32. 32 Redo Logging – Nonquiescent Checkpoint  Write a log record, and flush to the log. The (T 1, …,T k ) transactions are the current active transactions that did not commit/abort yet  Write to disk all DB elements that were written to buffers (dirty buffers) that are not written yet to disk by transactions that had already committed when the record was written to the log  Write an record and flush to the log Redo Logging

33. 33 Undo/Redo Logging

34. 34 Undo/Redo Logging – Key drawbacks:  Undo logging: cannot bring backup DB copies up to date  Redo logging: need to keep all modified blocks in memory until commit Solution: undo/redo logging! Update  page X Undo/Redo Rules:  Page X can be flushed before or after T i commit  Log record flushed before corresponding updated page (WAL)  Flush at commit (log only) Undo/Redo Logging

35. 35 Undo/Redo Logging – Nonquiescent checkpoint Write a log record, and flush to the log. The (T 1, …,T k ) transactions are the current active transactions that did not commit/abort yet, and flush the log Write to disk all the DB dirty buffers; unlike redo logging, we flush all dirty buffers and not only those written by committed transactions Write an record and flush to the log Undo/Redo Logging Start-ckpt active TR: Ti,T2,... end ckpt... Dirty buffers Pool pages flushed for Undo LOGLOG

36. 36 Undo/Redo Logging – Nonquiescent checkpoint Undo/Redo Logging LOGLOG  Undo T1: (undo a,b) No T1 commit... T1 a... T1 b... T1 c... T1 cmt... ckpt- end ckpt-s T1 LOGLOG  Redo T1: (redo b,c) T1 commit T1,- a... Ckpt T1... Ckpt end... T1- b...

37. 37 Undo/Redo Logging – Recovery Process: Backwards pass (end of log  latest valid checkpoint start)  construct set S of committed transactions  undo actions of transactions not in S Undo pending transactions  follow undo chains for transactions in (checkpoint active list) - S Forward pass (latest checkpoint start  end of log)  redo actions of S transactions Undo/Redo Logging backward pass forward pass start check- point

38. 38 Protecting Against Media Failures

39. 39 Media failure (loss of non-volatile storage) Solution: Make copies of data! Protecting Against Media failures A: 16

40. 40 Example 1: Triple modular redundancy  Keep 3 copies on separate disks  Output(X) --> three outputs  Input(X) --> three inputs + vote Example 2: Redundant writes, Single reads  Keep N copies on separate disks  Output(X) --> N outputs  Input(X) --> Input one copy - if ok, done - else try another one  Assumes bad data can be detected Protecting Against Media failures

41. 41 Example 3: DB Dump + Log If active database is lost,  restore active database from backup  bring up-to-date using redo entries in log Protecting Against Media failures backup database active database log

42. 42 When can Log be discarded Protecting Against Media failures check- point db dump last needed undo not needed for media recovery not needed for undo after system failure not needed for redo after system failure log time

43. 43 END