CPSC-608 Database Systems

Slides:



Advertisements
Similar presentations
ICS 214A: Database Management Systems Fall 2002
Advertisements

Crash Recovery John Ortiz. Lecture 22Crash Recovery2 Review: The ACID properties  Atomicity: All actions in the transaction happen, or none happens 
IDA / ADIT Lecture 10: Database recovery Jose M. Peña
1 CPS216: Data-intensive Computing Systems Failure Recovery Shivnath Babu.
Transactions and Recovery Checkpointing Souhad Daraghma.
Crash Recovery, Part 1 If you are going to be in the logging business, one of the things that you have to do is to learn about heavy equipment. Robert.
1 Crash Recovery Chapter Review: The ACID properties  A  A tomicity: All actions of the Xact happen, or none happen.  C  C onsistency: If each.
Recovery CPSC 356 Database Ellen Walker Hiram College (Includes figures from Database Systems by Connolly & Begg, © Addison Wesley 2002)
1 Failure Recovery Checkpointing Undo/Redo Logging Source: slides by Hector Garcia-Molina.
Transactions A process that reads or modifies the DB is called a transaction. It is a unit of execution of database operations. Basic JDBC transaction.
Recovery from Crashes. Transactions A process that reads or modifies the DB is called a transaction. It is a unit of execution of database operations.
Recovery from Crashes. ACID A transaction is atomic -- all or none property. If it executes partly, an invalid state is likely to result. A transaction,
1 ICS 214A: Database Management Systems Fall 2002 Lecture 16: Crash Recovery Professor Chen Li.
ACID A transaction is atomic -- all or none property. If it executes partly, an invalid state is likely to result. A transaction, may change the DB from.
CPSC-608 Database Systems Fall 2011 Instructor: Jianer Chen Office: HRBB 315C Phone: Notes #6.
1 Transaction Management Overview Yanlei Diao UMass Amherst March 15, 2007 Slides Courtesy of R. Ramakrishnan and J. Gehrke.
1 Anna Östlin Pagh and Rasmus Pagh IT University of Copenhagen Advanced Database Technology March 25, 2004 SYSTEM FAILURES Lecture based on [GUW ,
CPSC-608 Database Systems Fall 2011 Instructor: Jianer Chen Office: HRBB 315C Phone: Notes #11.
CPSC-608 Database Systems Fall 2011 Instructor: Jianer Chen Office: HRBB 315C Phone: Notes #13.
CPSC-608 Database Systems Fall 2010 Instructor: Jianer Chen Office: HRBB 315C Phone: Notes #5.
Cs4432recovery1 CS4432: Database Systems II Lecture #20 Failure Recovery Professor Elke A. Rundensteiner.
CPSC-608 Database Systems Fall 2011 Instructor: Jianer Chen Office: HRBB 315C Phone: Notes #9.
CPSC-608 Database Systems Fall 2011 Instructor: Jianer Chen Office: HRBB 315C Phone: Notes #12.
CPSC-608 Database Systems Fall 2011 Instructor: Jianer Chen Office: HRBB 315C Phone: Notes #14.
CPSC-608 Database Systems Fall 2010 Instructor: Jianer Chen Office: HRBB 315C Phone: Notes 1.
CPSC-608 Database Systems Fall 2010 Instructor: Jianer Chen Office: HRBB 315C Phone: Notes #6.
CPSC-608 Database Systems Fall 2011 Instructor: Jianer Chen Office: HRBB 315C Phone: Notes 1.
July 16, 2015ICS 5411 Coping With System Failure Chapter 17 of GUW.
1 Recovery Control (Chapter 17) Redo Logging CS4432: Database Systems II.
Academic Year 2014 Spring. MODULE CC3005NI: Advanced Database Systems “DATABASE RECOVERY” (PART – 1) Academic Year 2014 Spring.
CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 294 Database Systems II Coping With System Failures.
Database Systems/COMP4910/Spring05/Melikyan1 Transaction Management Overview Unit 2 Chapter 16.
Chapter 10 Recovery System. ACID Properties  Atomicity. Either all operations of the transaction are properly reflected in the database or none are.
CPSC-608 Database Systems Fall 2015 Instructor: Jianer Chen Office: HRBB 315C Phone: Notes #5.
1 Ullman et al. : Database System Principles Notes 08: Failure Recovery.
03/30/2005Yan Huang - CSCI5330 Database Implementation – Recovery Recovery.
CS422 Principles of Database Systems Failure Recovery Chengyu Sun California State University, Los Angeles.
1 Advanced Database Systems: DBS CB, 2 nd Edition Recovery Ch. 17.
Scholastic Dishonesty
DURABILITY OF TRANSACTIONS AND CRASH RECOVERY
CS422 Principles of Database Systems Failure Recovery
Recovery Control (Chapter 17)
Lecture 13: Recovery Wednesday, February 2, 2005.
Advanced Database Systems: DBS CB, 2nd Edition
Recovery 6/4/2018.
Database Applications (15-415) DBMS Internals- Part XIII Lecture 22, November 15, 2016 Mohammad Hammoud.
CS4432: Database Systems II
Chapter 10 Recover System
Database System Principles Notes 08: Failure Recovery
CPSC-608 Database Systems
CS 245: Database System Principles Notes 08: Failure Recovery
CPSC-608 Database Systems
Database Applications (15-415) DBMS Internals- Part XIII Lecture 25, April 15, 2018 Mohammad Hammoud.
CS 245: Database System Principles Notes 08: Failure Recovery
Introduction to Database Systems CSE 444 Lectures 15-16: Recovery
Database Recovery 1 Purpose of Database Recovery
CPSC-608 Database Systems
CPSC-608 Database Systems
Scholastic Dishonesty
CPSC-608 Database Systems
CPSC-608 Database Systems
CPSC-608 Database Systems
CPSC-608 Database Systems
CPSC-608 Database Systems
Database Applications (15-415) DBMS Internals- Part XIII Lecture 24, April 14, 2016 Mohammad Hammoud.
CPSC-608 Database Systems
Data-intensive Computing Systems Failure Recovery
CPSC-608 Database Systems
Lecture 17: Data Storage and Recovery
Lecture 16: Recovery Friday, November 4, 2005.
Presentation transcript:

CPSC-608 Database Systems Fall 2018 Instructor: Jianer Chen Office: HRBB 315C Phone: 845-4259 Email: chen@cse.tamu.edu Notes #35

Graduate Database …… DBMS User-1 User-2 User-3 User-n lock table DDL complier lock table file manager logging & recovery concurrency control User-1 DDL User-2 transaction manager index/file manager buffer manager ……,T2,T1 User-3 query execution engine …… DML complier DML …… Q2,Q1 main memory buffers User-n secondary storage (disks) DBMS Graduate Database

Tranactions What is a transaction? What is a logic step A sequence of actions that we want to be done as a single “logic step ” What is a logic step (what do we want a transaction to be)? ACID: Atomicity: either all steps of a transaction happen, or none happen Consistency: a transaction transforms a consistent DB into a consistent DB Isolation: execution of a transaction is isolated from that of other transactions Durability: if a transaction commits, its effects persist.

Coping with System Failures Therefore, the logging strategy must be carefully designed to ensure Atomicity and Durability. If you are going to be in the logging business, one of the things that you have to do is to learn about heavy equipment. Robert VanNatta, Logging History of Columbia County We will study three logging strategies: Undo-logging Redo-logging Undo/Redo-logging

Coping with System Failures Therefore, the logging strategy must be carefully designed to ensure Atomicity and Durability. If you are going to be in the logging business, one of the things that you have to do is to learn about heavy equipment. Robert VanNatta, Logging History of Columbia County We will study three logging strategies: Undo-logging Redo-logging Undo/Redo-logging

Redo-Logging Algorithms for Redo-logging Log recording Main idea of Redo-logging: If <T, commit> is not in disk, then no changes made by T have reached disk. Algorithms for Redo-logging Log recording 1. Add <T, start> to memory-log; 2. When T writes on A, add <T, A, anew> in memory-log (no output(A) is allowed yet); 3. Add <T, commit> to memory-log; 4. Flush memory-log to disk; 5. Perform output(A) for disk elements A that were written by T. Recovery with Redo-log 1. Bring disk-log to memory; 2. Scan the log forwards; 3. If <T, commit> is not in log Then Ignore T; \\but write <T, abort> to disk-log 4. For T with <T, commit> in log For each <T, A, anew> in log Redo by assigning anew to A;

Checkpoint for Redo-logging …… <T1,start> … <T2,commit> <T3,start> … <T4,commit> 7

Checkpoint for Redo-logging Place a log record <Start CKPT(T1, …, Tk)>, where T1, …, Tk are the currently active transactions; …… <T1,start> … <T2,commit> <T3,start> … <T4,commit> <Start CKPT (T1,T3)> 8

Copy changes by committed Xacts to disk Checkpoint for Redo-logging log Place a log record <Start CKPT(T1, …, Tk)>, where T1, …, Tk are the currently active transactions; Copy all elements in main memory that were written by committed transactions to disk; …… <T1,start> … <T2,commit> <T3,start> … <T4,commit> <Start CKPT (T1,T3)> Copy changes by committed Xacts to disk … 9

Checkpoint for Redo-logging Place a log record <Start CKPT(T1, …, Tk)>, where T1, …, Tk are the currently active transactions; Copy all elements in main memory that were written by committed transactions to disk; \\ at the meanwhile, the system can continuously \\ accept new transactions (non-quiescent) …… <T1,start> … <T2,commit> <T3,start> … <T4,commit> <Start CKPT (T1,T3)> Copy changes by committed Xacts to disk … <T5,start> Copy changes by committed Xacts to disk 10

Checkpoint for Redo-logging Place a log record <Start CKPT(T1, …, Tk)>, where T1, …, Tk are the currently active transactions; Copy all elements in main memory that were written by committed transactions to disk; \\ at the meanwhile, the system can continuously \\ accept new transactions (non-quiescent) When step 2 is done, place a log record <End CKPT>. …… <T1,start> … <T2,commit> <T3,start> … <T4,commit> <Start CKPT (T1,T3)> Copy changes by committed Xacts to disk … <T5,start> Copy changes by committed Xacts to disk <End CKPT> 11

Checkpoint for Redo-logging Place a log record <Start CKPT(T1, …, Tk)>, where T1, …, Tk are the currently active transactions; Copy all elements in main memory that were written by committed transactions to disk; \\ at the meanwhile, the system can continuously \\ accept new transactions (non-quiescent) When step 2 is done, place a log record <End CKPT>. \\ at this point, all log records before the earliest \\ <Ti, start> among (T1, …, Tk) can be ignored <End CKPT> …… <T1,start> … <T2,commit> <T3,start> … <T4,commit> <Start CKPT (T1,T3)> Copy changes by committed Xacts to disk … <T5,start> Copy changes by committed Xacts to disk <End CKPT> 12

Checkpoint for Redo-logging Place a log record <Start CKPT(T1, …, Tk)>, where T1, …, Tk are the currently active transactions; Copy all elements in main memory that were written by committed transactions to disk; \\ at the meanwhile, the system can continuously \\ accept new transactions (non-quiescent) When step 2 is done, place a log record <End CKPT>. \\ at this point, all log records before the earliest \\ <Ti, start> among (T1, …, Tk) can be ignored \\ if crash occurs between <Start CKPT(T1, …, Tk)> \\ and <End CKPT>, then look for a previous \\ <End CKPT> <End CKPT> …… <T1,start> … <T2,commit> <T3,start> … <T4,commit> <Start CKPT (T1,T3)> Copy changes by committed Xacts to disk … <T5,start> Copy changes by committed Xacts to disk <End CKPT> 13

Coping with System Failures Therefore, the logging strategy must be carefully designed to ensure Atomicity and Durability. If you are going to be in the logging business, one of the things that you have to do is to learn about heavy equipment. Robert VanNatta, Logging History of Columbia County We will study three logging strategies: Undo-logging Redo-logging Undo/Redo-logging

Undo/Redo Logging The format of the Undo/Redo log: <T, start> <T, A,aold,anew> …… <T, B,bold,bnew> <T, commit> (or <T, abort>) transaction T starts transaction T wrote on A/B, both old and new values are recorded transaction T commits/abort

Undo/Redo Logging The format of the Undo/Redo log: <T, start> <T, A,aold,anew> …… <T, B,bold,bnew> <T, commit> (or <T, abort>) transaction T starts transaction T wrote on A/B, both old and new values are recorded transaction T commits/abort Main idea of Undo/Redo logging: Increase flexibility and compromise undo logging and redo logging (at the expense of using more space).

Undo/Redo Logging The format of the Undo/Redo log: <T, start> <T, A,aold,anew> …… <T, B,bold,bnew> <T, commit> (or <T, abort>) transaction T starts Undo: helps guarantee Atomicity, but transactions commit later, more flush of log files Redo: helps guarantee Durability, but requires more main memory space transaction T wrote on A/B, both old and new values are recorded transaction T commits/abort Main idea of Undo/Redo logging: Increase flexibility and compromise undo logging and redo logging (at the expense of using more space).

Undo/Redo Logging Main idea of Undo/Redo logging: Compromising undo logging and redo logging, increasing flexibility. <T,V1, a1, b1> … <T, commit> New V1 = b1 <T, start> <T,V3, a3, b3> <T,V2, a2, b2> <T,V4, a4, b4> New V3 = b3 New V4 = b4 New V2 = b2 The order in which the values reach disk Undo/Redo logging rule (Rule UR): An undo/redo log record must reach disk before the corresponding change reaches disk.

Undo/Redo Logging Undo/Redo logging rule (UR): Log records reach disk before the corresponding changes reach disk. Thus, when the system crashes, we can look at disk-log: <T,V1, a1, b1> … <T, commit> New V1 = b1 <T, start> <T,V3, a3, b3> <T,V2, a2, b2> <T,V4, a4, b4> New V3 = b3 New V4 = b4 New V2 = b2 The order in which the values reach disk

Undo/Redo Logging Undo/Redo logging rule (UR): Log records reach disk before the corresponding changes reach disk. Thus, when the system crashes, we can look at disk-log: If we see <T, commit>, then all log records for T have reaches disk (although we are not sure if the corresponding changes have also reached disk), so we have all information to Redo T; <T,V1, a1, b1> … <T, commit> New V1 = b1 <T, start> <T,V3, a3, b3> <T,V2, a2, b2> <T,V4, a4, b4> New V3 = b3 New V4 = b4 New V2 = b2 The order in which the values reach disk

Undo/Redo Logging Undo/Redo logging rule (UR): Log records reach disk before the corresponding changes reach disk. Thus, when the system crashes, we can look at disk-log: If we see <T, commit>, then all log records for T have reaches disk (although we are not sure if the corresponding changes have also reached disk), so we have all information to Redo T; If we do not see <T, commit>, then we do not know if the log record for T is complete. However: <T,V1, a1, b1> … <T, commit> New V1 = b1 <T, start> <T,V3, a3, b3> <T,V2, a2, b2> <T,V4, a4, b4> New V3 = b3 New V4 = b4 New V2 = b2 The order in which the values reach disk

Undo/Redo Logging Undo/Redo logging rule (UR): Log records reach disk before the corresponding changes reach disk. Thus, when the system crashes, we can look at disk-log: If we see <T, commit>, then all log records for T have reaches disk (although we are not sure if the corresponding changes have also reached disk), so we have all information to Redo T; If we do not see <T, commit>, then we do not know if the log record for T is complete. However: * If record <T, V, aold, anew> is not in disk, then the corresponding change has not reached disk, yet, so we can ignore it; <T,V1, a1, b1> … <T, commit> New V1 = b1 <T, start> <T,V3, a3, b3> <T,V2, a2, b2> <T,V4, a4, b4> New V3 = b3 New V4 = b4 New V2 = b2 The order in which the values reach disk

Undo/Redo Logging Undo/Redo logging rule (UR): Log records reach disk before the corresponding changes reach disk. Thus, when the system crashes, we can look at disk-log: If we see <T, commit>, then all log records for T have reaches disk (although we are not sure if the corresponding changes have also reached disk), so we have all information to Redo T; If we do not see <T, commit>, then we do not know if the log record for T is complete. However: * If record <T, V, aold, anew> is not in disk, then the corresponding change has not reached disk, yet, so we can ignore it; * If record <T, V, aold, anew> is in disk, then, though we do not know if the corresponding change has reached disk, Undoing this action is always safe. <T,V1, a1, b1> … <T, commit> New V1 = b1 <T, start> <T,V3, a3, b3> <T,V2, a2, b2> <T,V4, a4, b4> New V3 = b3 New V4 = b4 New V2 = b2 The order in which the values reach disk

Undo/Redo Logging Undo/Redo logging rule (UR): Log records reach disk before the corresponding changes reach disk. Thus, when the system crashes, we can look at disk-log: If we see <T, commit>, then all log records for T have reaches disk (although we are not sure if the corresponding changes have also reached disk), so we have all information to Redo T; If we do not see <T, commit>, then we do not know if the log record for T is complete. However: * If record <T, V, aold, anew> is not in disk, then the corresponding change has not reached disk, yet, so we can ignore it; * If record <T, V, aold, anew> is in disk, then, though we do not know if the corresponding change has reached disk, Undoing this action is always safe. Redo should go forwards while Undo should go backwards. <T,V1, a1, b1> … <T, commit> New V1 = b1 <T, start> <T,V3, a3, b3> <T,V2, a2, b2> <T,V4, a4, b4> New V3 = b3 New V4 = b4 New V2 = b2 The order in which the values reach disk

Undo/Redo Logging Algorithms for Undo/Redo logging Undo/Redo logging rule (UR): Log records reach disk before the corresponding changes reach disk. Algorithms for Undo/Redo logging Undo/Redo log recording 1. Add <T, start> to memory-log; 2. When T writes on A, add <T, A, aold, anew> in memory-log; \\ output(A) can be performed now, \\ either before or after step 3 3. Add <T, commit> to memory-log. Recovery with Undo/Redo log 1. Bring disk-log to memory; 2. Scan the log forwards; 3. For T with <T,commit> in log For <T, A, aold, anew> in log Redo by assigning anew to A; 4. Scan the log backwards; 5. For T without <T,commit> in log Undo by assigning aold to A.

Undo/Redo-logging: Example Log recording 1. Add <T, start> to memory-log; 2. When T writes on A, add <T, A, aold, anew> in memory-log; \\ output(A) can be performed now, \\ either before or after step 3 3. Add <T, commit> to memory-log. Undo/Redo-logging: Example T1: A  A  3 B  B  3 read (A,t); t  t3; write (A,t); read (B,t); t  t3; write (B,t); output (A); output (B); CPU A=10 B=10 C=30 memory disk 26

Undo/Redo: log recording 1. Add <T, start> to memory-log; 2. When T writes on A, add <T, A, aold, anew> in memory-log; \\ output(A) can be performed now, \\ either before or after step 3 3. Add <T, commit> to memory-log. Undo/Redo: log recording T: A  A  3 B  B  3 add-log: <T, start> read (A,t); t  t3; write (A,t); add-log: <T, A, 10, 30> read (B,t); t  t3; write (B,t); add-log: <T, B, 10, 30> flush-log; output (B); add-log: <T, commit> output (A); CPU A=10 B=10 C=30 memory disk 27

Undo/Redo: log recording 1. Add <T, start> to memory-log; 2. When T writes on A, add <T, A, aold, anew> in memory-log; \\ output(A) can be performed now, \\ either before or after step 3 3. Add <T, commit> to memory-log. Undo/Redo: log recording T: A  A  3 B  B  3 add-log: <T, start> read (A,t); t  t3; write (A,t); add-log: <T, A, 10, 30> read (B,t); t  t3; write (B,t); add-log: <T, B, 10, 30> flush-log; output (B); add-log: <T, commit> output (A); CPU log A=10 B=10 C=30 A=30 <T, start> <T, A, 10, 30> <T, B, 10, 30> B=30 memory disk 28

Undo/Redo: log recording 1. Add <T, start> to memory-log; 2. When T writes on A, add <T, A, aold, anew> in memory-log; \\ output(A) can be performed now, \\ either before or after step 3 3. Add <T, commit> to memory-log. Undo/Redo: log recording T: A  A  3 B  B  3 add-log: <T, start> read (A,t); t  t3; write (A,t); add-log: <T, A, 10, 30> read (B,t); t  t3; write (B,t); add-log: <T, B, 10, 30> flush-log; output (B); add-log: <T, commit> output (A); CPU log A=10 B=10 C=30 A=30 <T, start> <T, A, 10, 30> <T, B, 10, 30> <T, start> <T, A, 10, 30> <T, B, 10, 30> B=30 memory disk 29

Undo/Redo: log recording 1. Add <T, start> to memory-log; 2. When T writes on A, add <T, A, aold, anew> in memory-log; \\ output(A) can be performed now, \\ either before or after step 3 3. Add <T, commit> to memory-log. Undo/Redo: log recording T: A  A  3 B  B  3 add-log: <T, start> read (A,t); t  t3; write (A,t); add-log: <T, A, 10, 30> read (B,t); t  t3; write (B,t); add-log: <T, B, 10, 30> flush-log; output (B); add-log: <T, commit> output (A); CPU log A=10 B=10 C=30 A=30 <T, start> <T, A, 10, 30> <T, B, 10, 30> B=30 B=30 <T, start> <T, A, 10, 30> <T, B, 10, 30> memory disk 30

Undo/Redo: log recording 1. Add <T, start> to memory-log; 2. When T writes on A, add <T, A, aold, anew> in memory-log; \\ output(A) can be performed now, \\ either before or after step 3 3. Add <T, commit> to memory-log. Undo/Redo: log recording T: A  A  3 B  B  3 add-log: <T, start> read (A,t); t  t3; write (A,t); add-log: <T, A, 10, 30> read (B,t); t  t3; write (B,t); add-log: <T, B, 10, 30> flush-log; output (B); add-log: <T, commit> output (A); CPU log A=10 B=30 C=30 A=30 <T, start> <T, A, 10, 30> <T, B, 10, 30> <T, commit> B=30 <T, start> <T, A, 10, 30> <T, B, 10, 30> memory disk 31

Undo/Redo: log recording 1. Add <T, start> to memory-log; 2. When T writes on A, add <T, A, aold, anew> in memory-log; \\ output(A) can be performed now, \\ either before or after step 3 3. Add <T, commit> to memory-log. Undo/Redo: log recording T: A  A  3 B  B  3 add-log: <T, start> read (A,t); t  t3; write (A,t); add-log: <T, A, 10, 30> read (B,t); t  t3; write (B,t); add-log: <T, B, 10, 30> flush-log; output (B); add-log: <T, commit> output (A); CPU log A=10 B=30 C=30 A=30 <T, start> <T, A, 10, 30> <T, B, 10, 30> <T, commit> <T, start> <T, A, 10, 30> <T, B, 10, 30> <T, commit> B=30 <T, start> <T, A, 10, 30> <T, B, 10, 30> memory disk 32

Undo/Redo: log recording 1. Add <T, start> to memory-log; 2. When T writes on A, add <T, A, aold, anew> in memory-log; \\ output(A) can be performed now, \\ either before or after step 3 3. Add <T, commit> to memory-log. Undo/Redo: log recording T: A  A  3 B  B  3 add-log: <T, start> read (A,t); t  t3; write (A,t); add-log: <T, A, 10, 30> read (B,t); t  t3; write (B,t); add-log: <T, B, 10, 30> flush-log; output (B); add-log: <T, commit> output (A); CPU log A=10 B=30 C=30 A=30 A=30 <T, start> <T, A, 10, 30> <T, B, 10, 30> <T, commit> B=30 <T, start> <T, A, 10, 30> <T, B, 10, 30> <T, commit> memory disk 33

Undo/Redo: Failure Recovery Recovery with Undo/Redo 1. Bring disk-log to memory; 2. Scan the log forwards; 3. For T with <T,commit> in log For <T, A, aold, anew> in log Redo by assigning anew to A; 4. Scan the log backwards; 5. For T without <T,commit> in log Undo by assigning aold to A. Undo/Redo: Failure Recovery CPU A=10 B=10 C=30 memory disk 34

Undo/Redo: Failure Recovery Recovery with Undo/Redo 1. Bring disk-log to memory; 2. Scan the log forwards; 3. For T with <T,commit> in log For <T, A, aold, anew> in log Redo by assigning anew to A; 4. Scan the log backwards; 5. For T without <T,commit> in log Undo by assigning aold to A. Undo/Redo: Failure Recovery T: A  A  3 B  B  3 add-log: <T, start> read (A,t); t  t3; write (A,t); add-log: <T, A, 10, 30> read (B,t); t  t3; write (B,t); add-log: <T, B, 10, 30> flush-log; output (B); add-log: <T, commit> output (A); CPU A=10 B=10 C=30 memory disk 35

crash occurs after <T, commit> is flushed to disk Recovery with Undo/Redo 1. Bring disk-log to memory; 2. Scan the log forwards; 3. For T with <T,commit> in log For <T, A, aold, anew> in log Redo by assigning anew to A; 4. Scan the log backwards; 5. For T without <T,commit> in log Undo by assigning aold to A. Undo/Redo: Failure Recovery T: A  A  3 B  B  3 add-log: <T, start> read (A,t); t  t3; write (A,t); add-log: <T, A, 10, 30> read (B,t); t  t3; write (B,t); add-log: <T, B, 10, 30> flush-log; output (B); add-log: <T, commit> output (A); CPU A=10 B=30 C=30 <T, start> <T, A, 10, 30> <T, B, 10, 30> <T, commit> crash occurs after <T, commit> is flushed to disk memory disk 36

crash occurs after <T, commit> is flushed to disk Recovery with Undo/Redo 1. Bring disk-log to memory; 2. Scan the log forwards; 3. For T with <T,commit> in log For <T, A, aold, anew> in log Redo by assigning anew to A; 4. Scan the log backwards; 5. For T without <T,commit> in log Undo by assigning aold to A. Undo/Redo: Failure Recovery T: A  A  3 B  B  3 add-log: <T, start> read (A,t); t  t3; write (A,t); add-log: <T, A, 10, 30> read (B,t); t  t3; write (B,t); add-log: <T, B, 10, 30> flush-log; output (B); add-log: <T, commit> output (A); CPU A=30 B=30 C=30 <T, start> <T, A, 10, 30> <T, B, 10, 30> <T, commit> crash occurs after <T, commit> is flushed to disk memory disk 37

crash occurs before <T, commit> is flushed to disk Recovery with Undo/Redo 1. Bring disk-log to memory; 2. Scan the log forwards; 3. For T with <T,commit> in log For <T, A, aold, anew> in log Redo by assigning anew to A; 4. Scan the log backwards; 5. For T without <T,commit> in log Undo by assigning aold to A. Undo/Redo: Failure Recovery T: A  A  3 B  B  3 add-log: <T, start> read (A,t); t  t3; write (A,t); add-log: <T, A, 10, 30> read (B,t); t  t3; write (B,t); add-log: <T, B, 10, 30> flush-log; output (B); add-log: <T, commit> output (A); CPU A=10 B=30 C=30 <T, start> <T, A, 10, 30> <T, B, 10, 30> crash occurs before <T, commit> is flushed to disk memory disk 38

crash occurs before <T, commit> is flushed to disk Recovery with Undo/Redo 1. Bring disk-log to memory; 2. Scan the log forwards; 3. For T with <T,commit> in log For <T, A, aold, anew> in log Redo by assigning anew to A; 4. Scan the log backwards; 5. For T without <T,commit> in log Undo by assigning aold to A. Undo/Redo: Failure Recovery T: A  A  3 B  B  3 add-log: <T, start> read (A,t); t  t3; write (A,t); add-log: <T, A, 10, 30> read (B,t); t  t3; write (B,t); add-log: <T, B, 10, 30> flush-log; output (B); add-log: <T, commit> output (A); CPU A=10 B=10 C=30 <T, start> <T, A, 10, 30> <T, B, 10, 30> crash occurs before <T, commit> is flushed to disk memory disk 39

Checkpoint for Undo/Redo logging …… <T1,start> … <T2,commit> <T3,start> … <T4,commit> 40

Checkpoint for Undo/Redo logging Place a log record <Start CKPT(T1, …, Tk)>, where T1, …, Tk are the currently active transactions; …… <T1,start> … <T2,commit> <T3,start> … <T4,commit> <Start CKPT (T1,T3)> 41

Checkpoint for Undo/Redo logging Place a log record <Start CKPT(T1, …, Tk)>, where T1, …, Tk are the currently active transactions; Copy all “dirty data” in main memory to disk, this can be due to either committed or incomplete transactions; \\ at the meanwhile, the system can continuously \\ accept new transactions (non-quiescent) …… <T1,start> … <T2,commit> <T3,start> … <T4,commit> <Start CKPT (T1,T3)> copy dirty data to disk … <T5,start> copy dirty data to disk 42

Checkpoint for Undo/Redo logging Place a log record <Start CKPT(T1, …, Tk)>, where T1, …, Tk are the currently active transactions; Copy all “dirty data” in main memory to disk, this can be due to either committed or incomplete transactions; \\ at the meanwhile, the system can continuously \\ accept new transactions (non-quiescent) When step 2 is done, place a log record <End CKPT>. …… <T1,start> … <T2,commit> <T3,start> … <T4,commit> <Start CKPT (T1,T3)> copy dirty data to disk … <T5,start> copy dirty data to disk <End CKPT> 43

Checkpoint for Undo/Redo logging Place a log record <Start CKPT(T1, …, Tk)>, where T1, …, Tk are the currently active transactions; Copy all “dirty data” in main memory to disk, this can be due to either committed or incomplete transactions; \\ at the meanwhile, the system can continuously \\ accept new transactions (non-quiescent) When step 2 is done, place a log record <End CKPT>. \\ at this point, nothing beyond <Start CKPT(…)> \\ needs Redone, but Undo still has to trace back to \\ the earliest <Ti, start> among (T1, …, Tk). …… <T1,start> … <T2,commit> <T3,start> … <T4,commit> <Start CKPT (T1,T3)> copy dirty data to disk … <T5,start> copy dirty data to disk <End CKPT> 44

Checkpoint for Undo/Redo logging Place a log record <Start CKPT(T1, …, Tk)>, where T1, …, Tk are the currently active transactions; Copy all “dirty data” in main memory to disk, this can be due to either committed or incomplete transactions; \\ at the meanwhile, the system can continuously \\ accept new transactions (non-quiescent) When step 2 is done, place a log record <End CKPT>. \\ at this point, nothing beyond <Start CKPT(…)> \\ needs Redone, but Undo still has to trace back to \\ the earliest <Ti, start> among (T1, …, Tk). \\ if crash occurs between <Start CKPT(T1, …, Tk)> \\ and <End CKPT>, then look for a previous \\ <End CKPT> <End CKPT> …… <T1,start> … <T2,commit> <T3,start> … <T4,commit> <Start CKPT (T1,T3)> copy dirty data to disk … <T5,start> copy dirty data to disk <End CKPT> 45

Tranactions using logging What can affect transaction ACID? Atomicity: either all steps of a transaction happen, or none happen Consistency: a transaction transforms a consistent DB into a consistent DB Isolation: execution of a transaction is isolated from that of other transactions Durability: if a transaction commits, its effects persist. using logging What can affect transaction ACID? Accident system failures may destroy Atomicity and Durability, thus Consistency; Concurrent executions of transactions (which is highly desired) may affect Isolation, thus Consistency.

Tranactions What can affect transaction ACID? Atomicity: either all steps of a transaction happen, or none happen Consistency: a transaction transforms a consistent DB into a consistent DB Isolation: execution of a transaction is isolated from that of other transactions Durability: if a transaction commits, its effects persist. Database T1 …… Tn What can affect transaction ACID? Accident system failures may destroy Atomicity and Durability, thus Consistency; Concurrent executions of transactions (which is highly desired) may affect Isolation, thus Consistency.

DBMS Graduate Database lock table DDL language DDL complier file administrator DDL complier lock table DDL language file manager logging & recovery concurrency control transaction manager database programmer index/file manager buffer manager query execution engine DML complier main memory buffers DML (query) language secondary storage (disks) DBMS Graduate Database

DBMS Graduate Database lock table DDL language DDL complier file administrator DDL complier lock table DDL language file manager logging & recovery concurrency control transaction manager database programmer index/file manager buffer manager query execution engine DML complier main memory buffers DML (query) language secondary storage (disks) DBMS Graduate Database