1. CSE544: Transactions
Recovery
Wednesday, 4/19/2006

2. Transactions Major component of database systems
Critical for most applications; arguably more so than SQL
Turing awards to database researchers:
Charles Bachman 1973
Edgar Codd 1981 for inventing relational dbs
Jim Gray 1998 for inventing transactions

3. Why Do We Need Transactions
Concurrency control
Recovery

4. Concurrency Control Lost update What’s wrong ? Client 1:
UPDATE Product SET Price = Price – WHERE pname = ‘Gizmo’ Client 2: UPDATE Product SET Price = Price*0.5 WHERE pname=‘Gizmo’
What’s wrong ?
Lost update

5. Concurrency Control Inconsistent reads What’s wrong ?
Client 1: INSERT INTO SmallProduct(name, price) SELECT pname, price FROM Product
WHERE price <= 0.99
DELETE Product WHERE price <=0.99 Client 2: SELECT count(*) FROM Product SELECT count(*) FROM SmallProduct
Inconsistent reads
What’s wrong ?

6. Concurrency Control What’s wrong ? Dirty reads
Client 1: UPDATE SET Account.amount = WHERE Account.number = ‘my-account’
Client 2: SELECT Account.amount FROM Account WHERE Account.number = ‘my-account’
Aborted by
system
What’s wrong ?
Dirty reads

7. Protection against crashes
Client 1:
INSERT INTO SmallProduct(name, price) SELECT pname, price FROM Product
WHERE price <= 0.99
DELETE Product WHERE price <=0.99
Crash !
What’s wrong ?

8. Definition
A transaction = one or more operations, single real-world transition
Examples
Transfer money between accounts
Purchase a group of products
Register for a class (either waitlist or allocated)

9. May be omitted: first SQL query starts txn
Transactions in SQL
In “ad-hoc” SQL:
Default: each statement = one transaction
In a program:
START TRANSACTION
[SQL statements]
COMMIT or ROLLBACK (=ABORT)
May be omitted: first SQL query starts txn

10. Revised Code Client 1: START TRANSACTION
UPDATE Product SET Price = Price – WHERE pname = ‘Gizmo’
COMMIT
Client 2: START TRANSACTION UPDATE Product SET Price = Price*0.5 WHERE pname=‘Gizmo’ COMMIT

11. ROLLBACK
If the app gets to a place where it can’t complete the transaction successfully, it can execute ROLLBACK
This causes the system to “abort” the transaction
The database returns to the state without any of the previous changes made by activity of the transaction

12. Reasons for Rollback User changes their mind (“ctl-C”/cancel)
Explicit in program, when app program finds a problem
e.g. when qty on hand < qty being sold
System-initiated abort
System crash
Housekeeping
e.g. due to timeouts

13. Transaction Properties ACID
Atomic
Consistent
Isolated
Durable

14. Atomic Two possible outcomes for a transaction
It commits: all the changes are made
It aborts: no changes are made
What it means for us: recovery

15. Consistent
The state of the database is restricted by integrity constraints
Constraints may be explicit or implicit
A transaction must take a database from a consistent state to a consistent state
What it means for us: nothing, follows from Atomicity and Isolation

16. Isolated A transaction executes concurrently with other transaction
Its effect must be as if each transaction executes in isolation of the others
What it means for us: concurrency control

17. Durable
The effect of a transaction must continue to exists after the transaction, or the whole program has terminated
What it means for us: write data to disk
Note: some papers/books interpret this as need for recovery

18. Transactions in SQL Isolation level “serializable” (i.e. ACID)
Golden standard
But often too inefficient
Weaker isolation levels
Sacrifice correctness for efficiency
Often used in practice
Sometimes are hard to understand

19. READ-ONLY Transactions
Client 1: START TRANSACTION
INSERT INTO SmallProduct(name, price) SELECT pname, price FROM Product
WHERE price <= 0.99
DELETE Product WHERE price <=0.99
COMMIT
Client 2: SET TRANSACTION READ ONLY
START TRANSACTION
SELECT count(*) FROM Product SELECT count(*) FROM SmallProduct COMMIT
Makes it faster

20. Isolation Levels in SQL
“Dirty reads” SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED
“Committed reads” SET TRANSACTION ISOLATION LEVEL READ COMMITTED
“Repeatable reads” SET TRANSACTION ISOLATION LEVEL REPEATABLE READ
Serializable transactions (default): SET TRANSACTION ISOLATION LEVEL SERIALIZABLE

21. Isolation Level: Dirty Reads
function AllocateSeat( %request)
SET ISOLATION LEVEL READ UNCOMMITED
START TRANSACTION
Let x = SELECT Seat.occupied FROM Seat
WHERE Seat.number = %request
If (x == 1) /* occupied */ ROLLBACK
UPDATE Seat
SET occupied = 1 WHERE Seat.number = %request
COMMIT
Plane seat allocation
What can go
wrong ?
What can go
wrong if only the function AllocateSeat modifies Seat ?

22. What if we switch the two updates ?
function TransferMoney( %amount, %acc1, %acc2)
START TRANSACTION
Let x = SELECT Account.balance FROM Account
WHERE Account.number = %acc1
If (x < %amount) ROLLBACK
UPDATE Account
SET balance = balance+%amount WHERE Account.number = %acc2
SET balance = balance-%amount WHERE Account.number = %acc1
COMMIT
Are dirty reads OK here ?
What if we switch the two updates ?

23. Isolation Level: Read Committed
Stronger than READ UNCOMMITTED
SET ISOLATION LEVEL READ COMMITED
Let x = SELECT Seat.occupied FROM Seat
WHERE Seat.number = %request
/* More stuff here */
Let y = SELECT Seat.occupied FROM Seat
/* we may have x  y ! */
It is possible to read twice, and get different values

24. Isolation Level: Repeatable Read
Stronger than READ COMMITTED
SET ISOLATION LEVEL REPEATABLE READ
Let x = SELECT Account.amount FROM Account
WHERE Account.number = ‘555555’
/* More stuff here */
Let y = SELECT Account.amount FROM Account
WHERE Account.number = ‘777777’
/* we may have a wrong x+y ! */
May see incompatible values:
another txn transfers from acc to

25. Isolation Level: Serializable
SET ISOLATION LEVEL SERIALIZABLE
Strongest level
Default

26. The Mechanics of Disk Mechanical characteristics:
Cylinder
Mechanical characteristics:
Rotation speed (5400RPM)
Number of platters (1-30)
Number of tracks (<=10000)
Number of bytes/track(105)
Spindle
Tracks
Disk head
Sector
Unit of read or write:
disk block
Once in memory:
page
Typically: 4k or 8k or 16k
Arm movement
Platters
Arm assembly

27. Disk Access Characteristics
Disk latency = time between when command is issued and when data is in memory
Disk latency = seek time + rotational latency
Seek time = time for the head to reach cylinder
10ms – 40ms
Rotational latency = time for the sector to rotate
Rotation time = 10ms
Average latency = 10ms/2
Transfer time = typically 40MB/s
Disks read/write one block at a time

28. Buffer Management in a DBMS
Page Requests from Higher Levels
READ
WRITE
BUFFER POOL
disk page
free frame
INPUT
OUTUPT
MAIN MEMORY
DISK DB
choice of frame dictated
by replacement policy
Data must be in RAM for DBMS to operate on it!
Table of <frame#, pageid> pairs is maintained 4

29. Buffer Manager Needs to decide on page replacement policy LRU
Clock algorithm
Both work well in OS, but not always in DB
Enables the higher levels of the DBMS to assume that the
needed data is in main memory.

30. Least Recently Used (LRU)
Order pages by the time of last accessed
Always replace the least recently accessed
P5, P2, P8, P4, P1, P9, P6, P3, P7
Access P6
P6, P5, P2, P8, P4, P1, P9, P3, P7
LRU is expensive (why ?); the clock algorithm is good approx

31. Buffer Manager Why not use the Operating System for the task??
- DBMS may be able to anticipate access patterns
- Hence, may also be able to perform prefetching
DBMS needs the ability to force pages to disk, for recovery purposes
need fine grained control for transactions

32. Recovery
From which of the events below can a database actually recover ?
Wrong data entry
Disk failure
Fire / earthquake / bankrupcy / ….
Systems crashes

33. Recovery Type of Crash Prevention Wrong data entry
Constraints and Data cleaning
Disk crashes
Redundancy: e.g. RAID, archive
Fire, theft, bankruptcy…
Buy insurance, Change jobs…
System failures: e.g. power
DATABASE RECOVERY
Most frequent

34. Recovery from System Failures
Use a log
A file that records every single action of the transaction
Last entry
Log on disc
flush
Log tail in memory

35. Transactions Assumption: the database is composed of elements
Usually 1 element = 1 block
Can be smaller (=1 record) or larger (=1 relation)
Assumption: each transaction reads/writes some elements

36. Primitive Operations of Transactions
READ(X,t)
copy element X to transaction local variable t
WRITE(X,t)
copy transaction local variable t to element X
INPUT(X)
read element X to memory buffer
OUTPUT(X)
write element X to disk

37. Example START TRANSACTION READ(A,t); t := t*2; WRITE(A,t); READ(B,t);
WRITE(B,t)
COMMIT;
Atomicity:
BOTH A and B are multiplied by 2

38. Action t Mem A Mem B Disk A Disk B INPUT(A) 8 READ(A,t) t:=t*2 16
READ(A,t); t := t*2; WRITE(A,t);
READ(B,t); t := t*2; WRITE(B,t)
Transaction
Buffer pool
Disk
Action t
Mem A
Mem B
Disk A
Disk B
INPUT(A) 8
READ(A,t)
t:=t*2 16
WRITE(A,t)
INPUT(B)
READ(B,t)
WRITE(B,t)
OUTPUT(A)
OUTPUT(B)

39. Crash occurs after OUTPUT(A), before OUTPUT(B) We lose atomicity
Action t
Mem A
Mem B
Disk A
Disk B
INPUT(A) 8
READ(A,t)
t:=t*2 16
WRITE(A,t)
INPUT(B)
READ(B,t)
WRITE(B,t)
OUTPUT(A)
OUTPUT(B)
Crash !
Crash occurs after OUTPUT(A), before OUTPUT(B)
We lose atomicity

40. The Log An append-only file containing log records
Note: multiple transactions run concurrently, log records are interleaved
After a system crash, use log to:
Redo some transaction that didn’t commit
Undo other transactions that didn’t commit
Three kinds of logs: undo, redo, undo/redo

41. Undo Logging Log records <START T> <COMMIT T>
transaction T has begun
<COMMIT T>
T has committed
<ABORT T>
T has aborted
<T,X,v>
T has updated element X, and its old value was v

42. INPUT(B) Action T Mem A Mem B Disk A Disk B Log <START T>
INPUT(A) 8
READ(A,t)
t:=t*2 16
WRITE(A,t)
<T,A,8>
INPUT(B)
READ(B,t)
WRITE(B,t)
<T,B,8>
OUTPUT(A)
OUTPUT(B)
COMMIT
<COMMIT T>

43. Crash ! WHAT DO WE DO ? INPUT(B) Action T Mem A Mem B Disk A Disk B
Log
<START T>
INPUT(A) 8
READ(A,t)
t:=t*2 16
WRITE(A,t)
<T,A,8>
INPUT(B)
READ(B,t)
WRITE(B,t)
<T,B,8>
OUTPUT(A)
OUTPUT(B)
COMMIT
<COMMIT T>
Were we UNDO the changes, writing back A=8, and B=8. The transaction is atomic: none of its actions was executed
Crash !
WHAT DO WE DO ?

44. Crash ! WHAT DO WE DO ? INPUT(B) Action T Mem A Mem B Disk A Disk B
Log
<START T>
INPUT(A) 8
READ(A,t)
t:=t*2 16
WRITE(A,t)
<T,A,8>
INPUT(B)
READ(B,t)
WRITE(B,t)
<T,B,8>
OUTPUT(A)
OUTPUT(B)
COMMIT
<COMMIT T>
Here we don’t need to do any recovery actions. The transaction is still atomic: both actions have been executed.
Crash !
WHAT DO WE DO ?

45. After Crash In the first example: In the second example
We UNDO both changes: A=8, B=8
The transaction is atomic, since none of its actions has been executed
In the second example
We don’t undo anything
The transaction is atomic, since both it’s actions have been executed

46. Undo-Logging Rules
U1: If T modifies X, then <T,X,v> must be written to disk before OUTPUT(X)
U2: If T commits, then OUTPUT(X) must be written to disk before <COMMIT T>
Hence: OUTPUTs are done early, before the transaction commits

47. INPUT(B) Action T Mem A Mem B Disk A Disk B Log <START T>
INPUT(A) 8
READ(A,t)
t:=t*2 16
WRITE(A,t)
<T,A,8>
INPUT(B)
READ(B,t)
WRITE(B,t)
<T,B,8>
OUTPUT(A)
OUTPUT(B)
COMMIT
<COMMIT T>

48. Recovery with Undo Log After system’s crash, run recovery manager
Idea 1. Decide for each transaction T whether it is completed or not
<START T>….<COMMIT T>…. = yes
<START T>….<ABORT T>……. = yes
<START T>……………………… = no
Idea 2. Undo all modifications by incomplete transactions

49. Recovery with Undo Log Recovery manager: Read log from the end; cases:
<COMMIT T>: mark T as completed
<ABORT T>: mark T as completed
<T,X,v>: if T is not completed then write X=v to disk else ignore
<START T>: ignore

50. Recovery with Undo Log crash … <T6,X6,v6> Question1 in class:
<START T5>
<START T4>
<T1,X1,v1>
<T5,X5,v5>
<T4,X4,v4>
<COMMIT T5>
<T3,X3,v3>
<T2,X2,v2>
Question1 in class:
Which updates are undone ?
Question 2 in class:
How far back do we need to read in the log ?
crash

51. Recovery with Undo Log Note: all undo commands are idempotent
If we perform them a second time, no harm is done
E.g. if there is a system crash during recovery, simply restart recovery from scratch

52. Recovery with Undo Log When do we stop reading the log ?
We cannot stop until we reach the beginning of the log file
This is impractical
Instead: use checkpointing

53. Checkpointing Checkpoint the database periodically
Stop accepting new transactions
Wait until all current transactions complete
Flush log to disk
Write a <CKPT> log record, flush
Resume transactions

54. Undo Recovery with Checkpointing…
<T9,X9,v9>
(all completed)
<CKPT>
<START T2>
<START T3
<START T5>
<START T4>
<T1,X1,v1>
<T5,X5,v5>
<T4,X4,v4>
<COMMIT T5>
<T3,X3,v3>
<T2,X2,v2>
other transactions
During recovery,
Can stop at first
<CKPT>
transactions T2,T3,T4,T5

55. Nonquiescent Checkpointing
Problem with checkpointing: database freezes during checkpoint
Would like to checkpoint while database is operational
Idea: nonquiescent checkpointing
Quiescent = being quiet, still, or at rest; inactive
Non-quiescent = allowing transactions to be active

56. Nonquiescent Checkpointing
Write a <START CKPT(T1,…,Tk)> where T1,…,Tk are all active transactions
Continue normal operation
When all of T1,…,Tk have completed, write <END CKPT>

57. Undo Recovery with Nonquiescent Checkpointing…
<START CKPT T4, T5, T6>
<END CKPT>
earlier transactions plus
T4, T5, T5
During recovery,
Can stop at first
<CKPT>
T4, T5, T6, plus
later transactions
Answer: because if the system crashes before it writes <END CKPT> then we cannot use the checkpoint. We use <END CKPT> to ensure that the system didn’t crash and the checkpoint terminated.
later transactions
Q: why do we need
<END CKPT> ?

58. Redo Logging Log records <START T> = transaction T has begun
<COMMIT T> = T has committed
<ABORT T>= T has aborted
<T,X,v>= T has updated element X, and its new value is v

59. Action T
Mem A
Mem B
Disk A
Disk B
Log
<START T>
READ(A,t) 8
t:=t*2 16
WRITE(A,t)
<T,A,16>
READ(B,t)
WRITE(B,t)
<T,B,16>
<COMMIT T>
OUTPUT(A)
OUTPUT(B)

60. Redo-Logging Rules
R1: If T modifies X, then both <T,X,v> and <COMMIT T> must be written to disk before OUTPUT(X)
Hence: OUTPUTs are done late

61. Action T
Mem A
Mem B
Disk A
Disk B
Log
<START T>
READ(A,t) 8
t:=t*2 16
WRITE(A,t)
<T,A,16>
READ(B,t)
WRITE(B,t)
<T,B,16>
<COMMIT T>
OUTPUT(A)
OUTPUT(B)

62. Recovery with Redo Log After system’s crash, run recovery manager
Step 1. Decide for each transaction T whether it is completed or not
<START T>….<COMMIT T>…. = yes
<START T>….<ABORT T>……. = yes
<START T>……………………… = no
Step 2. Read log from the beginning, redo all updates of committed transactions

63. Recovery with Redo Log <START T1> <T1,X1,v1>
<COMMIT T2>
<T3,X4,v4>
<T1,X5,v5> …

64. Nonquiescent Checkpointing
Write a <START CKPT(T1,…,Tk)> where T1,…,Tk are all active transactions
Flush to disk all blocks of committed transactions (dirty blocks), while continuing normal operation
When all blocks have been written, write <END CKPT>

65. Redo Recovery with Nonquiescent Checkpointing…
<START T1>
<COMMIT T1>
… <START T4>
<START CKPT T4, T5, T6>
<END CKPT>
<START CKPT T9, T10>
Step 2: redo
from the earliest start of T4, T5, T6 ignoring
transactions
committed
earlier
Step 1: look for
The last
<END CKPT>
All OUTPUTs
of T1 are
known to be on disk

66. Comparison Undo/Redo Undo logging: Redo logging
OUTPUT must be done early
If <COMMIT T> is seen, T definitely has written all its data to disk (hence, don’t need to redo) – inefficient
Redo logging
OUTPUT must be done late
If <COMMIT T> is not seen, T definitely has not written any of its data to disk (hence there is not dirty data on disk, no need to undo) – inflexible
Would like more flexibility on when to OUTPUT: undo/redo logging (next)

67. Undo/Redo Logging Log records, only one change
<T,X,u,v>= T has updated element X, its old value was u, and its new value is v

68. Undo/Redo-Logging Rule
UR1: If T modifies X, then <T,X,u,v> must be written to disk before OUTPUT(X)
Note: we are free to OUTPUT early or late relative to <COMMIT T>

69. Can OUTPUT whenever we want: before/after COMMIT
Action T
Mem A
Mem B
Disk A
Disk B
Log
<START T>
REAT(A,t) 8
t:=t*2 16
WRITE(A,t)
<T,A,8,16>
READ(B,t)
WRITE(B,t)
<T,B,8,16>
OUTPUT(A)
<COMMIT T>
OUTPUT(B)
Can OUTPUT whenever we want: before/after COMMIT

70. Recovery with Undo/Redo Log
After system’s crash, run recovery manager
Redo all committed transaction, top-down
Undo all uncommitted transactions, bottom-up

71. Recovery with Undo/Redo Log
<START T1>
<T1,X1,v1>
<START T2>
<T2, X2, v2>
<START T3>
<T1,X3,v3>
<COMMIT T2>
<T3,X4,v4>
<T1,X5,v5> …

72. The Log in Real Databases
Uses the ARIES algorithm
Same high level idea: undo/redo, lots of clever improvements (pointers between log entries etc)
ARIES is in the book - reading is optional
Franklin’s paper - reading is mandatory