1. Information Resources Management April 10, 2001

2. Agenda n Administrivia n Database Design n Denormalization n Database Administration n Security n Backup & Recovery n Concurrency Controls

3. Administrivia

4. Schema Tuning - Staying Normal n Split Tables - Vertical Partitioning n Highly used vs. infrequently used columns n Don’t partition if result will be more joins n Keys are duplicated

5. Schema Tuning - Staying Normal n Variable length fields (VARCHAR, others) n Indeterminant record lengths n Row locations vary n Vertically partition row into two tables, one with fixed and one with variable columns

6. Schema Tuning - Leaving Normal n Normalization n Eliminates duplication n Reduces anomalies n Does not result in efficiency n Denormalize for performance

7. Denormalization Warnings n Increases chance of errors or inconsistencies n May result in reprogramming if business rules change n Optimizes based on current transaction mix n Increases duplication and space required n Increases programming complexity n Always normalize first then denormalize

8. Denormalization n Partition Rows n Combine Tables n Combine and Partition n Replicate Data

9. Combining Opportunities n One-to-one (optional) n allow nulls n Many-to-many (assoc. entity) n 2 tables instead of 3 n Reference data (one-to-many) n “one” not use elsewhere n few of “many”

10. Combining Examples n Employee-Spouse (name and SSN only) n Owner-PctOwned - Property n few owners with multiple properties n Property-Type (description) n one type per property

11. Partitioning n Horizontal n By row type n Separate processing by type n Supertype/subtype decision n Vertical (already seen) n Both

12. Replication n Intentionally repeating data n Example: Owner-PctOwned-Property n Owner includes PctOwned & PropertyID n Property includes majority OwnerSSN and PctOwned

13. Performance Tuning n Not a one-time event n Monitoring probably more important n Things change n applications, database (table) sizes, data characteristics n hardware, operating system, DBMS

14. Database Administration n Security n Backup & Recovery n Concurrency Controls

15. Security - Authorization Row Operations n Read n Insert n Update n Delete Table Operations n Index n Creation/Removal n Resource n New Tables n Alteration n Drop

16. Authorization Granularity n Table-level only n View is the same as a table

17. Views n Select statement that is given a table name n Views can select from other views CREATE VIEW OfficeEmps AS (SELECT O.OfficeNbr, E1.EmpID, E1.Name, M.EmpID, E2.Name AS MgrName) FROM Office AS O, Manager AS M, Employee AS E1, Employee as E2 WHERE O.OfficeNbr = M.OfficeNbr AND M.EmpID = E2.EmpID AND O.OfficeNbr = E1.OfficeNbr and E1.EmpID <> E2.EmpID)

18. Enhancing Granularity Through Views n Specific Columns - SELECT xxxx n Specific Rows - WHERE xxxx=yyyy n Both

19. SQL n GRANT priviledge ON table TO user n (WITH GRANT OPTION) n REVOKE priviledge ON table FROM user n (RESTRICT or CASCADE) n GRANTS by that user on that table

20. Types of Failures n Transaction n Logical n System n Operating System n Hardware n Network n Disk

21. Recovery Approaches n Switch - mirror DB needed (RAID-1) n Restore/Rerun n Previous backup n Rerun all transactions (needed) n Log-Based n Rollback - undo incomplete n Rollforward - previous backup

22. Requirements n Permanently write changes without changing the database n Transaction States n Partially Committed - transaction is done n Fully Committed - changes have been made n can fail from partially committed state

23. Log-Based Recovery n Log - record of all database activity n Log Records n Transaction start n Transaction write (update) n new and old values n Transaction abort n Transaction commit

24. Log-Based Recovery n Deferred n Immediate

25. Deferred Log TransLog DB Database modification occurs after transaction commits

26. Deferred Log n Only new values kept in update log record n Only committed changes need to be reapplied at recovery n Uncommitted changes can be removed from the log

27. Deferred Log Example

28. Recovery only deletes from log

29. Deferred Log Example REDO(T1) - commit vs. actual database update

30. Deferred Log Example REDO(T1); Delete T2 from Log

31. Deferred Log Example REDO(T1); REDO(T2)

32. Failure During Recovery n Recovery from recovery must be possible n Redo must be executable multiple times without any differences from a single execution

33. Immediate Modification TransLog DB Database modified as transaction proceeds

34. Immediate Modification n Update log records require old and new values n Recovery requires either a REDO or an UNDO based on whether or not each transaction was committed

35. Immediate Example

36. UNDO(T1)

37. Immediate Example REDO(T1)

38. Immediate Example UNDO(T2); REDO(T1) -- order can be important

39. Immediate Example REDO(T1); REDO(T2)

40. Logging Requirements n Log must always be in “stable storage” n All log writes must be successful n Log kept separate from database n Backup copy of database that coincides with start of a new log n Recovery needed dependent on type of failure n Database restart must recovery completely before allowing new transactions

41. Checkpoints n Recovery has to search entire log n Many REDOs are unnecessary n Recovery can be a lengthy process n Checkpoints are used to limit the recovery action that is needed

42. Checkpoints 1. Flush all log records to permanent storage 2. Flush all data buffers to permanent storage 3. Write a to the permanent storage copy of the log storage copy of the log No updates are allowed while checkpointing

43. Checkpoint Recovery 1. Search from end of log to most recent 1. Search from end of log to most recent 2. Continue searching backward until the first transaction before the 2. Continue searching backward until the first transaction before the 3. From that onward, UNDO and REDO all transactions (Serial execution only)

44. Advantages of Logging n Less Overhead at Commit n No Data Fragmentation n No Need for Garbage Collection n Faster recovery n Support for Concurrency

45. Transactions n Concept n State n Serializability n Maintaining Serializability

46. Transaction n Single Unit of Work - User’s Perspective n Multiple Operations n Required Properties (ACID) n Atomicity - all or none n Consistency - database consistency maintained n Isolation - appearance of being alone n Durability - changes persist

47. Transaction State n Active n Partially Committed n Failed n Aborted n Committed Active Partially Committed CommittedFailedAborted

48. Implementing Transactions in SQL n COMMIT WORK n ROLLBACK WORK

49. Atomicity & Durability n Easiest n Completely new copy of database n Update new copy n Don’t update pointer until commit n Recoverable from failure at any point provided the acknowledgement of the commit and the update of the pointer occur simultaneously.

50. Concurrency n Multiple Transactions n Serial (one at a time) is best but n combination of slow & fast in single transaction n short and long transactions n Concurrency must be handled carefully

51. Example Employee (EmpID, Grade, Salary) Grade (Grade, Midpoint) Employee:75, 10, 25000 Grade:10, 20000 11, 30000

52. Example n T1 - Change employee #75 to grade 11 READ (Employee) Grade = 11 WRITE (Employee) n T2 - Update salaries by 5% of midpoint READ (Employee) READ (Grade) Salary = Salary + (0.05 * Midpoint) WRITE (Employee)

53. Example - Serial Execution n T1 then T2 n Result: Salary = 26500 (25000 +.05*30000) n T2 then T1 n Result: Salary = 26000 (25000 +.05*20000)

54. Concurrent Execution Result?

55. Concurrent Execution Result?

56. Recoverable Schedules n If T2 reads an item updated by T1, T1 must commit before T2 Cascadeless Schedule n If T2 reads an item updated by T1, T1 must commit before T2 reads

57. Not Recoverable Result?

58. Recoverable

59. Recoverable

60. Cascadeless

61. Ensuring Serializability n Concurrency Control Schemes n Can’t analyze transactions n some in progress n analysis longer than transaction n already running continue to run

62. Concurrency Control - Locks n Shared - Read onlyLOCK-S n Exclusive - Read/WriteLOCK-X n Compatibility of Locks n multiple transactions can have the same lock n shared locks only

63. Deadlocks T1: READ(A), READ(B), WRITE(A) T2: READ(B), READ(A), WRITE(B)

64. Locking Protocol n Set of Rules n Reduce Possibility of Deadlocks n Create “appearance” of serial execution n to each transaction

65. Two-Phase Locking Protocol n Growing Phase n Can obtain but not release locks n Shrinking Phase n Can release but not obtain locks n First release of a lock is the transition between phases (lock point)

66. Two-Phase Locking n Strict n Prevent cascading rollbacks n Exclusive locks (LOCK-X) held until commit n Rigorous n All locks held until commit

67. Lock Conversion n Changing a Lock n Upgrade - shared to exclusive n Downgrade - exclusive to shared n Can only upgrade in growing phase n Can only downgrade in shrinking phase

68. Most Used Locking Scheme n Read  LOCK-S(A), READ(A) n Write n If LOCK-S(A),  UPGRADE(A), WRITE(A) n If no lock,  LOCK-X(A), WRITE(A) n Locks held until COMMIT or ROLLBACK n Strict - exclusive only n Rigorous - all locks

69. Granularity n Lock only what is needed n Could be n Row n Table n Set of Tables n Entire Database n Model as a tree with the database at the root and the rows as the leaves

70. Intention Locking n To lock a row n Traverse the tree from the root to the row n Put intention locks on the nodes on the way down n Intention locks provide knowledge of lower level locks when a higher level lock is desired -- prevents having to traverse the entire tree to lock the database

71. Intention Locking n Locks Acquired n Top-Down n Locks Released n Bottom-Up

72. Deadlocks n Prevention n Recovery

73. Deadlock Prevention 1. Acquire all locks simultaneously 2. Rollback instead of waiting for a lock 2a. Lock wait timeouts

74. Deadlock Recovery n If not prevented, deadlocks must be detected and recovered n Detection - periodically search for problems n Recovery n Select a victim - which one? n Rollback - how far? n Avoid Starvation n Always killing the same victim which never gets to execute

75. Recovery with Concurrency n Locking Protocol n Transaction Rollback n Checkpoints n Restart

76. Recovery Locking Protocol n Recovery Dependent on Locking n Multiple UNDOs may not work correctly if a second transaction reads a value updated by a prior transaction before the prior transaction commits n Use Two-Phase Locking that is at least Strict

77. Transaction Rollback n Use log records to complete the rollback n Must rollback from most recent to earlier updates n Release exclusive locks after rollback is completed

78. Checkpoints n Multiple transactions can be active at a checkpoint n Change log record to include list of all currently active transactions n Still have to halt other processing while checkpointing

79. Checkpointing - When? n More often checkpoints -> faster recovery n Less often -> longer recovery n MTBF - all components n Timing n Amount of Activity n # transactions n # updates n log file size

80. Restart Recovery n Redo list - commit found n Undo list - start found - not on Redo list n Scan log backwards from the end n Stop at n Stop at n For each transaction on the checkpoint list not on the Redo list, add it to the Undo list

81. Restart Recovery 1. Starting again at the end of the log, Undo all transactions on the Undo list 2. Return to the most recent checkpoint 3. Move forward and redo all transactions on the redo list

82. Homework #8 n Database Design n Database Administration