1. H.Lu/HKUST L07: Lock Tuning

2. L07-Lock Tuning - 2 H.Lu/HKUST Lock Tuning  The key is to combine the theory of concurrency control with practical DBMS knowledge  Goal: maximise DBMS throughput (not the response time of any single transaction)  Understand DBMS  Each DBMS product may have different default locking behaviours  Example: SQL Server and Sybase Write locks are held to the end of a transaction Read locks are released immediately after use Not 2PL!  Understand applications  What is the requirement for your transaction?  What will be the transactions that will run in parallel to your transaction?  Where is the start and the end of your transaction?  Understand runtime environment  What are the concurrent transactions?  What are their priorities?  How your system has performed so far?

3. L07-Lock Tuning - 3 H.Lu/HKUST Ideal Transaction  Acquires few locks and favors shared locks over exclusive locks  Reduce the number of conflicts -- conflicts are due to exclusive locks  Acquires locks with fine granularity  Reduce the scope of each conflict  Holds locks for a short time  Reduce waiting

4. L07-Lock Tuning - 4 H.Lu/HKUST Lock Tuning – Some Guidelines  Use special system facilities for long reads  E.g., multiversion read consistency  Eliminate locking when it is unnecessary  E.g. when bulk loading, or statistical analysis  Take advantages of transactional context to chop transactions into small pieces  Refer the textbook for a formal method  Weaken isolation guarantees when the application allows it  SQL allows 4 level of consistency options  Select the appropriate locking granularity  Table, record, field…  Do DDL statements when not busy  Think about partitioning  E.g., use multiple physical disks  Void hot spots  Tune the deadlock intervals  How long to time-out a transaction?

5. L07-Lock Tuning - 5 H.Lu/HKUST Snapshot Isolation T1 T2 T3 TIME R(Y) returns 1 R(Z) returns 0 R(X) returns 0 W(Y:=1) W(X:=2, Z:=3) X=Y=Z=0  A facility provided by some DBMS (e.g. Oracle) that allows read-only queries hold no locks, yet appear to execute serializably.  Each transaction executes against the version of the data items that was committed when the transaction started:  No locks for read  Costs space (old copy of data must be kept)  When extended to read/write, no guarantee of correctness  T1: x:=y  T2: y:= x  Initially x=3 and y =17  Serial execution: x,y=17 or x,y=3  Snapshot isolation: x=17, y=3 if both transactions start at the same time.

6. L07-Lock Tuning - 6 H.Lu/HKUST Eliminate Unnecessary Locking Locking is unnecessary  when only one transaction runs at a time  e.g. loading the database  When all transactions are read-only  When doing decision support queries on archival database

7. L07-Lock Tuning - 7 H.Lu/HKUST Weaken Isolation Levels  Correctness vs. Performance  Number of locks held by each transaction  Kind of locks  Length of time a transaction holds locks  Life is full of compromises  High performance, at the cost of allowing some bad things happing  Application programmer and DBA should make a decision An informed decision!

8. L07-Lock Tuning - 8 H.Lu/HKUST SQL Isolation Levels  Degree 0:  Allow dirty read and lost update/nonrepeatable reads  Write-locks released immediately after writing, and no read locks  Degree 1: Read Uncommitted (No lost update)  Read may read dirty data, and will not be repeatable.  Writes may not overwrite other transactions ’ dirty data.  Write-locks held for the duration of the transactions  No read-locks  Degree 2: Read Committed (No dirty retrieval)  Reads may access only committed data; but may not repeatable.  Write locks follow two phase locking; read-locks released immediately after the read operation.  SQL Server default option  Degree 3: Serializable (no unrepeatable reads for read/write )  Reads may access only committed data, and reads are repeatable.  Writes may not overwrite other transactions ’ dirty data.  Two phase locking.

9. L07-Lock Tuning - 9 H.Lu/HKUST Value of Serializability: Data Settings: accounts( number, branchnum, balance); create clustered index c on accounts(number);  100000 rows  Cold buffer; same buffer size on all systems.  Row level locking  Isolation level (SERIALIZABLE or READ COMMITTED)  SQL Server 7, DB2 v7.1 and Oracle 8i on Windows 2000  Dual Xeon (550MHz,512Kb), 1Gb RAM, Internal RAID controller from Adaptec (80Mb), 4x18Gb drives (10000RPM), Windows 2000.

10. L07-Lock Tuning - 10 H.Lu/HKUST Value of Serializability: Transactions Concurrent Transactions:  T1: summation query [1 thread] select sum(balance) from accounts;  T2: swap balance between two account numbers (in order of scan to avoid deadlocks) [N threads] valX:=select balance from accounts where number=X; valY:=select balance from accounts where number=Y; update accounts set balance=valX where number=Y; update accounts set balance=valY where number=X;

11. L07-Lock Tuning - 11 H.Lu/HKUST Value of Serializability: Results  With SQL Server and DB2 the scan returns incorrect answers if the read committed isolation level is used (default setting)  With Oracle correct answers are returned (snapshot isolation), but beware of swapping

12. L07-Lock Tuning - 12 H.Lu/HKUST Cost of Serializability Because the update conflicts with the scan, correct answers are obtained at the cost of decreased concurrency and thus decreased throughput.

13. L07-Lock Tuning - 13 H.Lu/HKUST Chop Transactions  Long transactions  The more locks a transaction requests, the more likely it is that it will have to wait for some other transactions to release a lock  The longer a transaction T executes, the more time another transaction will have to wait if it is blocked by T  Chop a long transaction into a number of shorter transactions.

14. L07-Lock Tuning - 14 H.Lu/HKUST Chop Transactions: An Example  Example: A bank allows customers to withdraw up to $1000 a day. In the evening it does batch update, that is update customers account balance and then the branch balance. During evening, there are occasional credit checks  T1: update client’s account balance one by one followed by update to the appropriate branch balance.  T2: credit check: Checking a particular client’s balance  Possible solutions  Chop T1 into many small transactions, T i, each T i update the balance of one account (i) and related branch balance.  Further chop T i into two transactions, update client’s account balance and branch balance, respectively.

15. L07-Lock Tuning - 15 H.Lu/HKUST Chop Transactions: Correctness  In the previous example, add one more transaction that checks whether the sum of individual account balances tally with the branch balances.  How to chop?  Need to be careful to guarantee the correctness.  Will the transaction that concurrent with T cause T to produce and inconsistent state or to observe an inconsistent value if T is broken up?  Will the transactions that are concurrent with T be made inconsistent if T is broken up?  A formal solution is available. A rule of thumb:  If T access X and Y, and any other concurrent transactions T’ access at most one of X or Y and nothing else, then T can be divided into two transactions, accessing X and Y respectively.

16. L07-Lock Tuning - 16 H.Lu/HKUST Transactions with Human Interaction  A transaction that holds locks during a screen interaction is an invitation to bottlenecks  Airline Reservation 1. Retrieve list of seats available 2. Talk with customer regarding availability 3. Secure seat  Single transaction is intolerable, because each customer would hold lock on seats available.  Keep user interaction outside a transactional context  Problem: ask for a seat but then find it’s unavailable. More tolerable.

17. L07-Lock Tuning - 17 H.Lu/HKUST Granularity of Locking  Different granularities of locks  Record-level by default  page-level locking: prevent concurrent transactions from accessing/modifying all records on a page  page-level locking: prevent concurrent transactions from accessing/modifying all pages that are part of that table.  By default, recorded-level locking. Table level locking is useful:  Avoid blocking long transactions  Used to avoid deadlocks  Reduce locking overhead  Lock escalation  Automatically upgrades row-level locks into a single page-level locking when the number of row-level locks reaches  Guidelines  Long transactions should use table-level locking, and short transactions should use record locks to enhance concurrency

18. L07-Lock Tuning - 18 H.Lu/HKUST Record vs. Table Locking : Data Settings: accounts( number, branchnum, balance); create clustered index c on accounts(number);  100000 rows  Cold buffer  SQL Server 7, DB2 v7.1 and Oracle 8i on Windows 2000  No lock escalation on Oracle; Parameter set so that there is no lock escalation on DB2; no control on SQL Server.  Dual Xeon (550MHz,512Kb), 1Gb RAM, Internal RAID controller from Adaptec (80Mb), 4x18Gb drives (10000RPM), Windows 2000. No Concurrent Transactions:  Update [10 000 updates] update accounts set balance = Val;  Insert [10 000 transactions], e.g. typical one: insert into accounts values(664366,72255,2296.12);

19. L07-Lock Tuning - 19 H.Lu/HKUST Record vs. Table Locking : Results Row locking is barely more expensive than table locking because recovery overhead is higher than row locking overhead  Exception is updates on DB2 where table locking is distinctly less expensive than row locking.

20. L07-Lock Tuning - 20 H.Lu/HKUST Control Locking Granularity  Explicit control of the granularity  Setting the escalation point  Set the threshold high enough so that in an online environment of relatively short transactions, escalation will never take place  Size of lock table  Small size of lock table will cause the system to escalate the lock granularity even transactions are short

21. L07-Lock Tuning - 21 H.Lu/HKUST Lock Table Implementation A Lock info for A A H Syslockinfo in SQL Server 7

22. L07-Lock Tuning - 22 H.Lu/HKUST Logical Bottleneck: Sequential Key Generation  Consider an application in which one needs a sequential number to act as a key in a table, e.g. invoice numbers for bills.  Ad hoc approach: a separate table holding the last invoice number. Fetch and update that number on each insert transaction.  Counter approach: use facility such as Sequence (Oracle)/Identity(MSSQL).

23. L07-Lock Tuning - 23 H.Lu/HKUST Counter Facility: Data Settings:  default isolation level: READ COMMITTED; Empty tables  Dual Xeon (550MHz,512Kb), 1Gb RAM, Internal RAID controller from Adaptec (80Mb), 4x18Gb drives (10000RPM), Windows 2000. accounts( number, branchnum, balance); create clustered index c on accounts(number); counter ( nextkey ); insert into counter values (1);

24. L07-Lock Tuning - 24 H.Lu/HKUST Counter Facility: Transactions No Concurrent Transactions:  System [100 000 inserts, N threads] SQL Server 7 (uses Identity column) insert into accounts values (94496,2789); Oracle 8i insert into accounts values (seq.nextval,94496,2789);  Ad-hoc [100 000 inserts, N threads] begin transaction NextKey:=select nextkey from counter; update counter set nextkey = NextKey+1; commit transaction begin transaction insert into accounts values(NextKey,?,?); commit transaction

25. L07-Lock Tuning - 25 H.Lu/HKUST Avoid Bottlenecks: Counters  System generated counter (system) much better than a counter managed as an attribute value within a table (ad hoc).  The Oracle counter can become a bottleneck if every update is logged to disk, but caching many counter numbers is possible.  Counters may miss ids.

26. L07-Lock Tuning - 26 H.Lu/HKUST Insertion Points: Transactions No Concurrent Transactions:  Sequential [100 000 inserts, N threads] Insertions into account table with clustered index on ssnum Data is sorted on ssnum Single insertion point  Non Sequential [100 000 inserts, N threads] Insertions into account table with clustered index on ssnum Data is not sorted (uniform distribution) 100 000 insertion points  Hashing Key [100 000 inserts, N threads] Insertions into account table with extra attribute att with clustered index on (ssnum, att) Extra attribute attribute contains hash key (1021 possible values) 1021 insertion points

27. L07-Lock Tuning - 27 H.Lu/HKUST Insertion Points  Page locking: single insertion point is a source of contention (sequential key with clustered index, or heap)  DB2 v7.1 and Oracle 8i do not support page locking.  Row locking: No contention between successive insertions.

28. L07-Lock Tuning - 28 H.Lu/HKUST Summary –Physical DB Design and Tuning  Brief introduction to a few major issues in physical database design and tuning  Index selection  Tuning relational schema (denormalization, horizontal and vertical partitioning)  Tuning relational queries  Lock related tuning  There are other issues in tuning which are not covered in the class