1. 1 Database Administration and Performance Tuning --Introduction

2. 2 Course Structure Lecturers: –Dr. Yanghua Xiao ( 肖仰华 ) –Email: shawyh@fudan.edu.cn –Mobile phone: 13918452801 –http://gdm.fudan.edu.cn/GDMWiki/Wiki.jsp?pa ge=Dbtune

3. 3 Goals of the Course Appreciation of DBMS architecture Study the effect of various components on the performance of the systems Tune your application or system built upon DBMS Performance criteria for choosing a DBMS Tuning principles Troubleshooting techniques for chasing down performance problems

4. 4 Text/Reference Books: Dennis Shasha and Phillipe Bonnet: Database Tuning : Principles Experiments and Troubleshooting Techniques. Morgan Kaufmann Publishers. 2002 (released in June 2002). TEXT. Dennis Shasha: Database tuning : a principled approach. Prentice Hall, 1992. REFERENCE (a good reference if cannot get the text book) Database Management Systems, 3rd edition. Raghu Ramakrishnan & Johannes Gehrke, McGraw-Hill, 2002. Hector Garcia-Molina, Jeffrey D. Ullman, and Jennifer Widom: Database Systems -- The Complete Book. Prentice Hall, 2001. Sitansu S.Mittra, Database Performance Tuning and Optimization, Springer,2003 G. J. Vaidyanatha, K. Deshpande and J. Kostelac: Oracle Performance Tuning 101. Osborne/Mc-Graw-Hill. 2001. REFERENCE. Jim Gray (ed): The Benchmark handbook : for database and transaction processing systems. M. Kaufmann Publishers, 1991. REFERENCE. Richard J.Niemiec, Oracle Database 10g Performance Tuning tips and techniques, Mc Graw Hill Education,2009 牛新庄, DB2 数据库性能调整与优化, 清华大学出版社, 2009 胡百敬, 等, SQL Server 2005 Performance tuning, 电子工业出版社， 2008

6. 6 Copyright: Many slides belong to the tutorial: Database Tuning Principles, Experiments and Troubleshooting Techniques Dennis Shasha (shasha@cs.nyu.edu) Philippe Bonnet (bonnet@diku.dk) And lecture notes provided by Database Management Systems, 3 rd edition. Raghu Ramakrishnan & Johannes Gehrke McGraw-Hill, 2002. and some from the web …bonnet@diku.dk

7. What is Database Tuning Database Tuning is the activity of making a database application run more quickly. “More quickly” –usually means higher throughput, –though it may mean shorter response time for time-critical applications A 5% improvement is significant. 7

8. Why Database Tuning? Troubleshooting: –Make managers and users happy given an application and a DBMS Capacity Sizing: –Buy the right DBMS given application requirements Application Programming: –Coding your application for performance

9. Viewpoint about DB tuning Easy –Need not to struggle with complicated formulas and theorems Difficult –Need broad and deep understanding about DBMS Application OS hardware 9

10. Why is Database Tuning hard? The following query runs too slowly select * from R where R.a > 5; What do you do? PARSER OPTIMIZER EXECUTION SUBSYSTEM DISK SYBSYSTEM CACHE MANAGER LOGGING SUBSYSTEM LOCKING SUBSYSTEM NETWORK DISK/ CONTROLLER CPUMEMORY sqlcommands

11. Where to tune?-physical 11

12. Where to tune? -logical 12

13. Optimization at each level Conceptual level –E.g. normalization vs denormailization Internal level –E.g. tablespace management, index strategy External level –E.g. Query optimization, optimal access plan 13

14. How to tune? process Process for DB tuning –Localize the problem by starting at the location where the problem surfaces for the first time and trace it to the root –Fix the root cause and test to ensure that the problem does not reappear –Look for any adverse side effects caused by the fix 14

15. How to tune? Troubleshooting Methodology Troubleshooting Methodology: –Troubleshooting (what is happening?) –Hypothesis formulation What is the cause of the problem? Apply tuning principles to propose a fix –Hypothesis verification (experiments) 15

16. 16 Tuning Principles Think globally, fix locally Partitioning breaks bottlenecks (temporal and spatial) Start-up costs are high; running costs are low (disk transfer, cursors) Render unto server what is due unto server Be prepared for trade-offs (indexes and inserts)

17. Think globally, fix locally Think globally to find the root case or best solution for bottlenecks –For example : High disk I/O may be caused by: No index resulting into table scan Log and data file reside in the same disk Fail to increase the db buffer ….. Fix locally to minimize the adverse side-effect –E.g. Optimize the critical query 17

18. Partitioning breaks bottlenecks Bottleneck: the ONE part of the system limits the overall performance Partitioning Reducing the load on bottlenecks either by dividing the load over more resources or by spreading the load over time Partitioning in space (physical resource) –E.g. distribute account data to each branch Partitioning in logical space –E.g. lock contention on free list will cause the free list to be a bottleneck –Divide the free list into several pieces, each thread only lock one free list Partitioning in time –Long transactions may lock resource for a long time, use large portion of buffers, slowing down short transactions –Serialize long transactions when there are little short transactions 18

19. Side-effect of partitioning Suppose we need to query data across branch of a bank Merging data causes extra communication cost Lesson –When you find a bottleneck, first try to speed up that component; if that doesn’t work, then partition

20. 20 Tuning Principles Start-up costs are high; running costs are low –Start-up costs include Disk access –Reading 64k segment from disk is less than twice as expensive as reading 0.5k –Frequently scanned tables should be consecutively laid out on disk –Vertical partitioning tables with hundreds of columns Data transfer –Latency of sending message dominate the whole costs Packing data and send large chunk of data rather than small ones Query processing –Compile often executed queries System calls –Opening a call is expensive –Cache connections –Reduce the number of start-ups

21. 21 An example: Time = Seek Time + Rotational Delay + Transfer Time + Other Rule ofRandom I/O: Expensive Thumb Sequential I/O: Much less Ex:1 KB Block »Random I/O:  20 ms. »Sequential I/O:  1 ms.

22. Render onto server what is due onto Server Task allocation between the server and the application programs –Factors: Relative computing resources and load of client, application servers and data server –Should checking be done in the middle tier? Where information is located? –Trigger costs less cost than Polling Whether the database task involves the user interaction? –Interaction with screen or user takes long time –Separate the user interaction from a long transaction 22

23. Be prepared for trade-offs Performance vs cost/resource –Adding Buffer size need more RAM Query performance vs storage cost for index Query performance vs Update performance …… Keep in mind –You want speed, how much are you willing to pay? 23

24. 24 Tuning Mindset 1.Set reasonable performance tuning goals 2.Measure and document current performance 3.Identify current system performance bottleneck 4.Identify current OS bottleneck 5.Tune the required components eg: application, DB, I/O, contention, OS etc 6.Track and exercise change-control procedures 7.Measure and document current performance 8.Repeat step 3 through 7 until the goal is met

25. DateSubjectLiteratureMaterial 1Sep 14Introduction (slides )slides (RG 1.5-1.8, RG 4.2); SP 1 2Sep. 21Schema Refinement(slides)RG19-20,SP4.2database design project 3Sep. 28Data Storage(slides)HG 11.1-11.5 4Sep.30Storage Refinement(slides)SP 2.5, HG 11.6- 11.7 5Oct. 7Disk Organization(slides)HG 12 6Oct.14Index Tuning-Convential Index(slides)HG 13.1-13.2 7Oct.21Index Tuning-B+tree(slides)RG 10, HG 13.3Reading and Writing 8Oct.28External Sorting(slides),Index Tuning-Hash Index(slides), Performance Tuning(slides) HG13.4,RG 11, SP 3 9Nov. 4Operator Evaluation(slides)RG14Programming 10Nov. 11Query Optimization(slides)RG15 11Nov. 18Query Tuning(slides)SP 4.6-4.7DB Tuning Project, Example 1, Example 2, Example 3 12Nov. 25Overview of Transaction Management(slides), Lock Tuning(slides) HG 16-17, SP 2.2 13Dec. 2Log Tuning(slides)HG 18, SP 2.3 14Dec.9Troubleshooting(slides)SB 7 15Dec. 16API Tuning(slides)SB 5 16Dec. 23Exam