1. Database Tuning Principles, Experiments and Troubleshooting Techniques Baseado nos slides do tutorial com o mesmo nome da autoria de: Dennis Shasha (shasha@cs.nyu.edu) Philippe Bonnet (bonnet@diku.dk) E nos slides disponíveis no Web site do livro: http://www.databasetuning.org/

2. Introduction

3. AOBD 2007/08 H. Galhardas What is Database Tuning? Activity of making a database application run faster: – Faster means higher throughput (or response time) – Avoiding transactions that create bottlenecks or avoiding queries that run for hours unnecessarily is a must. – A 5% improvement is significant.

4. AOBD 2007/08 H. Galhardas Why Database Tuning? Troubleshooting: Make managers and users happy given an application and a DBMS Capacity Sizing: Buy the right DBMS given application requirements Application Programming: Coding your application for performance

5. AOBD 2007/08 H. Galhardas Why is Database Tuning hard? The following query runs too slowly select * from R where R.a > 5; What do you do? PARSER OPTIMIZER EXECUTION SUBSYSTEM DISK SYBSYSTEM CACHE MANAGER LOGGING SUBSYSTEM LOCKING SUBSYSTEM NETWORK DISK/ CONTROLLER CPUMEMORY sqlcommands

6. AOBD 2007/08 H. Galhardas Application Programmer (e.g., business analyst, Data architect) Sophisticated Application Programmer (e.g., SAP admin) DBA, Tuner Hardware [Processor(s), Disk(s), Memory] Operating System Concurrency ControlRecovery Storage Subsystem Indexes Query Processor Application

7. AOBD 2007/08 H. Galhardas Course Objectives Relevant notions concerning the internals of commercial DBMS 1. Tuning Principles Backed by experiments : How do tuning principles impact performances on my system? 2. Troubleshooting Methodology: Troubleshooting (what is happening?) Hypothesis formulation What is the cause of the problem? Apply tuning principles to propose a fix Hypothesis verification (experiments)

8. AOBD 2007/08 H. Galhardas Outline 1. Basic Principles 2. Tuning the guts 1. Concurrency control and recovery 2. OS configuration 3. HW modifications 3. Indexes 4. Relational Systems 1. Design of table schema, normalization, etc 2. Query tuning 5. Application Interface 6. Ecommerce Applications 7. Data warehouse Applications 8. Distributed Applications 9. Troubleshooting

9. AOBD 2007/08 H. Galhardas Tuning Principles 1. Think globally, fix locally 2. Partitioning breaks bottlenecks temporal and spatial 3. Start-up costs are high; running costs are low 4. Render unto server what is due unto server 5. Be prepared for trade-offs

10. AOBD 2007/08 H. Galhardas Think globally, fix locally Proper identification of problem; minimal intervention Understand the whole, including the application goals before taking a set of queries and find the indexes that speed them up. Example: High I/O, paging and processor utilization may be due to frequent query scans instead of using an index or log sharing a disk with some frequently accessed data.

11. AOBD 2007/08 H. Galhardas Partitioning breaks bottlenecks Technique for reducing the load on a certain component of the system either by dividing the load over more resources or by spreading the load over time Partitioning may not always solve bottleneck: First, try to speed up the component If it doesnt work, partition Example: Lock and resource contention among long and short transactions

12. AOBD 2007/08 H. Galhardas Start-up costs are high; running costs are low Obtain the effect you want with the fewest possible start-ups Examples: It is expensive to begin a read operation on a disk, but once it starts disk can deliver data at high speed. So, frequently scanned tables should be laid out consecutively on disk. Cost of parsing, semantic analysis, and selecting access paths for simple queries is significant So, often executed queries should be compiled

13. AOBD 2007/08 H. Galhardas Render unto server what is due unto server Important design question is the allocation of work between the DB system (server) and the application program (client) Depends on: Relative computing resources of client and server Where the relevant information is located Whether the DB task interacts with the screen

14. AOBD 2007/08 H. Galhardas Be prepared for trade-offs Increasing speed of application requires combination of memory, disk and computational resources Examples: Adding an index => speeds up critical query, but increases disk storage, and space in RAM Increasing RAM => Decreasing I/O, speed up query, but spending more money

15. AOBD 2007/08 H. Galhardas Outline Basic Principles Tuning the guts 1. Concurrency control and recovery 2. OS configuration 3. HW modifications 3. Indexes 4. Relational Systems 1. Design of table schema, normalization, etc 2. Query tuning 5. Application Interface 6. Ecommerce Applications 7. Data warehouse Applications 8. Distributed Applications 9. Troubleshooting