Advanced DatabaseTechniquesAdvanced DatabaseTechniques Optimisation & Performance Tuning University of Derby - School of Maths & Computing University of Derby 2000 The aim of all applications optimisation is to minimise some cost function - usually I/O or transaction time Database Server Database Administrator PERFORMANCE issues concerning query decomposition PERFORMANCE TUNING - INTRODUCTION contribute to

Advanced DatabaseTechniquesAdvanced DatabaseTechniques Optimisation & Performance Tuning University of Derby - School of Maths & Computing University of Derby 2000 SERVER PROCEDURE FOR SATISFYING QUERY database front-end SQL Query Scan, Validate & Parse Query Low-Level Data Requests Query Modifier (Permits, Views, Integrities) Query Optimiser Query Execution Data Manipulation Facility Precompiled Queries

Advanced DatabaseTechniquesAdvanced DatabaseTechniques Optimisation & Performance Tuning University of Derby - School of Maths & Computing University of Derby 2000 OPTIMISER Relation Descriptions Column Descriptions Secondary Indices Statistics Histograms Incoming Query Query Execution Plan THE QUERY OPTIMISER

Advanced DatabaseTechniquesAdvanced DatabaseTechniques Optimisation & Performance Tuning University of Derby - School of Maths & Computing University of Derby 2000 EXAMPLE JOINS 2 Tables - one of 100 rows - one of rows (of which only 50 rows are to be selected) Full Sort Merge Join Strategy Join Sort Possible QEP for Joining Two Tables Sort Proj-Rest Base Table Join selection of tuples

Advanced DatabaseTechniquesAdvanced DatabaseTechniques Optimisation & Performance Tuning University of Derby - School of Maths & Computing University of Derby 2000 Query Execution Plan Created to access the required data given the SQL statement at minimum cost Cost function is a combination of I/Os required, CPU cycles consumed, together with memory and sort-merge requirements QEP is defined and control is passed to the runtime supervisor - the data manager then invokes the buffer manager - which calls on the services of the lowest level operating system - deciding how to share resources amongst multiple users

Advanced DatabaseTechniquesAdvanced DatabaseTechniques Optimisation & Performance Tuning University of Derby - School of Maths & Computing University of Derby 2000 Implementation of the QEP - Indexing storage structure 5 different types of data organisation to be examined:- HEAPS INVERTED LIST ISAM (Indexed Sequential Access Method) B-tree (Binary Tree) Hashed

Advanced DatabaseTechniquesAdvanced DatabaseTechniques Optimisation & Performance Tuning University of Derby - School of Maths & Computing University of Derby 2000 HEAPS Unordered set of rows New data added to end of file No direct access to individual row Adding data very efficient - no need to re-order pages or indices Seems crude - why use it? - good for applications requiring large volumes of data entry on-line followed by over-night batch reporting - no ad hoc queries

Advanced DatabaseTechniquesAdvanced DatabaseTechniques Optimisation & Performance Tuning University of Derby - School of Maths & Computing University of Derby 2000 Inverted-Lists For every instance of the key value - store the RID of the records matching that value Inverted Index on road-name Allestree Close RID1,RID4,RID10,RID12,RID15 ….. Bentley Brook RID201,IRD202,RID250,RID251,RID254 …... Kedleston Road RID151,RID152,RID154,….. Example Table STREET_LIGHTS light_number,road_number,sector,road_name, control_type, wattage,etc. NB List is maintained in sorted order The index page is read into main storage, probably only one I/O needed. RIDs then used to locate data pages. Therefore, efficient data retrieval in query like:- SELECT*FROM STREET_LIGHTS WHERE road_name=Bently Brook

Advanced DatabaseTechniquesAdvanced DatabaseTechniques Optimisation & Performance Tuning University of Derby - School of Maths & Computing University of Derby 2000 ISAM (or VSAM for ABM DB2) Indexed Sequential Access Method is based on primary key only. Therefore, need to select a key which will serve most of the likely requests of the user. In practice a compound key is often used. Disadvantage - Static - overflow can rapidly increase. <= M > M < E < J <N < =BP >BP Allen Baker Bottorff Brubaker Cary Eagleton Arly INDEX PAGES DATA PAGES OVERFLOW PAGES

Advanced DatabaseTechniquesAdvanced DatabaseTechniques Optimisation & Performance Tuning University of Derby - School of Maths & Computing University of Derby 2000 B-Tree Leaf pages contain RIDs to the individual records. Data is returned in sorted order to the data buffer manager. Dynamic index - more balanced index than ISAM. More widely used. <=M > M < E <J<J <N INDEX PAGES DATA PAGES Allen Baker Bottorff Brubaker Cary Eagleton LEAF PAGES Allen Baker Bottorff Brubaker Cary Eagleton

Advanced DatabaseTechniquesAdvanced DatabaseTechniques Optimisation & Performance Tuning University of Derby - School of Maths & Computing University of Derby 2000 Hash In hashed files the key value is directly related one-to-one with a physical address by means of a mathematical formula. Address=f (key-value) Advantage no index need be used - no index I/O required -therefore fast access Disadvantage large amounts of storage space may remain unused Hash index may be a possible choice for an evenly distributed key value table.

Advanced DatabaseTechniquesAdvanced DatabaseTechniques Optimisation & Performance Tuning University of Derby - School of Maths & Computing University of Derby 2000 Data Storage Structures Data compressed is stored on disk in pages. A page is a unit of physical I/O typically containing many logical records each of which maps to a row in a database table. Each record is identified uniquely by a RID. free space record x 20 4 RID for a record x page no 20 offset to the location of a record pointer P P

Advanced DatabaseTechniquesAdvanced DatabaseTechniques Optimisation & Performance Tuning University of Derby - School of Maths & Computing University of Derby 2000 Database Re-organisation Utility Before and after of a single data page BEFORE AFTER 2% free space 5% free space Unload Sort by primary key Reload

Advanced DatabaseTechniquesAdvanced DatabaseTechniques Optimisation & Performance Tuning University of Derby - School of Maths & Computing University of Derby 2000 Parallel Processing Parallel processing has been described as : The key to high performance transaction and database processing Use the paper on Open Issues in Parallel Query Optimisation as a discussion document to test whether: Parallel Processing offers the possibility of vastly improved performance where the database is very large or can be conveniently partitioned across different disks to reduce the overheads of communication. Share-Nothing and Share-memory architectures. Load-balancing, disk access problems, communications overheads. Application applicability - especially in respect of Object-oriented databases. Look at the topics: