Optimization Exercises. Question 1 How do you think the following query should be computed? What indexes would you suggest to use? SELECT E.ename, D.mgr.

Slides:



Advertisements
Similar presentations
Tuning: overview Rewrite SQL (Leccotech)Leccotech Create Index Redefine Main memory structures (SGA in Oracle) Change the Block Size Materialized Views,
Advertisements

Database Management Systems, R. Ramakrishnan and J. Gehrke1 Evaluation of Relational Operations Chapter 12, Part A.
TURKISH STATISTICAL INSTITUTE 1 /34 SQL FUNDEMANTALS (Muscat, Oman)
1 Advanced SQL Queries. 2 Example Tables Used Reserves sidbidday /10/04 11/12/04 Sailors sidsnameratingage Dustin Lubber Rusty.
Evaluation of Relational Operators CS634 Lecture 11, Mar Slides based on “Database Management Systems” 3 rd ed, Ramakrishnan and Gehrke.
Database Management Systems 3ed, R. Ramakrishnan and Johannes Gehrke1 Evaluation of Relational Operations: Other Techniques Chapter 14, Part B.
Database Management Systems, R. Ramakrishnan and Johannes Gehrke1 Evaluation of Relational Operations: Other Techniques Chapter 12, Part B.
Database Management Systems, R. Ramakrishnan and Johannes Gehrke1 Evaluation of Relational Operations: Other Techniques Chapter 12, Part B.
ICS 624 Spring 2011 Overview of DB & IR Asst. Prof. Lipyeow Lim Information & Computer Science Department University of Hawaii at Manoa 1/12/20111Lipyeow.
Query Evaluation. An SQL query and its RA equiv. Employees (sin INT, ename VARCHAR(20), rating INT, age REAL) Maintenances (sin INT, planeId INT, day.
SPRING 2004CENG 3521 Query Evaluation Chapters 12, 14.
1 King Saud University College of Computer & Information Sciences IS 335 Database Management System Lecture 6 Query Processing and Optimization (Practice)
Agenda Overview of the optimizer How SQL is executed Identifying statements that need tuning Explain Plan Modifying the plan.
1 Implementation of Relational Operations Module 5, Lecture 1.
DB performance tuning using indexes Section 8.5 and Chapters 20 (Raghu)
1 Physical Database Design and Tuning Module 5, Lecture 3.
1 SQL (Simple Query Language). 2 Query Components A query can contain the following clauses –select –from –where –group by –having –order by Only select.
1 Optimization Recap and examples. 2 Optimization introduction For every SQL expression, there are many possible ways of implementation. The different.
David Konopnicki Choosing Access Path ä The basic methods. ä The access paths and when they are available. ä How the optimizer chooses among the.
CSCI 5708: Query Processing I Pusheng Zhang University of Minnesota Feb 3, 2004.
1 Optimization - Selection. 2 The Selection Operation Table: Reserves(sid, bid, day, agent) A page (block) can hold 100 Reserves tuples There are 1,000.
Oracle Optimizer. Types of Optimizers There are different modes for the optimizer RULE: Rule-based optimizer (RBO) –Deprecated; not updated since 1994.
Query Optimization 3 Cost Estimation R&G, Chapters 12, 13, 14 Lecture 15.
Database Systems: Design, Implementation, and Management Eighth Edition Chapter 11 Database Performance Tuning and Query Optimization.
1 The Oracle Database System Querying the Data Database Course The Hebrew University of Jerusalem.
1 Evaluation of Relational Operations: Other Techniques Chapter 12, Part B.
CSCI 5708: Query Processing I Pusheng Zhang University of Minnesota Feb 3, 2004.
Evaluation of Relational Operations. Relational Operations v We will consider how to implement: – Selection ( ) Selects a subset of rows from relation.
Oracle Optimizer. Combining Output From Multiple Index Scans AND-EQUAL: –select * from sailors where sname = 'Jim' and rating = 10 Suppose we have 2 indexes:
AN INTRODUCTION TO EXECUTION PLAN OF QUERIES These slides have been adapted from a presentation originally made by ORACLE. The full set of original slides.
1 Implementation of Relational Operations: Joins.
Query Optimization, part 2 CS634 Lecture 13, Mar Slides based on “Database Management Systems” 3 rd ed, Ramakrishnan and Gehrke.
Lecture 8 Index Organized Tables Clusters Index compression
Chapter 6 Additional Database Objects
Oracle Database Administration Lecture 6 Indexes, Optimizer, Hints.
Physical Database Design & Performance. Optimizing for Query Performance For DBs with high retrieval traffic as compared to maintenance traffic, optimizing.
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke1 Overview of Implementing Relational Operators and Query Evaluation Chapter 12.
Ashwani Roy Understanding Graphical Execution Plans Level 200.
11-1 Improve response time of interactive programs. Improve batch throughput. To ensure scalability of applications load vs. performance. Reduce system.
CPSC 404, Laks V.S. Lakshmanan1 Evaluation of Relational Operations: Other Operations Chapter 14 Ramakrishnan & Gehrke (Sections ; )
Relational Operator Evaluation. Overview Index Nested Loops Join If there is an index on the join column of one relation (say S), can make it the inner.
SQL Performance and Optimization l SQL Overview l Performance Tuning Process l SQL-Tuning –EXPLAIN PLANs –Tuning Tools –Optimizing Table Scans –Optimizing.
ICS 321 Fall 2011 The Relational Model of Data (i) Asst. Prof. Lipyeow Lim Information & Computer Science Department University of Hawaii at Manoa 8/29/20111Lipyeow.
Module 4 Database SQL Tuning Section 3 Application Performance.
The Oracle Database System. Connecting to the Database At the command line prompt, write: sqlplus In the beginning your password.
Physical Database Design Purpose- translate the logical description of data into the technical specifications for storing and retrieving data Goal - create.
Relational Operator Evaluation. Overview Application Programmer (e.g., business analyst, Data architect) Sophisticated Application Programmer (e.g.,
Chapter 5 Index and Clustering
More Optimization Exercises. Block Nested Loops Join Suppose there are B buffer pages Cost: M + ceil (M/(B-2))*N where –M is the number of pages of R.
Query Processing – Implementing Set Operations and Joins Chap. 19.
Implementation of Database Systems, Jarek Gryz1 Evaluation of Relational Operations Chapter 12, Part A.
Alon Levy 1 Relational Operations v We will consider how to implement: – Selection ( ) Selects a subset of rows from relation. – Projection ( ) Deletes.
1 Triggers. 2 PL/SQL reminder We presented PL/SQL- a Procedural extension to the SQL language. We reviewed the structure of an anonymous PL/SQL block:
Fan Qi Database Lab 1, com1 #01-08 CS3223 Tutorial 8.
Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke1 Evaluation of Relational Operations Chapter 14, Part A (Joins)
1 Indexes ► Sort data logically to improve the speed of searching and sorting operations. ► Provide rapid retrieval of specified rows from the table without.
1 CS122A: Introduction to Data Management Lecture #7 Relational Algebra I Instructor: Chen Li.
Cost Model and Estimating Result Sizes. מודל המחיר Cost Model בהרצאה הראנו איך לחשב את המחיר של כל שיטה (join) כדי לעשות זאת צריך לדעת את גודל היחסים,
Module 11: File Structure
Indexing Structures for Files and Physical Database Design
Choosing Access Path The basic methods.
Evaluation of Relational Operations
Evaluation of Relational Operations: Other Operations
Examples of Physical Query Plan Alternatives
Relational Operations
Implementation of Relational Operations
CS222P: Principles of Data Management Notes #13 Set operations, Aggregation, Query Plans Instructor: Chen Li.
Evaluation of Relational Operations: Other Techniques
Evaluation of Relational Operations: Other Techniques
Presentation transcript:

Optimization Exercises

Question 1 How do you think the following query should be computed? What indexes would you suggest to use? SELECT E.ename, D.mgr FROM Employees E, Departments D WHERE D.dname=‘Toy’ AND E.dno=D.dno

Question 2 How do you think the following query should be computed? What indexes would you suggest to use? SELECT E.ename, D.mgr FROM Employees E, Departments D WHERE D.dname=‘Toy’ AND E.dno=D.dno AND E.age = 25

Question 3 How do you think the following query should be computed? What indexes would you suggest to use? Must both tables be accessed? SELECT D.mgr FROM Departments D, Employees E WHERE D.dno=E.dno

Question 4 Emp(eid, age, sal, did) Dept(did, projid, budget, status) Proj(projid, code, report) Length of tuples, Number of tuples –Emp: 20 bytes, tuples –Dept: 40 bytes, 5000 tuples –Proj: 2000 bytes, 1000 tuples Pages contain 4000 bytes; 12 buffer pages

Question 4 (cont) Consider the queries: –find employees with age = 30 –find projects with code = 20 Assume that there are the same number of qualifying tuples in both. For which query is a clustered index more important?

Question 4 (cont) Consider the query: find employees with age > 30 Assume that there is an unclustered index on age. Suppose that there are N tuples for which age > 30 For which values of N is it cheaper to do a sequential scan?

Database Tuning

Problem: Make database run efficiently 80/20 Rule: 80% of the time, the database is running 20% of the queries/operations –find what is taking all the time, and tune these operations

Solutions Indexing –this can sometimes degrade performance. why? Tuning queries (using hints) Reorganization of tables; perhaps "denormalization" Changes in physical data storage

Denormalization Suppose you have tables: –emp(eid, ename, salary, did) –dept(did, budget, address, manager) Suppose you often ask queries which require finding the manager of an employee. You might consider changing the tables to: –emp(eid, ename, salary, did, manager) –dept(did, budget, address, manager) How will you ensure that the redundancy does not introduce errors into the database?

Creating Indexes Using Oracle

Creating an Index Syntax: create [bitmap] [unique] index index on table(column [,column]...) Notes: Oracle has two kinds of indexes –b-tree –bitmaps –No hash indexes! (Why?)

Automatically Created Indexes When you create a table with a –primary key constraint or –unique constraint a "unique" index is created automatically

Bitmap Indexes Appropriate for columns that may have very few possible values For each value c that appears in the column, a vector v of bits is created, with a 1 in v[i] if the i-th row has the value c Oracle can automatically compute the RowIDs for the bitmap entries during query processing (so they do not have to be explicitly stored)

Bitmap Indexes: Example create bitmap index rating_bit on Sailors(rating); Corresponding bitmaps: – 3: – 7: –10: SidSnameagerating 12Jim553 13John467 14Jane Sam373

Bitmap Indexes: Think About it How do you think that Oracle finds the RowIDs? What types of operations can be efficiently computed? –hint: aggregation functions –hint: logical conditions

Function-Based Indexes You can't use an index on sname for the following query: select * from Sailors where UPPER(sname) = 'SAM'; You can create a function-based index to speed up the query: create index upp_sname on Sailors(UPPER(sname));

Index-Organized Tables An index organized table keeps its data sorted by the primary key They store their data as if they were an index create table Sailors( sid number primary key, sname varchar2(30), age number, rating number) organization index;

Index-Organized Tables (2) What advantages does this have? What disadvantages? When to use?

Clustering Tables Together You can ask Oracle to store several tables close together on the disk This is useful if you usually query these tables together Note that the index created is actually a cluster index

Clustering Tables Together: Syntax create cluster sailor_reserves (X number); create table Sailors( sid number primary key, sname varchar2(30), age number, rating number) cluster sailor_reserves(sid); create index sailor_reserves_index on cluster sailor_reserves

Clustering Tables Together: Syntax (cont.) create table Reserves( sid number, bid number, day date, primary key(sid, bid, day) ) cluster sailor_reserves(sid); Note: We end up getting a clustered index!

Reserves sidbidday /7/ /10/ /12/96 Sailors sidsnameratingage 22Dustin Lubber Rusty Stored sidsnameratingagebidday 22Dustin /7/ /10/96 31Lubber Rusty /12/96

The Oracle Optimizer

Types of Optimizers There are different modes for the optimizer RULE: Rule-based optimizer; picks a plan based on syntactic rules CHOOSE: Cost-based optimizer; picks a plan based on statistics about database contents –Need to analyze the data in the database for cost- based optimizer ALL: Optimize for total throughput FIRST_ROWS: Optimize for first rows ALTER SESSION SET optimizer_mode = {choose|rule|first_rows(_n)|all_rows}

You can also use the following command: analyze table | index | compute statistics | estimate statistics [sample rows | percent] | delete statistics; analyze table Sailors estimate statistics sample 25 percent; Analyzing the Data

Viewing the Execution Plan First, you need a PLAN table. So, the first time that you want to see execution plans, run the command: Now, you can run: set autotrace on to see all

Viewing the Execution Plan (Option 2) Another option: explain plan set statement_id='test' for SELECT * FROM Sailors S WHERE sname='Joe'; explain plan set statement_id=‘ ’ for

Viewing the Execution Plan (Option 2, cont.) SELECT lpad(' ',2*Level)||Operation ||' '||Options||' '||Object_Name Execution_Path FROM Plan_Table WHERE Statement_Id='test' connect by prior ID = parent_id and statement_id = 'test' start with id = 1;

Operations that Access Tables TABLE ACCESS FULL: sequential table scan TABLE ACCESS BY ROWID: Given a rowid, you can access the table according to the rowid. –How do you get the rowid? From an index!

Operations that Use Indexes INDEX UNIQUE SCAN: Access of an index that is defined to be unique INDEX RANGE SCAN: Access of an index that is not unique or access of a unique index for a range of values

When are Indexes Used/Not Used? If you set an indexed column equal to a value, e.g., sname = 'Jim' If you specify a range of values for an indexed column, e.g., sname like 'J%' –sname like '%m': will not use an index –UPPER(sname) like 'J%' : will not use an index –sname is null: will not use an index, since null values are not stored in the index –sname is not null: will not use an index, since every value in the index would have to be accessed

When are Indexes Used? (cont) sname != 'Jim': Index will not be used. Why? MIN and MAX functions: Index will be used Equality of a column in a leading column of a multicolumn index. For example, suppose we have a multicolumn index on (sid, bid, day) –sid = 12: Can use the index –bid = 101: Cannot use the index Remember that indexes will only be used if they are selective

Is the table always accessed?

Combining Output From Multiple Index Scans AND-EQUAL: –select * from sailors where sname = 'Jim' and rating = 10 Suppose we have 2 indexes: sname, rating –Can search both and perform intersection Suppose we also have an index on (sname, rating) –How should the query be performed?

Combining Output From Multiple Index Scans CONCATENATION of scans: –select * from sailors where sname = 'Jim' or sname = 'Sam'

Operations that Manipulate Data Sets Up until now, all operations returned the rows as they were found There are operations that must find all rows before returning a single row –SORT ORDER BY: e.g., query with order by –SORT UNIQUE: e.g., query with distinct; query with minus, intersect or union (what about union all?) –SORT AGGREGATE, SORT GROUP BY: queries with aggregate functions or with group by functions

Operations that Manipulate Data Sets (cont.) Consider the query: –select sname from sailors union select bname from boats;

Operations that Manipulate Data Sets (cont.) Consider the query: –select sname from sailors minus select bname from boats; How do you think that Oracle implements intersect? union all?

Distinct What should Oracle do when processing the query (assuming that sid is the primary key): –select distinct sid from Sailors Does it matter if you write DISTINCT?

Join Methods Select * from Sailors, Reserves where Sailors.sid = Reserves.sid Oracle can use an index on Sailors.sid or on Reserves.sid (note that both will not be used) Join Methods: MERGE JOIN, NESTED LOOPS, HASH JOIN

Picking a Join Method MERGE JOIN: Performs badly when tables are of unequal size. Why? NESTED LOOPS: Perform well when tables are of unequal size (only used when there is an index on the join column of the inner table!) Appropriate if tables are small (so they fit in memory) or if results should be returned online. Why?

Hints

Hints You can give the optimizer hints about how to perform query evaluation Hints are written in /*+ */ right after the select Note: These are only hints. The oracle optimizer can choose to ignore your hints

Examples Select /*+ FULL (sailors) */ sid From sailors Where sname=‘Joe’; Select /*+ INDEX (sailors) */ sid From sailors Where sname=‘Joe’; Select /*INDEX (sailors s_ind) */ sid From sailors S, reserves R Where S.sid=R.sid AND sname=‘Joe’;

More Examples Select /*+ USE_NL (sailors) */ sid From sailors S, reserves R Where S.sid=R.sid AND sname=‘Joe’; Select /*+ USE_MERGE (sailors, reserves) */ sid From sailors S, reserves R Where S.sid=R.sid AND sname=‘Joe’; Select /*+ USE_HASH */ sid From sailors S, reserves R Where S.sid=R.sid AND sname=‘Joe’; inner table