1. cs44321 CS4432: Database Systems II Query Optimizer – Cost Based Optimization

2. CS 44322 parse convert apply laws estimate result sizes consider physical plans estimate costs pick best execute {P1,P2,…..} {(P1,C1),(P2,C2)...} Pi answer SQL query parse tree logical query plan “improved” l.q.p l.q.p. +sizes statistics

3. A Query (Evaluation) Plan  An extended relational algebra tree  Annotations at each node indicate:  access methods to use for each table.  implementation methods used for each relational operator. Reserves Sailors sid=sid bid=100 rating > 5 sname Reserves Sailors sid=sid bid=100 rating > 5 sname (Nested Loop Join) (On-the-fly)

4. How to cost a physical plan? We need estimated size of intermediate results – Chapter 16.4 Cost of each operator/algorithm – Chapter 15 Buffer available for the query cs44324

5. Result of cost-based optimization Good physical plan –Consider different join orderings –Consider different access methods for accessing the relations cs44325

6. 6 Many alternate search algorithms are possible: 1.Exhaustive listing of all possible plans 2.Dynamic programming 3.Branch and bound 4.Greedy bottom-up plan construction NOTE: often only left-deep trees are being considered to keep the search space small. How to generate that ‘good’ Physical Plan?

7. Why left-deep trees? Fundamental decision in System R (IBM): –Only left-deep join trees are considered. – Left-deep trees can generate all fully pipelined plans. Intermediate results not written to temporary files. Not all left-deep trees are fully pipelined (e.g., SM join). B A C D B A C D C D B A

8. Left-deep trees differ in : –the order of relations, –the access method for each relation, and –the join method for each join. Number of left deep plans still exponential – n relations implies n! left- deep tree orderings Enumeration of Left-Deep Trees

9. Enumerated using N passes (if N relations joined): – Pass 1: Find best 1-relation plan for each relation. – Pass 2: Find best way to join result of each 1-relation plan (as outer) to another relation. (All 2-relation plans.) – Pass N: Find best way to join result of a (N-1)-relation plan (as outer) to the N’th relation. (All N-relation plans.) For each subset of relations, retain: – Cheapest plan overall, plus – Cheapest plan for each interesting order of the tuples. Enumeration of Left-Deep Trees BACD Pass 1 Pass 2 Pass 3

10. Enumeration Example Example 16.3.3, Also read Chapter 16.6.5 If too many relations (Chapter 16.6.6): –Dynamic Programming expensive if too many relations (say more than 6 relations). –Use greedy (faster algorithm, but may yield plans not as good as Dynamic Programming) cs443210

11. cs443211 Operator Types Stateful versus stateless operators –Select is stateless –Join is stateful Blocking versus non-blocking operators –Select is non-blocking –Agg functions are blocking Pipelined versus non-pipelined operators –Select is pipelinable –What about Join ? (see next slide)

12. cs443212 Join? Join : Revelation is that it depends on the implementation strategy chosen for an operator –Iteration-join : pipelinable –Merge-sort join : blocking –Index join : pipelinable –Hash join : blocking

13. cs443213 Costing of a complete plan We went over an example query plan Important: first we classify operators as pipelined or not-pipelined If pipelined, then for stateless operators the IO cost is zero (for example, for Select or Project)

14. cs443214 Costing of a Complete Query Plan What about a Select? How is it implemented? If in middle of plan, pipeline it (one tuple at a time iteration) If at leaf of plan, identify any potential index to use index-lookup to implement the Select If index available, cost of implemention the select operator is equal to cost of an index lookup

15. cs443215 Costing of a complete plan Main idea: –Determine # of distinct values – V(R,a) –Determine physical impl. Strategies per operator –Then, compute IO costs for each operator –Then, sum up all costs. –Done.