cs44321 CS4432: Database Systems II Query Optimizer – Cost Based Optimization

CS parse convert apply laws estimate result sizes consider physical plans estimate costs pick best execute {P1,P2,…..} {(P1,C1),(P2,C2)...} Pi answer SQL query parse tree logical query plan “improved” l.q.p l.q.p. +sizes statistics

A Query (Evaluation) Plan  An extended relational algebra tree  Annotations at each node indicate:  access methods to use for each table.  implementation methods used for each relational operator. Reserves Sailors sid=sid bid=100 rating > 5 sname Reserves Sailors sid=sid bid=100 rating > 5 sname (Nested Loop Join) (On-the-fly)

How to cost a physical plan? We need estimated size of intermediate results – Chapter 16.4 Cost of each operator/algorithm – Chapter 15 Buffer available for the query cs44324

Result of cost-based optimization Good physical plan –Consider different join orderings –Consider different access methods for accessing the relations cs44325

6 Many alternate search algorithms are possible: 1.Exhaustive listing of all possible plans 2.Dynamic programming 3.Branch and bound 4.Greedy bottom-up plan construction NOTE: often only left-deep trees are being considered to keep the search space small. How to generate that ‘good’ Physical Plan?

Why left-deep trees? Fundamental decision in System R (IBM): –Only left-deep join trees are considered. – Left-deep trees can generate all fully pipelined plans. Intermediate results not written to temporary files. Not all left-deep trees are fully pipelined (e.g., SM join). B A C D B A C D C D B A

Left-deep trees differ in : –the order of relations, –the access method for each relation, and –the join method for each join. Number of left deep plans still exponential – n relations implies n! left- deep tree orderings Enumeration of Left-Deep Trees

Enumerated using N passes (if N relations joined): – Pass 1: Find best 1-relation plan for each relation. – Pass 2: Find best way to join result of each 1-relation plan (as outer) to another relation. (All 2-relation plans.) – Pass N: Find best way to join result of a (N-1)-relation plan (as outer) to the N’th relation. (All N-relation plans.) For each subset of relations, retain: – Cheapest plan overall, plus – Cheapest plan for each interesting order of the tuples. Enumeration of Left-Deep Trees BACD Pass 1 Pass 2 Pass 3

Enumeration Example Example , Also read Chapter If too many relations (Chapter ): –Dynamic Programming expensive if too many relations (say more than 6 relations). –Use greedy (faster algorithm, but may yield plans not as good as Dynamic Programming) cs443210

cs Operator Types Stateful versus stateless operators –Select is stateless –Join is stateful Blocking versus non-blocking operators –Select is non-blocking –Agg functions are blocking Pipelined versus non-pipelined operators –Select is pipelinable –What about Join ? (see next slide)

cs Join? Join : Revelation is that it depends on the implementation strategy chosen for an operator –Iteration-join : pipelinable –Merge-sort join : blocking –Index join : pipelinable –Hash join : blocking

cs Costing of a complete plan We went over an example query plan Important: first we classify operators as pipelined or not-pipelined If pipelined, then for stateless operators the IO cost is zero (for example, for Select or Project)

cs Costing of a Complete Query Plan What about a Select? How is it implemented? If in middle of plan, pipeline it (one tuple at a time iteration) If at leaf of plan, identify any potential index to use index-lookup to implement the Select If index available, cost of implemention the select operator is equal to cost of an index lookup

cs Costing of a complete plan Main idea: –Determine # of distinct values – V(R,a) –Determine physical impl. Strategies per operator –Then, compute IO costs for each operator –Then, sum up all costs. –Done.