1. Query Transformations
They Don’t Call it the Optimizer for Nothing
Steve Catmull
UTOUG Fall Symposium 2012

2. Agenda What is a Query Transformation
The Case for Query Transformations
Example Transformations
Summary

3. About the Examples Transformations are typically cost sensitive.
I will explain the idea behind the transformation
I will show you what the pattern looks like in the explain plan.
Version:

4. What is a Transformation
They are designed to not effect your query results.
The Optimizer thinks there is a better way to execute a query by rewriting it.
Sometimes the rewrite is significant
It is often very clever
Mathematically sound
Adds/Removes/Changes operations
ALTER TABLE OE.PRODUCT_DESCRIPTIONS
MODIFY(TRANSLATED_NAME NULL)

5. The Case for Transformation
Oracle presented transformation techniques at VLDB in Their tests showed significant improvement.
Especially in DSS that execute complex SQL. It’s important to note that if your queries are “select * from scott.emp where empno = 2 there will not be much benefit because there are not query transformations that address simple queries such as that.

6. Types of Transformation
Cost-Based
The proposed transformation is added to the set of candidate plans it considers.
Those plans are costed and one is selected.
Rule-Based
When certain conditions exist, the optimization is always desirable.
ORDER BY Elimination
Join Elimination
Source: Cost-Based Query Transformation in Oracle, VLDB 2006, Oracle USA Presentation

7. Rule Based Transformations

8. ORDER BY Elimination
Sort Aggregate is a bit misleading. If it were to sort it would say SORT ORDER BY

9. Join Elimination
Why join two tables if you don’t have to?

10. Join Elimination Change query to return only customer columns.
Why join to COUNTRIES?
INNER JOIN would be eliminated to IF the foreign key constraint is validated.

11. Join Elimination DDL
Primary / foreign key relationship between the tables
ENABALED / VALIDATE
DISABLED / RELY
Single column key
Source:

12. Cost-Based Transformations

13. Subquery Unnesting
Whoa! Where did those 2 NESTED LOOP joins come from?

14. Subquery Pseudo Code For every row in CUSTOMERS
Probe into SALES for sales by that customer ID who had >=2 quantity in the transaction.
We will read every row from SH.CUSTOMERS which is 55,000 and then notice it is probing into sales. Notice there is no more join which is how we wrote this query.
NOTE: Oracle is estimating this will take 29,195 seconds (~10 hours) instead of the 9 seconds when we threw in the hint to not do the transformation.
How do the performance of these two plans compare when executed on my test machine? Let’s find out.

15. This is a graph that compares the two. Please note a couple of things:
1st. This graph uses a logarithmic scale on the y axis because of the large variation in numbers. Otherwise. All you would see if one big bar and 7 other small indistinguishable ones.
So the way you read this is that every gridline on the y-axis represents an order of magnitude or 10x more. Let’s look at logical reads. Because there are over 3 gridlines between the results the one on the left is over 3 times as many (1600 versus 2.3 million).
Secondly. The one that looks like it is negative really is not. On a logarithmic scale the starting point is 1 versus 0. So, that one result is between 0 and 1 seconds.
With that out of the way, what do we learn from this chart.
The unnest query transformation (switch to nested loop and hit SALES first) finishes about 15-20x faster in execution time, uses well under 1/10th the CPU, has about the same amount of physical reads but over 1200x the number of physical reads.
This is a case where the performance gains are fantastic.

16. Un-Nesting Requirements
No HAVING in subquery
Subquery must be during or before WHERE clause in order of operations. ….

17. OR Expansion
How do you get Oracle to use 2 different normal indexes to access the same table. You expand the OR into separate queries and concatenate them together. CONCATENATION is semantically equivalent to UNION ALL.
Source:

18. OR Expansion Notes Sensitive to cardinalities / statistics
Enables two different NORMAL indexes on the same table to be accessed in the same SQL statement. Fancy trick.
Sightings in the wild depend a lot of your data and query patterns.

19. Complex View Merging
Source:

20. Complex View Merging Fail
Notice here that because we group on CHANNEL_ID, you would think that max(channel_id) = channel_id in the inner query. But what about null. It’s like the movie, What About Bob. No matter where you go there it is. The NULL. It turns out that after playing with this on for a while that even if you have the column marked not NULL or add a NOT NULL predicate it still will not perform the merging. There may be some logic I missed or this is just an optimization that they have chosen not to implement.

21. Predicate Pushdown

22. Join Predicate Pushdown
Opens up joining to a view using nested loop index methods.
“Normally, a view cannot be joined with an index-based nested loop join”
Source:

23. View Join Plan
Executes view independent of results of CUSTOMERS table.

24. View Join Plan (Pushdown)
Each row in CUSTOMERS, send the CUST_ID to step 6 and execute view.

25. Pushdown Takeaways Tight correlation with nested loops.
Let it happen naturally.
Errant cardinality estimates may cause this to be used. Ways to correct
Use hint
Simplify query
More clever than it looks.
Shows up frequently in DSS systems.

26. Coalescing Subqueries (ST)

27. Coalescing Subqueries Plan
Transformed 2 exists into a single OR.
Only 1 hit against CUSTOMERS.
Un-nested query into a hash join
Added a predicate from constraints

28. Materialized View Rewrite
You have users often group on common columns on a large table.

29. MV Rewrite (baseline)
select prod_id, promo_id, count(distinct cust_id) AS cust_id_cnt, sum(amount_sold) AS sold_amt from sh.sales group by prod_id, promo_id

30. Building an MV
If we build a materialized view to help speed up this query…

31. MV Rewrite (MVScript)
CREATE MATERIALIZED VIEW SH.SALES_SMRY1_MV BUILD IMMEDIATE REFRESH COMPLETE ENABLE QUERY REWRITE AS select prod_id, promo_id, count(distinct cust_id) AS cust_id_cnt, sum(amount_sold) AS sold_amt from sh.sales group by prod_id, promo_id

32. MV Rewrite – Another Query`

33. MV Rewrite Requirements

34. Temp Transformation

35. A Future Glimpse (uncorrelated subsumed subquery)
At 2009 VLDB, Oracle submitted a technique called uncorrelated subsumed subquery.
I have not seen this in the wild .

36. Known Transformation List
Name
1stIn
Description
Sort Elimination
10g
Eliminate sorts that do not impact output or logic.
Join Elimination
Does not join if data from table is not needed.
Subquery Unnesting 9i
OR Expansion 8i
Complex View Merging
Predicate Pushdown
Move WHERE expressions into a query block that is executed sooner (nested query block).
Join Predicate Pushdown
11g
Subquery Coalescing
Materialized View Rewrite
If a consistent answer can be given by a smaller more efficient materialized view, rewrite the query to use it.
Subquery Factoring (Temp Transformation)
When a query block is used more than once, dump it to a temp table and rewrite the query to use it.
Uncorrelated Subsumed Subquery ??

37. Summary Oracle is adding more and more transformation rules
For complex DSS environments, transformers are you friend. It doesn’t mean you don’t fight.
Constraints can enable some transformations even when disabled and not validated.