Getting the Right Cardinality Thomas Kyte

Why do we gather statistics?

“Cardinality”

“Wrong Plan => Wrong Cardinality”

“Wrong Cardinality => Wrong Plan”

Wrong Plan => Wrong Cardinality ops$tkyte%ORA11GR2> create table t 2 as select decode( mod(rownum,2), 0, 'N', 'Y' ) flag1, 3 decode( mod(rownum,2), 0, 'Y', 'N' ) flag2, a.* 4 from all_objects a 5 / Table created. ops$tkyte%ORA11GR2> create index t_idx on t(flag1,flag2); Index created. ops$tkyte%ORA11GR2> begin 2 dbms_stats.gather_table_stats 3 ( user, 'T', 4 method_opt=>'for all indexed columns size 254' ); 5 end; 6 / PL/SQL procedure successfully completed.

Wrong Plan => Wrong Cardinality ops$tkyte%ORA11GR2> select 'select * from t', num_rows 2 from user_tables where table_name = 'T' 3 union all 4 select 'select * from t where flag1 = "N"', num_rows/2 5 from user_tables where table_name = 'T' 6 union all 7 select 'select * from t where flag2 = "N"', num_rows/2 8 from user_tables where table_name = 'T' 9 union all 10 select 'select * from t where flag1 = "N" and flag2 = "N"', num_rows/2/2 11 from user_tables where table_name = 'T'; 'SELECT*FROMT' NUM_ROWS select * from t select * from t where flag1 = "N" select * from t where flag2 = "N" select * from t where flag1 = "N" and flag2 = "N"

Wrong Plan => Wrong Cardinality ops$tkyte%ORA11GR2> set autotrace traceonly explain ops$tkyte%ORA11GR2> select * from t where flag1='N'; Execution Plan Plan hash value: | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | | 0 | SELECT STATEMENT | | | 3635K| 301 (1)| 00:00:04 | |* 1 | TABLE ACCESS FULL| T | | 3635K| 301 (1)| 00:00:04 | Predicate Information (identified by operation id): filter("FLAG1"='N')

Wrong Plan => Wrong Cardinality ops$tkyte%ORA11GR2> select * from t where flag2='N'; Execution Plan Plan hash value: | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | | 0 | SELECT STATEMENT | | | 3608K| 301 (1)| 00:00:04 | |* 1 | TABLE ACCESS FULL| T | | 3608K| 301 (1)| 00:00:04 | Predicate Information (identified by operation id): filter("FLAG2"='N')

Wrong Plan => Wrong Cardinality ops$tkyte%ORA11GR2> select * from t where flag1='N' and flag2='N'; Execution Plan Plan hash value: | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | | 0 | SELECT STATEMENT | | | 1810K| 301 (1)| 00:00:04 | |* 1 | TABLE ACCESS FULL| T | | 1810K| 301 (1)| 00:00:04 | Predicate Information (identified by operation id): filter("FLAG2"='N' AND "FLAG1"='N')

Wrong Plan => Wrong Cardinality ops$tkyte%ORA11GR2> select /*+ gather_plan_statistics */ * 2 from t where flag1='N' and flag2='N'; no rows selected

Wrong Plan => Wrong Cardinality ops$tkyte%ORA11GR2> select * from table(dbms_xplan.display_cursor(null,null,'ALLSTATS LAST')); PLAN_TABLE_OUTPUT | Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | | 0 | SELECT STATEMENT | | 1 | | 0 |00:00:00.02 | 1080 | |* 1 | TABLE ACCESS FULL| T | 1 | | 0 |00:00:00.02 | 1080 | Predicate Information (identified by operation id): filter(("FLAG2"='N' AND "FLAG1"='N')) 19 rows selected.

Wrong Plan => Wrong Cardinality ops$tkyte%ORA11GR2> select /*+ dynamic_sampling(t 3) */ * from t where flag1='N' and flag2='N'; Execution Plan Plan hash value: | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | | 0 | SELECT STATEMENT | | 6 | 612 | 2 (0)| 00:00:01 | | 1 | TABLE ACCESS BY INDEX ROWID| T | 6 | 612 | 2 (0)| 00:00:01 | |* 2 | INDEX RANGE SCAN | T_IDX | 6 | | 1 (0)| 00:00:01 | Predicate Information (identified by operation id): access("FLAG1"='N' AND "FLAG2"='N') Note dynamic sampling used for this statement (level=2)

Wrong Plan => Wrong Cardinality SELECT /* OPT_DYN_SAMP */ /*+ ALL_ROWS IGNORE_WHERE_CLAUSE NO_PARALLEL(SAMPLESUB) opt_param('parallel_execution_enabled', 'false') NO_PARALLEL_INDEX(SAMPLESUB) NO_SQL_TUNE */ NVL(SUM(C1),:"SYS_B_00"), NVL(SUM(C2),:"SYS_B_01"), NVL(SUM(C3),:"SYS_B_02") FROM (SELECT /*+ IGNORE_WHERE_CLAUSE NO_PARALLEL("T") FULL("T") NO_PARALLEL_INDEX("T") */ :"SYS_B_03" AS C1, CASE WHEN "T"."FLAG1"= :"SYS_B_04" AND "T"."FLAG2"=:"SYS_B_05" THEN :"SYS_B_06" ELSE :"SYS_B_07" END AS C2, CASE WHEN "T"."FLAG2"=:"SYS_B_08" AND "T"."FLAG1"=:"SYS_B_09“ THEN :"SYS_B_10" ELSE :"SYS_B_11" END AS C3 FROM "OPS$TKYTE"."T" SAMPLE BLOCK (:"SYS_B_12", :"SYS_B_13") SEED (:"SYS_B_14") "T") SAMPLESUB

Small Change – but think about it… ops$tkyte%ORA11GR2> create table t 2 as 3 select substr(object_name, 1, 1 ) str, all_objects.* 4 from all_objects 5 order by dbms_random.random; Table created. ops$tkyte%ORA11GR2> create index t_idx on t(str,object_name); Index created. ops$tkyte%ORA11GR2> begin 2 dbms_stats.gather_table_stats 3 ( user, 'T', 4 method_opt => 'for all indexed columns size 254', 5 estimate_percent=>100 ); 6 end; 7 / PL/SQL procedure successfully completed.

ops$tkyte%ORA11GR2> select count(subobject_name) from t t1 where str = 'T'; … | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | | 0 | SELECT STATEMENT | | 1 | 19 | 296 (0)| 00:00:04 | | 1 | SORT AGGREGATE | | 1 | 19 | | | | 2 | TABLE ACCESS BY INDEX ROWID| T | 292 | 5548 | 296 (0)| 00:00:04 | |* 3 | INDEX RANGE SCAN | T_IDX | 292 | | 4 (0)| 00:00:01 | Small Change – but think about it…

ops$tkyte%ORA11GR2> insert into t 2 select 'T', all_objects.* 3 from all_objects 4 where rownum <= 1; 1 row created. ops$tkyte%ORA11GR2> begin 2 dbms_stats.gather_table_stats 3 ( user, 'T', 4 method_opt => 'for all indexed columns size 254', 5 estimate_percent=>100 ); 6 end; 7 / PL/SQL procedure successfully completed. Small Change – but think about it…

ops$tkyte%ORA11GR2> select count(subobject_name) from t t2 where str = 'T'; … | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | | 0 | SELECT STATEMENT | | 1 | 19 | 297 (1)| 00:00:04 | | 1 | SORT AGGREGATE | | 1 | 19 | | | |* 2 | TABLE ACCESS FULL| T | 293 | 5567 | 297 (1)| 00:00:04 | Small Change – but think about it…

Statistics Basics

Things Change over Time In 10g and before, it was very common to “program” our statistics gathering – Unique set of inputs for each and every table and index In 11g, you should as often as possible allow everything except the segment name and DOP to default – Why is this? However…

21 You are being WATCHED! ops$tkyte%ORA11GR2> create table t 2 as 3 select a.*, 4 case when rownum < then 1 6 else 99 7 end some_status 8 from all_objects a 9 / Table created.

22 You are being WATCHED! ops$tkyte%ORA11GR2> begin 2 dbms_stats.gather_table_stats(user,'T'); 3 end; 4 / PL/SQL procedure successfully completed.

23 You are being WATCHED! ops$tkyte%ORA11GR2> select histogram 2 from user_tab_cols 3 where table_name = 'T' 4 and column_name = 'SOME_STATUS'; HISTOGRAM NONE

24 You are being WATCHED! ops$tkyte%ORA11GR2> create index t_idx on t(some_status); Index created.

25 You are being WATCHED! ops$tkyte%ORA11GR2> select * from t where some_status = 99; Execution Plan Plan hash value: | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | | 0 | SELECT STATEMENT | | | 3526K| 300 (1)| 00:00:04 | |* 1 | TABLE ACCESS FULL| T | | 3526K| 300 (1)| 00:00:04 |

26 You are being WATCHED! ops$tkyte%ORA11GR2> begin 2 dbms_stats.gather_table_stats( user, 'T' ); 3 end; 4 / PL/SQL procedure successfully completed.

27 You are being WATCHED! ops$tkyte%ORA11GR2> select histogram 2 from user_tab_cols 3 where table_name = 'T' 4 and column_name = 'SOME_STATUS'; HISTOGRAM FREQUENCY

28 You are being WATCHED! ops$tkyte%ORA11GR2> select * from t where some_status = 1; Execution Plan Plan hash value: | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | 0 | SELECT STATEMENT | | 539 | | 10 (0)| 00:00 | 1 | TABLE ACCESS BY INDEX ROWID| T | 539 | | 10 (0)| 00:00 |* 2 | INDEX RANGE SCAN | T_IDX | 539 | | 2 (0)| 00:

29 You are being WATCHED! ops$tkyte%ORA11GR2> select * from t where some_status = 99; Execution Plan Plan hash value: | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | | 0 | SELECT STATEMENT | | | 7000K| 300 (1)| 00:00:04 | |* 1 | TABLE ACCESS FULL| T | | 7000K| 300 (1)| 00:00:04 |

30 Index Statistics – little known fact ops$tkyte%ORA11GR2> create table t 2 as 3 select * 4 from all_objects 5 / Table created. ops$tkyte%ORA11GR2> create index t_idx on t(object_id); Index created. ops$tkyte%ORA11GR2> select num_rows, sample_size, last_analyzed 2 from user_indexes 3 where index_name = 'T_IDX'; NUM_ROWS SAMPLE_SIZE LAST_ANAL JUL-12

Other Ways to Get Statistics

Other ways to get statistics DBMS_STATS.SET_xxx – If you know the statistics, just tell us DBMS_STATS.COPY_xxx – If you have something representative, just use to them to start Dynamic Sampling… SQL Profiles Extended Statistics Cardinality Feedback Cardinality Hint

Dynamic Sampling archive/2009/09-jan/o19asktom html archive/2009/09-jan/o19asktom html – Google: site: kyte dynamic Already demonstrated Levels – 2 – sample any unanalyzed table during hard parse – 3 – same as 2 but add “sample during guess” for single column guesses – 4 – same as 3 but add “sample during guess” for multi-column guesses

SQL Profiles Analyzing a query Stores better estimated cardinalities – Gets these by using your workload against your data – Important to be done in the right environment – Are just statistics really, so yes, they can go stale Applied at hard parse time

SQL Profiles create or replace procedure p 2 as 3 cursor c1 4 is 5 select object_id, object_name 6 from sqlprof 7 order by object_id; 9 l_object_id sqlprof.object_id%type; 10 l_object_name sqlprof.object_name%type; 11 begin 12 open c1; 13 for i in loop 15 fetch c1 into l_object_id, l_object_name; 16 exit when c1%notfound; end loop; 19 end; 20 / Procedure created.

SQL Profiles SELECT OBJECT_ID, OBJECT_NAME FROM SQLPROF ORDER BY OBJECT_ID call count cpu elapsed query current rows Parse Execute Fetch total Misses in library cache during parse: 1 Optimizer mode: ALL_ROWS Parsing user id: 410 (recursive depth: 1) Rows Row Source Operation SORT ORDER BY (cr=659 pr=0 pw=0 time= us) TABLE ACCESS FULL SQLPROF (cr=659 pr=0 pw=0 time=47604 us)

SQL Profiles declare 2 l_sql_id v$sql.sql_id%type; 3 begin 4 select sql_id into l_sql_id 5 from v$sql 6 where sql_text = 'SELECT OBJECT_ID, OBJECT_NAME FROM SQLPROF ORDER BY OBJECT_ID'; 7 dbms_output.put_line( 8 sys.dbms_sqltune.create_tuning_task 9 ( sql_id => l_sql_id, 10 task_name => 'sqlprof_query' ) || ' in place...' ); 11 dbms_sqltune.execute_tuning_task 12 ( task_name => 'sqlprof_query' ); 13 end; 14 / PL/SQL procedure successfully completed.

SQL Profiles SELECT DBMS_SQLTUNE.REPORT_TUNING_TASK( 'sqlprof_query') 2 FROM DUAL; DBMS_SQLTUNE.REPORT_TUNING_TASK('SQLPROF_QUERY') GENERAL INFORMATION SECTION … SQL ID : 3zfpa86satsm3 SQL Text: SELECT OBJECT_ID, OBJECT_NAME FROM SQLPROF ORDER BY OBJECT_ID FINDINGS SECTION (1 finding) SQL Profile Finding (see explain plans section below) A potentially better execution plan was found for this statement. Recommendation (estimated benefit: 99.45%) Consider accepting the recommended SQL profile. execute :profile_name := dbms_sqltune.accept_sql_profile(task_name => 'sqlprof_query')

SQL Profiles 1- Original With Adjusted Cost Plan hash value: | Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time| | 0 | SELECT STATEMENT | | | 1391K| | 546 (3)| 00:00:07 | | 1 | SORT ORDER BY | | | 1391K| 3736K| 546 (3)| 00:00:07 | | 2 | TABLE ACCESS FULL| SQLPROF | | 1391K| | 151 (2)| 00:00:02 | Using SQL Profile Plan hash value: | Id | Operation | Name | Rows | Bytes | Cost (%CPU)|Time | | 0 | SELECT STATEMENT | | 10 | 300 | 3 (0)|00:00:01 | | 1 | TABLE ACCESS BY INDEX ROWID| SQLPROF | | 1391K| 3 (0)|00:00:01 | | 2 | INDEX FULL SCAN | SQLPROF_PK | 10 | | 2 (0)|00:00:01 |

Extended Statistics isticshttps://blogs.oracle.com/optimizer/entry/extended_stat istics In fact, should be mandatory reading! Create statistics on sets of columns (correlated columns) Create statistics on expressions (functions)

41 Extended Statistics ops$tkyte%ORA11GR2> create table t 2 as 3 select * 4 from all_objects; Table created. ops$tkyte%ORA11GR2> create index t_idx on t(owner,object_type); Index created. ops$tkyte%ORA11GR2> exec dbms_stats.gather_table_stats( user, 'T', method_opt=>'for all columns size 254' ); PL/SQL procedure successfully completed.

42 Extended Statistics ops$tkyte%ORA11GR2> set autotrace traceonly explain ops$tkyte%ORA11GR2> select * from t where owner = 'PUBLIC' and object_type = 'JAVA CLASS'; Execution Plan Plan hash value: | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | | 0 | SELECT STATEMENT | | 9015 | 853K| 290 (1)| 00:00:04 | |* 1 | TABLE ACCESS FULL| T | 9015 | 853K| 290 (1)| 00:00:04 | Predicate Information (identified by operation id): filter("OBJECT_TYPE"='JAVA CLASS' AND "OWNER"='PUBLIC')

43 Extended Statistics ops$tkyte%ORA11GR2> select dbms_stats.create_extended_stats( user, 'T', '(owner,object_type)' ) x from dual; X SYS_STUXJ8K0YTS_5QD1O0PEA514IY ops$tkyte%ORA11GR2> exec dbms_stats.gather_table_stats( user, 'T', method_opt=>'for all columns size 254' ); PL/SQL procedure successfully completed.

44 Extended Statistics ops$tkyte%ORA11GR2> set autotrace traceonly explain ops$tkyte%ORA11GR2> select * from t where owner = 'PUBLIC' and object_type = 'JAVA CLASS'; Execution Plan Plan hash value: | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | | 0 | SELECT STATEMENT | | 7 | 763 | 2 (0)| 00:00:01 | | 1 | TABLE ACCESS BY INDEX ROWID| T | 7 | 763 | 2 (0)| 00:00:01 | |* 2 | INDEX RANGE SCAN | T_IDX | 7 | | 1 (0)| 00:00:01 | Predicate Information (identified by operation id): access("OWNER"='PUBLIC' AND "OBJECT_TYPE"='JAVA CLASS')

Extended Statistics See ow_what_extended_statistics_are_needed_for_a_giv en_workload for how to have these column groups automatically built for you ow_what_extended_statistics_are_needed_for_a_giv en_workload Extended statistics on functions done exactly the same

46 Extended Statistics ops$tkyte%ORA11GR2> create table t 2 as 3 select rownum a, -rownum b, all_objects.* 4 from all_objects; Table created. ops$tkyte%ORA11GR2> exec dbms_stats.gather_table_stats( user, 'T', method_opt=>'for all columns size 254' ); PL/SQL procedure successfully completed. ops$tkyte%ORA11GR2> select 0.05 * count(*) from t; 0.05*COUNT(*)

47 Extended Statistics ops$tkyte%ORA11GR2> set autotrace traceonly explain ops$tkyte%ORA11GR2> select * from t where (a+b)/2 > 50; Execution Plan Plan hash value: | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | | 0 | SELECT STATEMENT | | 3635 | 383K| 321 (1)| 00:00:04 | |* 1 | TABLE ACCESS FULL| T | 3635 | 383K| 321 (1)| 00:00:04 | Predicate Information (identified by operation id): filter(("A"+"B")/2>50)

48 Extended Statistics ops$tkyte%ORA11GR2> set autotrace off ops$tkyte%ORA11GR2> ops$tkyte%ORA11GR2> select dbms_stats.create_extended_stats( user, 'T', '((a+b)/2)' ) x from dual; X SYS_STUS9G#61NMFNG0T#HK9W8062Y ops$tkyte%ORA11GR2> exec dbms_stats.gather_table_stats( user, 'T', method_opt=>'for all columns size 254' ); PL/SQL procedure successfully completed.

49 Extended Statistics ops$tkyte%ORA11GR2> set autotrace traceonly explain ops$tkyte%ORA11GR2> select * from t where (a+b)/2 > 50; Execution Plan Plan hash value: | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | | 0 | SELECT STATEMENT | | 1 | 110 | 320 (1)| 00:00:04 | |* 1 | TABLE ACCESS FULL| T | 1 | 110 | 320 (1)| 00:00:04 | Predicate Information (identified by operation id): filter(("A"+"B")/2>50)

50 Cardinality Feedback ops$tkyte%ORA11GR2> create or replace type str2tblType as table of varchar2(30) 2 / Type created. ops$tkyte%ORA11GR2> create or replace function str2tbl( p_str in varchar2, p_delim in varchar2 default ',' ) return str2tblType 2 PIPELINED 3 as 4 l_str long default p_str || p_delim; 5 l_n number; 6 begin 7 loop 8 l_n := instr( l_str, p_delim ); 9 exit when (nvl(l_n,0) = 0); 10 pipe row( ltrim(rtrim(substr(l_str,1,l_n-1))) ); 11 l_str := substr( l_str, l_n+1 ); 12 end loop; 13 return; 14 end; 15 / Function created.

51 Cardinality Feedback ops$tkyte%ORA11GR2> var in_list varchar2(255) ops$tkyte%ORA11GR2> exec :in_list := 'DBMS_PIPE,DBMS_OUTPUT,UTL_FILE'; PL/SQL procedure successfully completed. ops$tkyte%ORA11GR2> ops$tkyte%ORA11GR2> with data as 2 ( select /*+ materialize */ distinct * 3 from TABLE(cast( str2tbl( :in_list ) as str2tblType) ) t 4 ) 5 select * from data; COLUMN_VALUE DBMS_PIPE DBMS_OUTPUT UTL_FILE

52 Cardinality Feedback | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| | 0 | SELECT STATEMENT | | | | 33 (100)| | 1 | TEMP TABLE TRANSFORMATION | | | | | | 2 | LOAD AS SELECT | | | | | | 3 | HASH UNIQUE | | 8168 | | 30 (4)| | 4 | COLLECTION ITERATOR PICKLER FETCH| STR2TBL | 8168 | | 29 (0)| | 5 | VIEW | | 8168 | 135K| 3 (0)| | 6 | TABLE ACCESS FULL | SYS_TEMP_0FD9D660F_52718F2 | 8168 | | 3 (0)| rows selected.

53 Cardinality Feedback | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| | 0 | SELECT STATEMENT | | | | 33 (100)| | 1 | TEMP TABLE TRANSFORMATION | | | | | | 2 | LOAD AS SELECT | | | | | | 3 | HASH UNIQUE | | 8168 | | 30 (4)| | 4 | COLLECTION ITERATOR PICKLER FETCH| STR2TBL | 8168 | | 29 (0)| | 5 | VIEW | | 3 | 51 | 3 (0)| | 6 | TABLE ACCESS FULL | SYS_TEMP_0FD9D6610_52718F2 | 3 | 6 | 3 (0)| Note cardinality feedback used for this statement 24 rows selected.

54 Cardinality Feedback ops$tkyte%ORA11GR2> create table t 2 as 3 select * from all_objects; Table created. ops$tkyte%ORA11GR2> ops$tkyte%ORA11GR2> create index t_idx on t(object_name); Index created. ops$tkyte%ORA11GR2> ops$tkyte%ORA11GR2> exec dbms_stats.gather_table_stats(user,'T'); PL/SQL procedure successfully completed.

55 Cardinality Feedback ops$tkyte%ORA11GR2> with data as 2 ( select /*+ materialize */ distinct * 3 from TABLE(cast( str2tbl( :in_list ) as str2tblType) ) t 4 ) 5 select t.object_id, t.object_name from data, t 6 where t.object_name = data.column_value; OBJECT_ID OBJECT_NAME DBMS_OUTPUT 4973 DBMS_OUTPUT 4982 UTL_FILE 4983 UTL_FILE 7929 DBMS_PIPE 7930 DBMS_PIPE 8842 DBMS_OUTPUT UTL_FILE DBMS_PIPE UTL_FILE 10 rows selected.

56 Cardinality Feedback | Id | Operation | Name | Rows | Cost (%CPU)| Time | 0 | SELECT STATEMENT | | | 324 (100)| | 1 | TEMP TABLE TRANSFORMATION | | | | | 2 | LOAD AS SELECT | | | | | 3 | HASH UNIQUE | | 8168 | 30 (4)| 00:00:01 | 4 | COLLECTION ITERATOR PICKLER FETCH| STR2TBL | 8168 | 29 (0)| 00:00:01 |* 5 | HASH JOIN | | | 294 (1)| 00:00:04 | 6 | VIEW | | 8168 | 3 (0)| 00:00:01 | 7 | TABLE ACCESS FULL | SYS_TEMP_0FD9D6611_52718F2 | 8168 | 3 (0)| 00:00:01 | 8 | TABLE ACCESS FULL | T | | 290 (1)| 00:00: Predicate Information (identified by operation id): access("T"."OBJECT_NAME"="DATA"."COLUMN_VALUE") 28 rows selected.

57 Cardinality Feedback | Id | Operation | Name | Rows | Cost (%CPU)| Time | 0 | SELECT STATEMENT | | | 42 (100)| | 1 | TEMP TABLE TRANSFORMATION | | | | | 2 | LOAD AS SELECT | | | | | 3 | HASH UNIQUE | | 8168 | 30 (4)| 00:00:01 | 4 | COLLECTION ITERATOR PICKLER FETCH| STR2TBL | 8168 | 29 (0)| 00:00:01 | 5 | NESTED LOOPS | | | | | 6 | NESTED LOOPS | | 5 | 12 (0)| 00:00:01 | 7 | VIEW | | 3 | 3 (0)| 00:00:01 | 8 | TABLE ACCESS FULL | SYS_TEMP_0FD9D6612_52718F2 | 3 | 3 (0)| 00:00:01 |* 9 | INDEX RANGE SCAN | T_IDX | 2 | 2 (0)| 00:00:01 | 10 | TABLE ACCESS BY INDEX ROWID | T | 2 | 3 (0)| 00:00: Predicate Information (identified by operation id): access("T"."OBJECT_NAME"="DATA"."COLUMN_VALUE") Note cardinality feedback used for this statement 34 rows selected.

58 Cardinality Hint ops$tkyte%ORA11GR2> with data as 2 ( select * 3 from TABLE(cast( str2tbl( :in_list ) as str2tblType) ) t 4 ) 5 select t.object_id, t.object_name 6 from data, t 7 where t.object_name = data.column_value; OBJECT_ID OBJECT_NAME DBMS_OUTPUT 4973 DBMS_OUTPUT 4982 UTL_FILE 4983 UTL_FILE 7929 DBMS_PIPE 7930 DBMS_PIPE 8842 DBMS_OUTPUT UTL_FILE DBMS_PIPE UTL_FILE 10 rows selected.

59 Cardinality Hint | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | | 0 | SELECT STATEMENT | | | | 320 (100)| | |* 1 | HASH JOIN | | | 425K| 320 (1)| 00:00:04 | | 2 | COLLECTION ITERATOR PICKLER FETCH| STR2TBL | 8168 | | 29 (0)| 00:00:01 | | 3 | TABLE ACCESS FULL | T | | 2130K| 290 (1)| 00:00:04 | Predicate Information (identified by operation id): access("T"."OBJECT_NAME"=VALUE(KOKBF$)) 22 rows selected.

60 Cardinality Hint ops$tkyte%ORA11GR2> with data as 2 ( select /*+ cardinality( t, 10 ) */ * 3 from TABLE(cast( str2tbl( :in_list ) as str2tblType) ) t 4 ) 5 select t.object_id, t.object_name 6 from data, t 7 where t.object_name = data.column_value; OBJECT_ID OBJECT_NAME DBMS_OUTPUT 4973 DBMS_OUTPUT 4982 UTL_FILE 4983 UTL_FILE 7929 DBMS_PIPE 7930 DBMS_PIPE 8842 DBMS_OUTPUT UTL_FILE DBMS_PIPE UTL_FILE 10 rows selected.

61 Cardinality Hint | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | | 0 | SELECT STATEMENT | | | | 59 (100)| | | 1 | NESTED LOOPS | | | | | | | 2 | NESTED LOOPS | | 17 | 544 | 59 (0)| 00:00:01 | | 3 | COLLECTION ITERATOR PICKLER FETCH| STR2TBL | 10 | 20 | 29 (0)| 00:00:01 | |* 4 | INDEX RANGE SCAN | T_IDX | 2 | | 2 (0)| 00:00:01 | | 5 | TABLE ACCESS BY INDEX ROWID | T | 2 | 60 | 3 (0)| 00:00:01 | Predicate Information (identified by operation id): access("T"."OBJECT_NAME"=VALUE(KOKBF$)) 25 rows selected.

How to Defeat Statistics

How to defeat Statistics Using the wrong data type Using ‘fake’ values Abusing functions

Using the wrong datatype ops$tkyte%ORA11GR2> create table t ( str_date, date_date, number_date, data ) 2 as 3 select to_char( dt+rownum,'yyyymmdd' ), 4 dt+rownum, 5 to_number( to_char( dt+rownum,'yyyymmdd' ) ), 6 rpad('*',45,'*') 7 from (select to_date('01-jan-1995','dd-mon-yyyy') dt 8 from stage) 9 order by dbms_random.random 10 / Table created. ops$tkyte%ORA11GR2> create index t_str_date_idx on t(str_date); Index created. ops$tkyte%ORA11GR2> create index t_date_date_idx on t(date_date); Index created. ops$tkyte%ORA11GR2> create index t_number_date_idx on t(number_date); Index created.

ops$tkyte%ORA11GR2> begin 2 dbms_stats.gather_table_stats 3 ( user, 'T', 4 method_opt=> 'for all indexed columns', 5 cascade=> true ); 6 end; 7 / PL/SQL procedure successfully completed. Using the wrong datatype

ops$tkyte%ORA11GR2> select * 2 from t 3 where str_date between ' ' and ' '; … ops$tkyte%ORA11GR2> select * from table(dbms_xplan.display_cursor); | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | | 0 | SELECT STATEMENT | | | | 212 (100)| | |* 1 | TABLE ACCESS FULL| T | 285 | | 212 (1)| 00:00:03 | Predicate Information (identified by operation id): filter(("STR_DATE" =' ')) 18 rows selected. Using the wrong datatype

ops$tkyte%ORA11GR2> select * 2 from t 3 where number_date between and ; … ops$tkyte%ORA11GR2> select * from table(dbms_xplan.display_cursor); | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | | 0 | SELECT STATEMENT | | | | 212 (100)| | |* 1 | TABLE ACCESS FULL| T | 285 | | 212 (1)| 00:00:03 | Predicate Information (identified by operation id): filter(("NUMBER_DATE" = )) 18 rows selected. Using the wrong datatype

ops$tkyte%ORA11GR2> select * from t where date_date 2 between to_date(' ','yyyymmdd') and to_date(' ','yyyymmdd'); … ops$tkyte%ORA11GR2> select * from table(dbms_xplan.display_cursor); | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | | 0 | SELECT STATEMENT | | | | 3 (100)| | | 1 | TABLE ACCESS BY INDEX ROWID| T | 1 | 47 | 3 (0)| 00:00:01 | |* 2 | INDEX RANGE SCAN | T_DATE_DATE_IDX | 1 | | 2 (0)| 00:00:01 | Predicate Information (identified by operation id): access("DATE_DATE">=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss') AND "DATE_DATE"<=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss')) 21 rows selected. Using the wrong datatype

Using Fake Values ops$tkyte%ORA11GR2> create table t 2 pctfree 20 3 as 4 select a.*, 5 case when mod(rownum,100) <= 50 6 then last_ddl_time 7 end end_date 8 from all_objects a; Table created.

Using Fake Values ops$tkyte%ORA11GR2> create index t_idx 2 on t(end_date); Index created.

Using Fake Values ops$tkyte%ORA11GR2> select count(*) 2 from t 3 where end_date 4 between to_date('01-sep-2010', 'dd-mon-yyyy') 5 and to_date('30-sep-2010', 'dd-mon-yyyy'); COUNT(*)

Using Fake Values ops$tkyte%ORA11GR2> begin 2 dbms_stats.gather_table_stats(user, 'T'); 3 end; 4 / PL/SQL procedure successfully completed.

Using Fake Values ops$tkyte%ORA11GR2> select count(*), 2 count(distinct end_date), 3 count(end_date), 4 min(end_date), 5 max(end_date) 6 from t; CNT CNTD CNT2 MIN MAX OCT SEP-11

Using Fake Values ops$tkyte%ORA11GR2> set autotrace traceonly explain ops$tkyte%ORA11GR2> select * 2 from t 3 where end_date 4 between to_date( '01-sep-2010', 'dd-mon-yyyy' ) 5 and to_date( '30-sep-2010', 'dd-mon-yyyy' ); Execution Plan Plan hash value:

Using Fake Values | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | | 0 | SELECT STATEMENT | | | 3588K| 339 (1)| 00:00:05 | |* 1 | TABLE ACCESS FULL| T | | 3588K| 339 (1)| 00:00:05 | Predicate Information (identified by operation id): filter("END_DATE"<=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss') AND "END_DATE">=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss'))

Using Fake Values ops$tkyte%ORA11GR2> update t 2 set end_date = 3 to_date( '01-jan-9999','dd-mon-yyyy' ) 4 where end_date is null; rows updated. ops$tkyte%ORA11GR2> commit; Commit complete.

Using Fake Values ops$tkyte%ORA11GR2> begin 2 dbms_stats.gather_table_stats(user, 'T'); 3 end; 4 / PL/SQL procedure successfully completed.

Using Fake Values ops$tkyte%ORA11GR2> select * 2 from t 3 where end_date 4 between to_date('01-sep-2010', 'dd-mon-yyyy') 5 and to_date('30-sep-2010', 'dd-mon-yyyy'); Execution Plan Plan hash value:

Using Fake Values | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| | 0 | SELECT STATEMENT | | 175 | | 10 (0)| | 1 | TABLE ACCESS BY INDEX ROWID| T | 175 | | 10 (0)| |* 2 | INDEX RANGE SCAN | T_IDX | 175 | | 2 (0)| Predicate Information (identified by operation id): filter("END_DATE"<=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss') AND "END_DATE">=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss'))

Abusing Functions ops$tkyte%ORA11GR2> create table t 2 as 3 select * 4 from all_objects; Table created. ops$tkyte%ORA11GR2> ops$tkyte%ORA11GR2> begin 2 dbms_stats.gather_table_stats 3 ( user, 'T', 4 method_opt => 'for all columns size 254' ); 5 end; 6 / PL/SQL procedure successfully completed.

Abusing Functions ops$tkyte%ORA11GR2> select 2 count( 3 case when 4 created >= to_date( '28-mar-2012', 'dd-mon-yyyy' ) 5 and 6 created < to_date( '28-mar-2012', 'dd-mon-yyyy' )+1 7 then 1 8 end 9 ) cnt, 10 count(*)*0.01 one_pct 11 from t 12 / CNT ONE_PCT

Abusing Functions ops$tkyte%ORA11GR2> select * 2 from t 3 where trunc(created) = to_date( '28-mar-2012', 'dd-mon-yyyy' ); Execution Plan Plan hash value: | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | | 0 | SELECT STATEMENT | | 727 | | 293 (2)| 00:00:04 | |* 1 | TABLE ACCESS FULL| T | 727 | | 293 (2)| 00:00:04 | Predicate Information (identified by operation id): filter(TRUNC(INTERNAL_FUNCTION("CREATED"))=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss'))

Abusing Functions ops$tkyte%ORA11GR2> select * 2 from t 3 where created >= to_date( '28-mar-2012', 'dd-mon-yyyy' ) 4 and created < to_date( '28-mar-2012', 'dd-mon-yyyy' )+1; Execution Plan Plan hash value: | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | | 0 | SELECT STATEMENT | | 5726 | 542K| 290 (1)| 00:00:04 | |* 1 | TABLE ACCESS FULL| T | 5726 | 542K| 290 (1)| 00:00:04 | Predicate Information (identified by operation id): filter("CREATED">=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss') AND "CREATED"<TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss'))

When all else Fails…

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 85 If you can hint it, baseline it  Its not always possible to add hints to third party applications  Hints can be extremely difficult to manage over time  Once added never removed Alternative approach to hints Solution  Use SQL Plan Management (SPM)  Influence the execution plan without adding hints directly to queries  SPM available in EE, no additional options required

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 86 If you can hint it, baseline it SQL Plan Management Parse HJ GB Plan history Plan baseline Execute Plan Acceptable HJ GB Users

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 87 If you can hint it, baseline it SQL Plan Management Parse NL GB Plan history Plan baseline HJ GB Users NL GB

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 88 If you can hint it, baseline it SQL Plan Management Parse Plan history Plan baseline Execute Plan Acceptable HJ GB Users NL GB HJ GB

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 89 Influence execution plan without adding hints  Simple two table join between the SALES and PRODUCTS tables Example Overview Group By HASH JOIN TABLE ACCESS SALES TABLE ACCESS PRODUCTS Group By HASH JOIN TABLE ACCESS SALES INDEX RANGE SCAN PROD_SUPP_ID_INDX Current PlanDesired Plan

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 90 Influence execution plan without adding hints Step 1. Execute the non-hinted SQL statement

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 91 Influence execution plan without adding hints Default plan is uses full table scans followed by a hash join

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 92 Influence execution plan without adding hints Step 2. Find the SQL_ID for the non-hinted statement in V$SQL

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 93 Influence execution plan without adding hints Step 3. Create a SQL plan baseline for the non-hinted SQL statement

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 94 Influence execution plan without adding hints Step 4. Captured Plan is not our desired plan so it should be disabled

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 95 Influence execution plan without adding hints Step 5. Modify the SQL statement to use the hint(s) & execute it

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 96 Influence execution plan without adding hints Step 6. Find SQL_ID & PLAN_HASH_VALUE for hinted SQL stmt

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 97 Influence execution plan without adding hints Step 7. Associate hinted plan with original SQL stmt’s SQL HANDLE Sql_id & plan_hash_value belong to hinted statement sql_handle is for the non-hinted statement

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 98 Influence execution plan without adding hints Step 8. Confirm SQL stmt has two plans in it’s baseline Hinted plan only accepted plan for non-hinted SQL stmt

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 99 Influence execution plan without adding hints Step 9. Confirm hinted plan is being used Non-hinted SQL text but it is using the plan hash value for the hinted stmt Note section also confirms SQL plan baseline used for stmt

Getting the right Cardinality Thomas Kyte