Presentation is loading. Please wait.

Presentation is loading. Please wait.

Optimizer Statistics Julian Dyke Independent Consultant Web Version

Similar presentations


Presentation on theme: "Optimizer Statistics Julian Dyke Independent Consultant Web Version"— Presentation transcript:

1 Optimizer Statistics Julian Dyke Independent Consultant Web Version
juliandyke.com

2 Agenda Statistics Strategies ANALYZE DBMS_STATS
ANALYZE versus DBMS_STATS System Statistics Automatic Statistics Collection Statistics History Manual Statistics Partition Statistics Oracle 11g Enhancements

3 Statistics Strategies

4 Statistics Strategies Introduction
Only gather statistics when object has changed significantly Consider generating manual statistics for: Very large objects Temporary tables Highly volatile tables Try to collect statistics during a specified window Do not execute SQL statements during statistics collection Use DBMS_STATS parallelism Review sample sizes for very large objects In complex databases do not rely on Auto job Unpredictable collection behaviour / duration Unpredictable execution plan changes See Metalink Note Statistics Gathering; Frequency and Strategy

5 ANALYZE

6 ANALYZE Introduction SQL statement Introduced in Oracle 7.0
Collects object statistics for Cost Based Optimizer Initially collected statistics on tables and indexes Subsequently on histograms Advantages Online analyze - Oracle 9.0 and above Optionally validates structure of tables, indexes and clusters Optionally detects chained / migrated rows Disadvantages Serial statistics collection only Limited partition-awareness

7 ANALYZE VALIDATE STRUCTURE
Syntax is ANALYZE INDEX <index_name> VALIDATE STRUCTURE; Populates session-level dynamic performance views INDEX_STATS INDEX_HISTOGRAMS Reports index statistics including: height number of leaf blocks number of branch blocks number of distinct keys Reports space released by deletions Use to determine when to coalesce or rebuild index Recommends index compression columns Use to determine which indexes should be compressed

8 ANALYZE VALIDATE STRUCTURE ONLINE
Introduced in Oracle 9.0.1 Syntax is ANALYZE INDEX <index_name> VALIDATE STRUCTURE ONLINE; ONLINE option only checks structure does not require TM (DML) lock does not populate INDEX_STATS or INDEX_HISTOGRAM

9 DBMS_STATS

10 DBMS_STATS Introduction
PL/SQL supported package Introduced in Oracle 8.1.5 In Oracle 11.1 there are 107 subroutines Defined in $ORACLE_HOME/rdbms/admin/dbmsstat.sql Advantages Parallel statistics collection Partition-awareness Disadvantages Cannot perform structure validation Cannot detect chaining / migration

11 DBMS_STATS Gathering Statistics
Statistics can be gathered using the following subroutines: GATHER_DATABASE_STATS GATHER_SCHEMA_STATS GATHER_TABLE_STATS GATHER_INDEX_STATS These routines allow you to specify: Sample size Granularity (partitioned tables only) Histogram policy Cursor invalidation policy

12 DBMS_STATS Deleting Statistics
Statistics can be deleted using the following subroutines: DELETE_DATABASE_STATS DELETE_SCHEMA_STATS DELETE_TABLE_STATS DELETE_INDEX_STATS DELETE_COLUMN_STATS DELETE_TABLE_STATS includes CASCADE_PART - optionally delete partition statistics CASCADE_COLUMNS - optionally delete column statistics CASCADE_INDEXES - optionally delete index statistics DELETE_INDEX_STATS, DELETE_COLUMN_STATS include:

13 DBMS_STATS Statistics Tables
Used to export / import statistics store object statistics values PRIOR to GATHER operations To create a statistics table use CREATE_STATS_TABLE. For example: dbms_stats.create_stats_table ( ownname => 'SYSTEM', tabname => 'STATS1', tblspace=> 'SYSAUX' ); To drop a statistics table use DROP_STATS_TABLE To drop a statistics table use DROP_STATS_TABLE. For example: dbms_stats.drop_stats_table ( ownname => 'SYSTEM', tabname => 'STATS1' );

14 DBMS_STATS Exporting and Importing Statistics
Statistics can be exported to a statistics table using: EXPORT_COLUMN_STATS EXPORT_DATABASE_STATS EXPORT_INDEX_STATS EXPORT_SCHEMA_STATS EXPORT_TABLE_STATS Statistics can be imported from a statistics table using: IMPORT_COLUMN_STATS IMPORT_DATABASE_STATS IMPORT_INDEX_STATS IMPORT_SCHEMA_STATS IMPORT_TABLE_STATS To transfer the statistics to another database export and import the statistics table

15 DBMS_STATS OPTIONS parameter
Specifies how to determine which objects to gather statistics for: GATHER - gather statistics for all objects GATHER AUTO - gather statistics for all objects needing new statistics GATHER STALE - gather statistics for all objects having stale statistics GATHER EMPTY- gather statistics for all objects having no statistics LIST AUTO - list objects needing new statistics LIST STALE - list objects having stale statistics LIST EMPTY - list objects having stale statistics The OBJLIST parameter must be specified as an OUT parameter for the LIST options GATHER STALE and GATHER EMPTY can also return a list of objects if this parameter is specified

16 DBMS_STATS OPTIONS parameter
If GATHER STALE is specified statistics will be gathered for objects where at least 10% of rows have changed Total of INSERTS, UPDATES and DELETES since last statistics collection In Oracle 10g percentage is fixed at 10% In Oracle 11g percentage is user-configurable

17 DBMS_STATS Stale Statistics
In Oracle 10.1 and above table modification statistics: are enabled if STATISTICS_LEVEL is TYPICAL or ALL are disabled if STATISTICS_LEVEL is BASIC are collected in the SGA are automatically flushed to SYS.MON_MODS$ by SMON background process every 15 minutes to SYS.MON_MODS$ if a significant number of rows has changed (possibly 10%) can be manually flushed to SYS.MON_MOD$ using: DBMS_STATS.FLUSH_DATABASE_MONITORING_INFO are reported in DBA_TAB_MODIFICATIONS

18 DBMS_STATS Stale Statistics
SYS.MON_MODS$ has the following columns: Column Name Data Type OBJ# NUMBER INSERTS UPDATES DELETES TIMESTAMP DATE FLAGS DROP_SEGMENTS

19 DBMS_STATS Stale Statistics
In Oracle 8i, table modification monitoring can only be enabled manually using: ALTER TABLE <table_name> MONITORING; ALTER TABLE <table_name> NOMONITORING; In Oracle 9i, table modification monitoring can also be enabled using DBMS_STATS.ALTER_SCHEMA_TAB_MONITORING ('<owner>',TRUE); DBMS_STATS.ALTER_DATABASE_TAB_MONITORING (TRUE); Reported in DBA_TABLES.MONITORING In Oracle 10.1 and above MONITORING clause is deprecated and ignored

20 DBMS_STATS ESTIMATE_PERCENT Parameter
Specified using ESTIMATE_PERCENT parameter in GATHER_DATABASE_STATS GATHER_DICTIONARY_STATS GATHER_INDEX_STATS GATHER_SCHEMA_STATS GATHER_TABLE_STATS Valid range is to 100 NULL means COMPUTE Default (10.2) is DBMS_STATS.AUTO_SAMPLE_SIZE Oracle determines sample size automatically Default can be changed using SET_PARAM procedure

21 DBMS_STATS GRANULARITY Parameter
Only relevant for partitioned objects ALL - gather global, partition and subpartition statistics AUTO - determine granularity based on partitioning type. DEFAULT - gather global and partition-level statistics. Now obsolete. Use GLOBAL AND PARTITION instead GLOBAL - gather global statistics GLOBAL AND PARTITION - gather global and partition-level statistics PARTITION - gather partition-level statistics SUBPARTITION - gather subpartition-level statistics In Oracle 10.2 default is AUTO (not DEFAULT)

22 DBMS_STATS NO_INVALIDATE parameter
In earlier versions ANALYZE and DBMS_STATS gather procedures always invalidated cursors referencing affected object In Oracle 10.1 and above, invalidation can be controlled using NO_INVALIDATE parameter TRUE - do not invalidate dependent cursors FALSE - invalidate dependent cursors DBMS_STATS.AUTO_INVALIDATE (default) - Oracle determines whether to invalidate dependent cursors or not. In Oracle 10.2 the default value can be changed using SET_PARAM procedure Only relevant for DBMS_STATS gather procedures ANALYZE always invalidates analyzed objects

23 DBMS_STATS Data Dictionary Statistics
Oracle recommends that data dictionary statistics are collected in Oracle and above Subroutines are: GATHER_DICTIONARY_STATS DELETE_DICTIONARY_STATS EXPORT_DICTONARY_STATS RESTORE_DICTIONARY_STATS In Oracle 9i data dictionary statistics should only be collected using: dbms_stats.gather_schema_stats (ownname => 'SYS',cascade=> TRUE); In Oracle 9i data dictionary statistics can be deleted using: dbms_stats.delete_schema_stats (ownname => 'SYS'); See Metalink Note Note ANALYZE should not be used with tables owned by SYS. See Metalink Note

24 DBMS_STATS Fixed Object Statistics
In Oracle 10.1 and above Oracle recommends that statistics are collected on fixed objects These include X$ and V$ views Subroutines include: GATHER_FIXED_OBJECTS_STATS DELETE_FIXED_OBJECTS_STATS EXPORT_FIXED_OBJECT_STATS RESTORE_FIXED_OBJECTS_STATS For example: dbms_stats.gather_fixed_objects_stats; According to Metalink Note fixed object statistics can be collected and stored in a temporary table and subsequently written to the data dictionary. To create the temporary statistics table use: EXECUTE dbms_stats.create_stat_table ('SYS','FIXED_STATS'); To collect statistics and to store them in the temporary statistics table use: EXECUTE dbms_stats.gather_fixed_objects ('FIXED_STATS'); To copy the statistics from the temporary statistics table to the data dictionary use: EXECUTE dbms_stats.gather_fixed_objects (NULL); JMD Not tested by me. Only gather fixed object statistics after the database has been running a representative workload See Metalink Note

25 DBMS_STATS METHOD_OPT Parameter
Specifies histogram collection policy Can be specified for: GATHER_DATABASE_STATS, GATHER_DICTIONARY_STATS, GATHER_SCHEMA_STATS GATHER_TABLE_STATS Default is FOR ALL COLUMNS SIZE AUTO Uses data from COL_USAGE$ table

26 DBMS_STATS METHOD_OPT Parameter
Syntax is: FOR ALL [ INDEXED | HIDDEN ] COLUMNS [ size_clause ] FOR COLUMNS [ [ size_clause ] column|attribute [size_clause] [, column|attribute [size_clause]..] size_clause is: SIZE { integer | REPEAT | AUTO | SKEWONLY } where: integer - number of histogram buckets (1..254) REPEAT - only collect histograms for columns that already have histograms AUTO - determine which columns need histograms automatically SKEWONLY - determine which columns need histograms based on data distribution

27 DBMS_STATS Column Usage Monitoring
Column usage is recorded in SYS.COL_USAGE$ Column Name Data Type OBJ# NUMBER INTCOL# EQUALITY_PREDS EQUIJOIN_PREDS NONEQUIJOIN_PREDS RANGE_PREDS LIKE_PREDS NULL_PREDS TIMESTAMP DATE COL_USAGE$ is Updated when statements are parsed Reset when statistics are gathered for table

28 DBMS_STATS Histograms Summary
Do not collect histograms if not necessary Expensive to collect Check accuracy of histograms Watch for columns with high proportion of NULL values Watch for highly skewed columns Height-balanced histograms Limited use if column is not highly skewed Object oriented designs rarely work with histograms Using histograms introduces bind variable peeking issues

29 DBMS_STATS Locking Statistics
Statistics can be locked for schemas, tables, and partitions LOCK_SCHEMA_STATS / UNLOCK_SCHEMA_STATS LOCK_TABLE_STATS / UNLOCK_TABLE_STATS LOCK_PARTITION_STATS / UNLOCK_PARTITION_STATS dbms_stats.lock_table_stats ( ownname => 'USER1', tabname => 'T1', stattype => 'ALL' ); Possible values for STATTYPE are NULL, DATA, CACHE and ALL By default GATHER procedures will not overwrite locked statistics Specify FORCE => TRUE to overwrite Statistics locking reported in STATTYPE_LOCKED column in DBA_TAB_STATISTICS DBA_IND_STATISTICS

30 DBMS_STATS Default Parameters
In Oracle 10.1 and above default parameters can be specified System-wide defaults can be set for Column Name Factory Default CASCADE DBMS_STATS.AUTO_CASCADE DEGREE NULL ESTIMATE_PERCENT DBMS_STATS.AUTO_SAMPLE_SIZE METHOD_OPT FOR ALL COLUMNS SIZE AUTO NO_INVALIDATE DBMS_STATS.AUTO_INVALIDATE GRANULARITY AUTO AUTOSTATS_TARGET AUTOSTATS_TARGET can be ALL collect statistics for all objects in the database ORACLE collect statistics for all Oracle-owned objects AUTO determine which objects need new statistics Statistics default values will be used in GATHER, EXPORT and IMPORT functions. Note that CASCADE is only applicable for GATHER Factory defaults in Oracle 10.2 are: CASCADE - DBMS_STATS.AUTO_CASCADE DEGREE - NULL ESTIMATE_PERCENT - DBMS_STATS.AUTO_SAMPLE_SIZE METHOD_OPT- FOR ALL COLUMNS SIZE AUTO NO_INVALIDATE - DBMS_STATS.NO_INVALIDATE GRANULARITY - AUTO AUTOSTATS_TARGET - AUTO When setting parameters NULL resets parameter to factory default 'NULL' specifies a value of NULL Parameters can be reset to factory settings using the RESET_PARAM_DEFAULTS procedure

31 DBMS_STATS Default Parameters
To set default parameters use SET_PARAM: dbms_stats.set_param (pname => 'CASCADE',pval => TRUE); Take care with NULL values For example: dbms_stats.set_param ('ESTIMATE_PERCENT','NULL'); Sets the default to COMPUTE statistics dbms_stats.set_param ('ESTIMATE_PERCENT',NULL); Restores factory default (DBMS_STATS.AUTO_SAMPLE_SIZE) To check default parameters use GET_PARAM e.g.: GET_PARAM is obsolete in Oracle use GET_PREFS instead SET_PARAM is obsolete in Oracle use SET_GLOBAL_PREFS instead RESET_PARAM_DEFAULTS is obsolete in Oracle 11. Use RESET_GLOBAL_PREF_DEFAULTS instead SELECT dbms_stats.get_param (pname => 'CASCADE') FROM dual; To restore all factory default parameters use: dbms_stats.reset_param_defaults;

32 ANALYZE versus DBMS_STATS

33 ANALYZE versus DBMS_STATS Differences
ANALYZE updates the following columns in DBA_TAB_STATISTICS EMPTY_BLOCKS AVG_SPACE DBMS_STATS sets these columns to 0 Tables AVG_ROW_LEN differs - e.g.: Indexes No obvious differences between ANALYZE and DBMS_STATS Columns AVG_COL_LEN differs - e.g.: ANALYZE 43 DBMS_STATS 39 ANALYZE 4 DBMS_STATS 5

34 ANALYZE versus DBMS_STATS Differences
ANALYZE <table> gathers statistics for related indexes Prior to Oracle 10.1 DBMS_STATS requires CASCADE => TRUE For empty indexes ANALYZE sets BLEVEL to 0 DBMS_STATS sets BLEVEL > 0

35 ANALYZE versus DBMS_STATS Summary
Use ANALYZE to Collect index statistics without updating data dictionary Identify candidates for index coalesce/rebuild Identify candidates for index compression Identify tables containing excessive chained rows Use DBMS_STATS Gather statistics on tables / indexes Do not mix ANALYZE / DBMS_STATS

36 System Statistics

37 System Statistics Introduction
System statistics were introduced in Oracle 9.0.1 Subroutines are: GATHER_SYSTEM_STATS DELETE_SYSTEM_STATS GET_SYSTEM_STATS SET_SYSTEM_STATS EXPORT_SYSTEM_STATS IMPORT_SYSTEM_STATS RESTORE_SYSTEM_STATS System statistics are stored in SYS.AUX_STATS$

38 System Statistics Workload versus No Workload Statistics
Oracle 10.1 and above supports: noworkload statistics - default values supplied during database creation workload statistics - collected from representative workload Workload statistics include: CPUSPEED - CPU speed SREADTIM - Single block read time in milliseconds MREADTIM - Multi block read time in milliseconds MBRC - Multi block read count MAXTHR - Maximum I/O system throughput (parallel execution only) SLAVETHR - Average slave I/O throughput (parallel execution only) No workload statistics include: CPUSPEEDNW - CPU speed IOSEEKTIM - IO Seek Time IOTFRSPEED - IO Transfer Speed In Oracle 10.2 the optimizer uses the value of MBRC instead of db_file_multiblock_read_count to compute costs for full table scans and index fast full scans. Oracle 9.0 includes CPUSPEED, SREADTIM, MREADTIM and MBRC. MAXTHR and SLAVETHR were added in Oracle 9.2 to set a lower limit for full table scans CPUSPEEDNW, IOSEEKTIM and IOTFRSPEED were added in Oracle 10.1 Values for CPUSPEEDNW, IOSEEKTIM and IOTFRSPEED can be collected using: DBMS_STAT.GATHER_SYSTEM_STATS (gathering_mode =>NOWORKLOAD);

39 System Statistics Example
Create a statistics table using: dbms_stats.create_stat_table ('SYS','OLTP_STATS'); Gather system statistics for a typical period using: dbms_stats.gather_system_stats ( gathering_mode => 'INTERVAL', interval => 60, seconds stattab => OLTP_STATS', statid => 'OLTP' ); Import system statistics into AUX_STATS$ using: In order to collect system statistics JOB_QUEUE_PROCESSES must be set to 1 or more: ALTER SYSTEM SET job_queue_processes = 1; dbms_stats.import_system_stats ( stattab => OLTP_STATS', statid => 'OLTP', statown => 'SYS' );

40 System Statistics Summary
Enable system statistics for single instance databases Usually improve execution plans Consider carefully before enabling system statistics in RAC databases System statistics are database-specific Watch for asymmetric nodes Hardware differences Service configuration System statistics are: Difficult to monitor Very difficult to update / remove If exporting object statistics to another system for testing: Remember to export system statistics

41 Automatic Statistics Collection

42 Automatic Statistics Collection Introduction
Oracle 10.1 and above Statistics collected during Maintenance Window Monday - Friday 22:00 to 06:00 Saturday / Sunday All day Note that weekend window effectively starts at 06:00 on Saturday morning Scheduler job GATHER_STATS_JOB Scheduler job class AUTO_TASKS_JOB_CLASS Scheduler windows WEEKNIGHT_WINDOW WEEKEND_WINDOW Scheduler window group MAINTENANCE_WINDOW_GROUP

43 Statistics History

44 Statistics History Introduction
In Oracle 10.1 and above, existing statistics are stored in the data dictionary when new statistics are collected Statistics can be restored using: RESTORE_DATABASE_STATS RESTORE_DICTIONARY_STATS RESTORE_FIXED_OBJECTS_STATS RESTORE_SCHEMA_STATS RESTORE_SYSTEM_STATS RESTORE_TABLE_STATS Statistics history for tables only is reported in DBA_TAB_STATS_HISTORY SELECT stats_update_time FROM dba_tab_stats_history WHERE owner = 'USER1' AND table_name = 'T1'; 12-FEB PM +00:00

45 Statistics History Optimizer Statistics Operations
Statistics gathering operations are reported in DBA_OPTSTAT_OPERATIONS SELECT operation, target, start_time, end_time FROM dba_optstat_operations ORDER BY start_time; gather_database_stats(auto) 07-FEB FEB gather_database_stats(auto) 09-FEB FEB gather_database_stats(auto) 10-FEB FEB DBA_OPTSTAT_OPERATIONS includes: gather_database_stats(auto) gather_schema_stats DBA_OPTSTAT_OPERATIONS does not include: gather_table_stats gather_index_stats

46 Statistics History Data Dictionary Tables
Historic statistics are stored in tables created by $ORACLE_HOME/rdbms/admin/catost.sql Tables created are: WRI$_OPTSTAT_TAB_HISTORY WRI$_OPTSTAT_IND_HISTORY WRI$_OPTSTAT_HISTHEAD_HISTORY WRI$_OPTSTAT_HISTGRM_HISTORY WRI$_OPTSTAT_AUX_HISTORY WRI$_OPTSTAT_OPR OPTSTAT_HIST_CONTROL$

47 Statistics History Retention Period
To check statistics history retention period use: SELECT dbms_stats.get_stats_history_retention FROM dual; Default is 31 days To check earliest historic statistics use: SELECT dbms_stats.get_stats_history_availability FROM dual; 12-JAN PM +00:00 To set statistics history retention period to 90 days: dbms_stats.alter_stats_history_retention ( retention => 90 );

48 Statistics History Summary
Statistics history rows contain previous values for optimizer statistics Statistics history does not include current value SAVTIME column is time row was written ANALYZETIME column is time statistics were analyzed SAVTIME != ANALYZETIME ANALYZE does not update statistics history [UN]LOCK_TABLE_STATS do update statistics history

49 Manual Statistics

50 Setting Table Statistics Introduction
Table statistics can be set manually using SET_TABLE_STATS Values can be specified for: NUMROWS - number of rows NUMBLKS - number of blocks AVGRLEN - average row length

51 Setting Table Statistics Example (1 of 4)
DECLARE l_numrows NUMBER; l_numblks NUMBER; l_avgrlen NUMBER; l_flags NUMBER; l_total_blocks NUMBER; l_total_bytes NUMBER; l_unused_blocks NUMBER; l_unused_bytes NUMBER; l_last_used_extent_file_id NUMBER; l_last_used_extent_block_id NUMBER; l_last_used_block NUMBER;

52 Setting Table Statistics Example (2 of 4)
BEGIN dbms_stats.get_table_stats ( ownname => 'GP', tabname => 'CAR', numrows => l_numrows, numblks => l_numblks, avgrlen => l_avgrlen, ); l_numrows := 0; l_numblks := 0; SELECT COUNT(*) INTO l_numrows FROM gp.car;

53 Setting Table Statistics Example (3 of 4)
dbms_space.unused_space ( segment_owner => 'GP', segment_name => 'CAR', segment_type => 'TABLE', total_blocks => l_total_blocks, total_bytes => l_total_bytes, unused_blocks => l_unused_blocks, unused_bytes => l_unused_bytes, last_used_extent_file_id => l_last_used_extent_file_id, last_used_extent_block_id => l_last_used_extent_block_id, last_used_block => l_last_used_block ); l_numblks := l_total_blocks;

54 Setting Table Statistics Example (4 of 4)
dbms_stats.set_table_stats ( ownname => 'GP', tabname => 'CAR', numrows => l_numrows, numblks => l_numblks, avgrlen => l_avgrlen ); END;

55 Setting Index Statistics Introduction
Index statistics can be set manually using SET_INDEX_STATS Values can be specified for: NUMROWS - number of rows NUMBLKS - number of blocks NUMDIST - number of distinct values AVGLBLK - average leaf blocks per key AVGDBLK - average data blocks per key CLSTFCT - clustering factor INDLEVEL - index level (height) Use ANALYZE INDEX VALIDATE STRUCTURE to determine values

56 Setting Column Statistics Introduction
Column statistics can be set manually using SET_COLUMN_STATS Values can be specified for: DISTCNT - number of distinct values DENSITY - 1 / (number of distinct values) NULLCNT - number of NULL values Low and high values Histograms Note that DISTCNT usage differs for columns with histograms

57 Setting Column Statistics Data Types
Requires additional data types declared in DBMS_STATS package See $ORACLE_HOME/rdbms/admin/dbmsstat.sql TYPE numarray IS VARRAY(256) OF NUMBER; TYPE chararray IS VARRAY(256) OF VARCHAR2(4000); TYPE statrec IS RECORD ( EPC NUMBER, MINVAL RAW(2000), MAXVAL RAW(2000), BKVALS NUMARRAY, NOVALS NUMARRAY, CHVALS CHARARRAY, EAVS NUMBER )

58 Setting Column Statistics Example (1 of 3)
PROCEDURE copy_col_stats (p_owner VARCHAR2,p_table_name VARCHAR2) IS l_row DBA_TAB_COLUMNS%ROWTYPE; l_statrec dbms_stats.statrec; l_numvals dbms_stats.numarray := dbms_stats.numarray(); l_charvals dbms_stats.chararray := dbms_stats.chararray(); l_datevals dbms_stats.datearray := dbms_stats.datearray(); CURSOR c1 RETURN DBA_TAB_COLUMNS%ROWTYPE IS SELECT * FROM dba_tab_columns WHERE owner = p_owner AND table_name = p_table_name; BEGIN l_numvals.extend(2); l_charvals.extend(2); l_datevals.extend(2); OPEN c1; LOOP FETCH c1 INTO l_row; EXIT WHEN c1%NOTFOUND;

59 Setting Column Statistics Example (2 of 3)
IF l_row.num_buckets IS NOT NULL THEN l_statrec.epc := 2; l_statrec.bkvals := NULL; l_numvals(1) := NULL; l_numvals(2) := NULL; l_charvals(1) := NULL; l_charvals(2) := NULL; l_datevals(1) := NULL; l_datevals(2) := NULL; IF l_row.column_name = 'COL1' THEN l_numvals(1) := get_min_col1 (p_owner,p_table_name); l_numvals(2) := get_max_col2 (p_owner,p_table_name); DBMS_STATS.PREPARE_COLUMN_VALUES (l_statrec,l_numvals); ELSIF l_row.column_name = 'COL2' THEN l_datevals(1) := TO_DATE (v_target_date,'YYYYMMDD'); l_datevals(2) := TO_DATE (v_target_date,'YYYYMMDD'); DBMS_STATS.PREPARE_COLUMN_VALUES (l_statrec,l_datevals); ELSIF l_row.column_name = 'COL3' THEN l_charvals(1) := get_min_col3 (p_owner,p_table_name); l_charvals(2) := get_max_col3 (p_owner,p_table_name); DBMS_STATS.PREPARE_COLUMN_VALUES (l_statrec,l_charvals); END IF;

60 Setting Column Statistics Example (3 of 3)
Setting minimum and maximum values (without histograms) DBMS_STATS.SET_COLUMN_STATS ( ownname => p_owner, tabname => p_table_name, colname => l_row.column_name, distcnt => l_row.num_distinct, density => l_row.density, nullcnt => l_row.num_nulls, srec => l_statrec, avgclen => l_row.avg_col_len ); END IF; END LOOP; CLOSE c1; END;

61 Setting Column Statistics - Histograms Example (1 of 2)
DECLARE l_statrec dbms_stats.statrec; l_charvals dbms_stats.chararray := dbms_stats.chararray (); l_bkvals dbms_stats.numarray := dbms_stats.numarray (); BEGIN l_charvals.extend (11); l_bkvals.extend (11); l_charvals(1) := 'MSCH'; l_bkvals(1) := 91; l_charvals(2) := 'APRO'; l_bkvals(2) := 51; l_charvals(3) := 'ASEN'; l_bkvals(3) := 41; l_charvals(4) := 'NMAN'; l_bkvals(4) := 31; l_charvals(5) := 'JSTE'; l_bkvals(5) := 27; l_charvals(6) := 'NLAU'; l_bkvals(6) := 25; l_charvals(7) := 'JCLA'; l_bkvals(7) := 25; l_charvals(8) := 'NPIQ'; l_bkvals(8) := 23; l_charvals(9) := 'FALO'; l_bkvals(9) := 22; l_charvals(10) := 'DHIL'; l_bkvals(10) := 22; l_charvals(11) := 'MHAK'; l_bkvals(11) := 20;

62 Setting Column Statistics - Histograms Example (2 of 2)
l_statrec.epc := 11; l_statrec.bkvals := l_bkvals; l_statrec.eavs := 0; DBMS_STATS.PREPARE_COLUMN_VALUES (l_statrec,l_charvals); DBMS_STATS.SET_COLUMN_STATS ( ownname => 'GP', tabname => 'CAR', colname => 'DRIVER_KEY', distcnt => 11, density => , / 476 rows nullcnt => 0, srec => l_statrec, avgclen => 4 ); END; /

63 Partition Statistics

64 Partition Statistics Introduction
Partition statistics are reported by: DBA_TABLES DBA_TAB_PARTITIONS DBA_TAB_STATISTICS DBA_TAB_SUBPARTITIONS (Oracle 10.1 and above) Indexes DBA_INDEXES DBA_IND_PARTITIONS DBA_IND_STATISTICS DBA_IND_SUBPARTITIONS (Oracle 10.1 and above) Columns DBA_TAB_COL_STATISTICS DBA_PART_COL_STATISTICS DBA_SUBPART_COL_STATISTICS Histograms DBA_TAB_HISTOGRAMS DBA_PART_HISTOGRAMS DBA_SUBPART_HISTOGRAMS

65 Partition Statistics Example (1 of 12)
Create a range partitioned table containing data for last four seasons CREATE TABLE car3 PARTITION BY RANGE (season_key) ( PARTITION p2005 VALUES LESS THAN ('2006'), PARTITION p2006 VALUES LESS THAN ('2007'), PARTITION p2007 VALUES LESS THAN ('2008'), PARTITION p2008 VALUES LESS THAN ('2009') ) AS SELECT * FROM car WHERE season_key >= '2005'; CREATE INDEX car3_i1 ON car3(season_key,race_key,position) LOCAL; dbms_stats.gather_table_stats ( ownname => 'GP', tabname => 'CAR3', estimate_percent => NULL, cascade => TRUE );

66 Partition Statistics Example (2 of 12)
Table Statistics SELECT num_rows,blocks,avg_row_len FROM dba_tables WHERE owner = 'GP' AND table_name = 'CAR3'; NUM_ROWS BLOCKS AVG_ROW_LEN 1518 24 37 SELECT partition_name,num_rows,blocks,avg_row_len FROM dba_tab_partitions WHERE table_owner = 'GP' AND table_name = 'CAR3'; PARTITION_NAME NUM_ROWS BLOCKS AVG_ROW_LEN P2005 380 6 36 P2006 396 38 P2007 374 37 P2008 368

67 Partition Statistics Example (3 of 12)
Index Statistics SELECT blevel, leaf_blocks, distinct_keys, clustering_factor, num_rows FROM dba_indexes WHERE owner = 'GP' AND index_name = 'CAR3_I1'; BLEVEL LEAF_BLOCKS DIST KEYS CLUFAC NUM_ROWS 1 8 1518 15 SELECT partition_name, blevel, leaf_blocks, distinct_keys, clustering_factor, num_rows FROM dba_ind_partitions WHERE index_owner = 'GP' AND index_name = 'CAR3_I1'; PARTITION_NAME BLEVEL LEAF_BLOCKS DIST KEYS CLUFAC NUM_ROWS P2005 1 2 380 3 P2006 396 P2007 374 5 P2008 368 4 Note in the above examples, AVG_LEAF_BLOCKS_PER_KEY and AVG_DATA_BLOCKS_PER_KEY were always 1so have been omitted to improve readability.

68 Partition Statistics Example (4 of 12)
Add a new partition for 2009 season ALTER TABLE car3 ADD PARTITION p2009 VALUES LESS THAN ('2010'); Gather statistics again dbms_stats.gather_table_stats ( ownname => 'GP', tabname => 'CAR3', estimate_percent => NULL, cascade => TRUE );

69 Partition Statistics Example (5 of 12)
Table Statistics SELECT num_rows,blocks,avg_row_len FROM dba_tables WHERE owner = 'GP' AND table_name = 'CAR3'; NUM_ROWS BLOCKS AVG_ROW_LEN 1518 24 37 SELECT partition_name,num_rows,blocks,avg_row_len FROM dba_tab_partitions WHERE table_owner = 'GP' AND table_name = 'CAR3'; PARTITION_NAME NUM_ROWS BLOCKS AVG_ROW_LEN P2005 380 6 36 P2006 396 38 P2007 374 37 P2008 368 P2009

70 Partition Statistics Example (6 of 12)
Index Statistics SELECT blevel, leaf_blocks, distinct_keys, clustering_factor, num_rows FROM dba_indexes WHERE owner = 'GP' AND index_name = 'CAR3_I1'; BLEVEL LEAF_BLOCKS DIST KEYS CLUFAC NUM_ROWS 1 8 1518 15 SELECT partition_name, blevel, leaf_blocks, distinct_keys, clustering_factor, num_rows FROM dba_ind_partitions WHERE index_owner = 'GP' AND index_name = 'CAR3_I1'; PARTITION_NAME BLEVEL LEAF_BLOCKS DIST KEYS CLUFAC NUM_ROWS P2005 1 2 380 3 P2006 396 P2007 374 5 P2008 368 4 P2009 Note in the above examples, AVG_LEAF_BLOCKS_PER_KEY and AVG_DATA_BLOCKS_PER_KEY were always 1so have been omitted to improve readability.

71 Partition Statistics Example (7 of 12)
Copy table statistics from 2008 to 2009 partitions dbms_stats.copy_table_stats ( ownname => 'GP', tabname => 'CAR3', srcpartname => 'P2008', dstpartname => 'P2009' );

72 Partition Statistics Example (8 of 12)
Table Statistics SELECT num_rows,blocks,avg_row_len FROM dba_tables WHERE owner = 'GP' AND table_name = 'CAR3'; NUM_ROWS BLOCKS AVG_ROW_LEN 1518 24 37 SELECT partition_name,num_rows,blocks,avg_row_len FROM dba_tab_partitions WHERE table_owner = 'GP' AND table_name = 'CAR3'; PARTITION_NAME NUM_ROWS BLOCKS AVG_ROW_LEN P2005 380 6 36 P2006 396 38 P2007 374 37 P2008 368 P2009

73 Partition Statistics Example (9 of 12)
Index Statistics SELECT blevel, leaf_blocks, distinct_keys, clustering_factor, num_rows FROM dba_indexes WHERE owner = 'GP' AND index_name = 'CAR3_I1'; BLEVEL LEAF_BLOCKS DIST KEYS CLUFAC NUM_ROWS 1 8 1518 15 SELECT partition_name, blevel, leaf_blocks, distinct_keys, clustering_factor, num_rows FROM dba_ind_partitions WHERE index_owner = 'GP' AND index_name = 'CAR3_I1'; PARTITION_NAME BLEVEL LEAF_BLOCKS DIST KEYS CLUFAC NUM_ROWS P2005 1 2 380 3 P2006 396 P2007 374 5 P2008 368 4 P2009 Note in the above examples, AVG_LEAF_BLOCKS_PER_KEY and AVG_DATA_BLOCKS_PER_KEY were always 1so have been omitted to improve readability.

74 Partition Statistics Example (10 of 12)
Copy rows from 2008 to For example: INSERT INTO car3 SELECT '2009', race_key, driver_key, team_key, engine_key, position, laps_completed, classification_key, notes, driver_points, team_points FROM gp.car3 WHERE season_key = '2008'; Statistics are unchanged

75 Partition Statistics Example (11 of 12)
Execution plans for statement against 2008 and 2009 partitions: SELECT SUM(team_points) FROM gp.car3 WHERE season_key = '2008'; 0 SELECT STATEMENT SORT AGGREGATE PARTITION RANGE (SINGLE) TABLE ACCESS (FULL) OF 'CAR3' SELECT SUM(team_points) FROM gp.car3 WHERE season_key = '2009'; 0 SELECT STATEMENT SORT AGGREGATE PARTITION RANGE (SINGLE) TABLE ACCESS (BY INDEX ROWID) OF 'CAR3' INDEX (RANGE SCAN) OF 'CAR3_I1' Plans are different even though statistics and data are theoretically identical

76 Partition Statistics Example (12 of 12)
In Oracle column values are not updated by COPY_TABLE_STATS SELECT partition_name,low_value,high_value FROM dba_part_col_statistics WHERE owner = 'GP' AND table_name = 'CAR3' AND column_name = 'SEASON_KEY'; PARTITION_NAME LOW_VALUE HIGH_VALUE P2005 0x (2005) P2006 0x (2006) P2007 0x (2007) P2008 0x (2008) P2009 Caused by bug only affects Fixed in patch includes fixes for COPY_TABLE_STATS and APPROX_GLOBAL

77 Partition Statistics Summary
Range-partitioning Only collect statistics for partitions undergoing change Specify partition names and GRANULARITY parameter to restrict partitions that are analyzed Consider setting statistics manually for new partitions COPY_STATS has limited functionality Watch for changing high values Use DBMS_STATS NO_INVALIDATE option ANALYZE invalidates all cursors referencing table being analyzed Can result in hard-parse rates approaching 100%

78 Oracle 11g Enhancements

79 Statistics Preferences

80 Oracle 11g Enhancements Statistics Preferences
In Oracle 10g preferences are: set using SET_PARAM reported by GET_PARAM stored in SYS.OPTSTAT_HIST_CONTROL$ In Oracle 11.1 and above preferences are: set using: SET_TABLE_PREFS SET_SCHEMA_PREFS SET_DATABASE_PREFS SET_GLOBAL_PREFS reported by GET_PREFS DBA_TAB_STAT_PREFS stored in SYS.OPTSTAT_USER_PREFS$ In Oracle 11.1 and above global preferences can be reset using RESET_GLOBAL_PREF_DEFAULTS By default SET_DATABASE_PREFS set statistics preferences for all tables in the database EXCEPT for tables owned by Oracle. Tables owned by Oracle can be included by specifying ADD_SYS => TRUE SET_SCHEMA_PREFS sets statistics preferences for all tables in the specified schema. Specifying a NULL value in SET_SCHEMA_PREFS restores all tables in the schema to the default value. Rows in OPTSTAT_USER_PREFS$ are updated, but not deleted.

81 Oracle 11g Enhancements Statistics Preferences
Allow automatic statistics collection to be customized for individual objects For example heavily skewed data distributions Can be set at table, schema, database and global level In Oracle 10g preferences can be set for: CASCADE DEGREE ESTIMATE_PERCENT METHOD_OPT NO_INVALIDATE GRANULARITY In Oracle 11g preferences can also be set for: PUBLISH - if TRUE publish statistics; if FALSE store as pending STALE_PERCENT - threshold level at which statistics considered stale INCREMENTAL - if TRUE collect incremental global partition statistics

82 Oracle 11g Enhancements Statistics Preferences
Delete preferences using: DELETE_TABLE_PREFS DELETE_SCHEMA_PREFS DELETE_DATABASE_PREFS Export and import preferences using: EXPORT_TABLE_PREFS / IMPORT_TABLE_PREFS EXPORT_SCHEMA_PREFS / IMPORT_SCHEMA_PREFS EXPORT_DATABASE_PREFS / IMPORT_DATABASE_PREFS

83 Pending Statistics

84 Oracle 11g Enhancements Pending Statistics
In Oracle 11.1 and above statistics can be Published - current statistics used by optimizer Pending - private statistics not yet published In Oracle 11.1 statistics for a set of objects can be collected as an atomic transaction Published at same time Avoids inconsistencies PUBLISH mode can be set at the following levels Global Database Schema Table

85 Oracle 11g Enhancements Pending Statistics
To defer publication of all statistics collected by USER1 use: dbms_stats.set_schema_prefs ('USER1','PUBLISH','FALSE'); To make pending statistics temporarily available to the optimizer use: ALTER SESSION optimizer_use_pending_statistics = TRUE; To make pending statistics permanently available to the optimizer use: dbms_stats.publish_pending_statistics ( ownname => 'USER1', tabname => NULL ); The default value for OPTIMIZER_USE_PENDING_STATISTICS is FALSE. To make all pending statistics permanently available to the optimizer use: dbms_stats.publish_pending_statistics ( ownname => NULL, tabname => NULL ); Statistics can also be published for individual tables. For example: dbms_stats.publish_pending_statistics ( ownname => 'USER1', tabname => 'T1' ); Both the NO_INVALIDATE and FORCE options can be specified for PUBLISH_PENDING_STATISTICS. Pending statistics can be deleted using DELETE_PENDING_STATS Pending statistics can be exported using EXPORT_PENDING_STATS. There is no corresponding import procedure Statistics can be published at database, schema or table level To delete pending statistics use DELETE_PENDING_STATS To export pending statistics use EXPORT_PENDING_STATS There is no corresponding import procedure

86 Extended Statistics

87 Oracle 11g Enhancements Extended Statistics
Extended statistics attempt to reflect true selectivity of data There are two types: multi-column statistics expression statistics Extended statistics use the following procedures: CREATE_EXTENDED_STATS DROP_EXTENDED_STATS SHOW_EXTENDED_STATS_NAME

88 Oracle 11g Enhancements Multi-Column Statistics
In Oracle 11g the following statistics can be gathered on multiple columns in a table (column groups): Number of distinct values Density Number of nulls Frequency histograms Multicolumn statistics Appear to work for frequency histograms <= 254 combinations of values Do not appear to work for height-based histograms > 254 combinations of values

89 Oracle 11g Enhancements Multi-Column Statistics
SELECT COUNT(*) FROM gp.car WHERE team_key = 'FER' AND engine_key = 'FER'; COUNT(*) Id Operation Name Rows Bytes Cost (%CPU) Time SELECT STATEMENT 35 (100) 1 SORT AGGREGATE 13 2 TABLE ACCESS FULL CAR 137 1781 35 (0) 00:00:01 Incorrect Cardinality

90 Oracle 11g Enhancements Multi-Column Statistics
DECLARE l_extension_name VARCHAR2(30); BEGIN l_extension_name := dbms_stats.create_extended_stats ( ownname => 'GP', tabname => 'CAR', extension => '(team_key,engine_key)' ); END; BEGIN dbms_stats.gather_table_stats ( ownname => 'GP', tabname => 'CAR', estimate_percent => NULL, method_opt => 'FOR ALL COLUMNS SIZE 254 '|| 'FOR COLUMNS (TEAM_KEY,ENGINE_KEY) SIZE 254' ); END;

91 Oracle 11g Enhancements Multi-Column Statistics
SELECT COUNT(*) FROM gp.car WHERE team_key = 'FER' AND engine_key = 'FER'; COUNT(*) Id Operation Name Rows Bytes Cost (%CPU) Time SELECT STATEMENT 35 (100) 1 SORT AGGREGATE 13 2 TABLE ACCESS FULL CAR 1410 18330 35 (0) 00:00:01 Correct Cardinality

92 Oracle 11g Enhancements Multi-Column Statistics
SELECT extension_name,extension FROM dba_stat_extensions WHERE owner = 'GP' AND table_name = 'CAR'; Extension Name Extension SYS_STUWIHETSRHT#5P210Z$AO1ON9 ("TEAM_KEY","ENGINE_KEY")

93 Oracle 11g Enhancements Multi-Column Statistics
SELECT c.intcol#,c.col#,c.name FROM sys.col$ c, sys.obj$ o, sys.user$ u WHERE c.obj# = o.obj# AND o.name = 'CAR' AND o.owner# = u.user# AND u.name = 'GP'; INTCOL# COL# NAME 1 SEASON_KEY 2 RACE_KEY 3 DRIVER_KEY 4 TEAM_KEY 5 ENGINE_KEY 6 POSITION 7 LAPS_COMPLETED 8 CLASSIFICATION_KEY 9 NOTES 10 SYS_STUWIHETSRHT#5P210Z$AO1ON9 New Virtual Column in Table for Multi-Column Statistics

94 Oracle 11g Enhancements Expression Statistics
CREATE OR REPLACE FUNCTION points (season_key VARCHAR2,race_key NUMBER,position NUMBER) RETURN NUMBER DETERMINISTIC IS l_result NUMBER; BEGIN l_result := CASE position WHEN 1 THEN 10 WHEN 2 THEN 8 WHEN 3 THEN 6 WHEN 4 THEN 5 WHEN 5 THEN 4 WHEN 6 THEN 3 WHEN 7 THEN 2 WHEN 8 THEN 1 ELSE 0 END; RETURN l_result; END; Note: The real algorithm is MUCH more complicated for various reasons including: changes to number of points awarded disqualifications, penalties etc differences between drivers and team championship points systems

95 Oracle 11g Enhancements Expression Statistics
SELECT COUNT(*) FROM gp.car WHERE POINTS (season_key,race_key,position) = 10; COUNT(*) 709 Id Operation Name Rows Bytes Cost (%CPU) Time SELECT STATEMENT 38 (100) 1 SORT AGGREGATE 13 2 TABLE ACCESS FULL CAR 177 4248 38 (0) 00:00:01 Incorrect Cardinality

96 Oracle 11g Enhancements Expression Statistics
DECLARE l_extension_name VARCHAR2(30); BEGIN l_extension_name := dbms_stats.create_extended_stats ( ownname => 'GP', tabname => 'CAR', extension => '(points(season_key,race_key,position))' ); END; BEGIN dbms_stats.gather_table_stats ( ownname => 'GP', tabname => 'CAR', estimate_percent => NULL, method_opt => 'FOR ALL COLUMNS SIZE 254 '|| 'FOR COLUMNS (points(season_key,race_key,position)) SIZE 254' ); END;

97 Oracle 11g Enhancements Expression Statistics
SELECT COUNT(*) FROM gp.car WHERE POINTS (season_key,race_key,position) = 10; COUNT(*) 709 Id Operation Name Rows Bytes Cost (%CPU) Time SELECT STATEMENT 38 (100) 1 SORT AGGREGATE 13 2 TABLE ACCESS FULL CAR 709 19143 38 (0) 00:00:01 Correct Cardinality

98 Oracle 11g Enhancements Incremental Statistics
For partitioned tables statistics can be collected: For sub-partitions For partitions Globally Global statistics should reflect underlying partitioned statistics Prior to Oracle 11.1 calculation of global statistics required full table scan of each partition in table For many applications using range partitioning only the most recent partition is subject to change Older partitions contain historical data

99 Oracle 11g Enhancements Incremental Statistics
For example: CREATE TABLE car4 PARTITION BY RANGE (season_key) ( PARTITION p2006 VALUES LESS THAN ('2005'), PARTITION p2007 VALUES LESS THAN ('2006'), PARTITION p2008 VALUES LESS THAN ('2007'), PARTITION p2009 VALUES LESS THAN ('2008') ) AS SELECT * FROM car WHERE season_key >= '2005'; CREATE INDEX i_car4_1 ON carp(season_key,race_key,position) LOCAL; INSERT INTO car4 SELECT * FROM car WHERE season_key BETWEEN 2006 AND 2009; Create table with partitions for four years data Insert data

100 Oracle 11g Enhancements Incremental Statistics
Gather statistics DBMS_STATS.GATHER_TABLE_STATS ( ownname => 'GP', tabname => 'CAR4' ); Requires full table scan for each partition 2006 Full Table Scan 2007 Full Table Scan 2008 Full Table Scan 2009 Full Table Scan

101 Oracle 11g Enhancements Incremental Statistics
Update data in one partition: UPDATE car4 SET team_points = driver_points WHERE season_key = '2009'; Gather table statistics again: DBMS_STATS.GATHER_TABLE_STATS ( ownname => 'GP', tabname => 'CAR4' ); Still requires full table scan for each partition 2006 Full Table Scan 2007 Full Table Scan 2008 Full Table Scan 2009 Full Table Scan

102 Oracle 11g Enhancements Incremental Statistics
In Oracle 11.1 and above statistics can be gathered incrementally for partitioned tables To gather incremental statistics on a specific table INCREMENTAL and PUBLISH preferences for table must be TRUE ESTIMATE_PERCENT must be AUTO_SAMPLE_SIZE GRANULARITY must be AUTO For each partition a synopsis is created Contains data about distinct values for each column in partition Stored in SYS.WRI$_OPTSTAT_SYNOPSIS_HEAD$ SYS.WRI$_OPTSTAT_SYNOPSIS$ If a partition has not been modified synopsis can be used to calculate global statistics Synopsis must be generated for all partitions first time statistics are gathered after incremental statistics are enabled

103 Oracle 11g Enhancements Incremental Statistics
Set INCREMENTAL preference to TRUE for table: BEGIN DBMS_STATS.SET_TABLE_PREFS ( ownname => 'GP', tabname => 'CAR4', pname => 'INCREMENTAL', pvalue => 'TRUE' ); END; / Gather table statistics again to generate synopsis for each partition DBMS_STATS.GATHER_TABLE_STATS ( ownname => 'GP', tabname => 'CAR4' ); All partitions will be scanned first time to create synopsis

104 Oracle 11g Enhancements Incremental Statistics
Update data in one partition: UPDATE car4 SET team_points = driver_points WHERE season_key = '2009'; Gather table statistics again: DBMS_STATS.GATHER_TABLE_STATS ( ownname => 'GP', tabname => 'CARP' ); Uses synopsis for unmodified partitions Requires full table scan for modified partition 2006 Synopsis 2007 Synopsis 2008 Synopsis 2009 Full Table Scan

105 Thanks to the following for corrections:
Greg Rahn Tony Hasler


Download ppt "Optimizer Statistics Julian Dyke Independent Consultant Web Version"

Similar presentations


Ads by Google