DB2 UDB for Linux, UNIX, and Windows Performance and Tuning

DB2 UDB for Linux, UNIX, and Windows Performance and Tuning

Module 1 Tablespace Design for Performance

DISCLAIMER This material references numerous hardware and software products by their trade names. References to other companies and their trademarks are for information purposes only, and all trademarks are the properties of their respective companies. DB2 is a registered trademark of IBM Corporation. AIX is a registered trademark of IBM Corporation. UNIX is a registered trademark of the Open Group.

Objectives Upon completion of this module you should be able to:
Understand the DB2 Storage Model Systems Managed Space Database Managed Space Containers DB2 striping NOTES:

Objectives Continued Understand the following:
how to place containers on various types of disk DB2 use of extents, extentsize, and meaning of DB2_PAGE_CONTAINER_TAG, DB2_PARALLEL_IO registry variables Use of Raw Devices NOTES:

Objectives Continued Understand the following:
Direct I/O Concurrent I/O Use of multi-page file allocation multipage_alloc (emphasis with MDC) Synchronous I/O Asynchronous I/O NOTES:

Tablespace Design SMS Tablespaces Managed by Operating System
Directories Allocated a page at a time Cannot add containers except via redirected restore Can’t separate indexes and data Cannot use raw devices NOTES: System Managed (SMS) Tablespaces use OS to manage the space. By default, system tablespaces are created as SMS. By default, many packaged applications use SMS tablespaces.

Tablespace Design DMS tablespaces Managed by DB2
Allocated extent-at-a-time Formatted ahead of time Separation of indexes and data Add containers on the fly Add container without requiring a rebalance (Begin New Stripe Set) NOTES:

Tablespace Design – DMS Continued
Use of Raw devices Can extend/resize/drop container Can breakout indexes and data into separate bufferpools NOTES: Database managed (DMS) tablespaces are managed by DB2. Think of DMS as sort of like DB2’s own special file system. All space managed by DB2.

Tablespace Basics Tablespaces use files or “containers” to store table and index data DMS Device containers refers to RAW Devices DB2 takes control of the device EXTENTSIZE Number of pages written to a container or prefetched from a container NOTES: Specified by EXTENTSIZE parameter on create tablespace statement. Cannot be changed once tablespace is created. However, PREFTECHSIZE can be changed via alter tablespace command.

Tablespace Basics PREFETCHSIZE
Is the number of pages to be read from a tablespace when prefetching is used Prefetching is used to pre-stage data into the bufferpool when the optimizer determines that data access is sequential Application doesn’t have to wait for the data NOTES:

Tablespace Basics Prefetch size should be set to a multiple of the number of containers being used Example: EXTENTSIZE = 32, 4 containers Set PREFETCHSIZE size to 128 Enables 4 extent size prefetch requests to be issued in parallel For OLTP environments smaller extent size is preferred NOTES: For OLTP environments an extent size of 32 is generally a good starting point. For BI/DW an extent size of 64 or higher is a good starting point. Note that the extent size cannot be changed once a tablespace is created. The prefetch size can be changed using the ALTER command.

Tablespace Basics Overhead and transfer rate
These values are used by the optimizer to determine the costs of I/O during query optimization Both values are measured in milliseconds Default for overhead is 24.1 Default for transfer rate is 0.9 These defaults are outdated and should not be used NOTES: Overhead is associated with controller activity, disk seek time and rotational delay (latency). Transfer rate is the amount of time required to read one page into memory. OVERHEAD AND TRANSFERRATE can be altered with alter tablespace command.

Tablespace Basics For 10K RPM disks, specify OVERHEAD=8 and FOR 4K pages a TRANSFERRATE=0.1 For 15K RPM disks specify OVERHEAD=6 If the tablespace is spread across disks with different performance characteristics, choose an average of all the values NOTES:

Tablespace Design When developing your tablespace strategy, some things to think about How many tables per tablespace? Where should indexes be placed? Bufferpool strategy Block-based bufferpool considerations Recovery and integrity considerations Data and index must be recovered together NOTES:

Tablespace Design From a performance standpoint, one table per tablespace facilitates monitoring and tuning Can easily tie tablespace activity to the table! At least consider for new tables where behavior and characteristics are not known NOTES: With the influx of packaged applications, most have few tablespaces with hundreds or thousands of tables --- consider placing most important tables in separate table space for monitoring and tuning purposes.

Tablespace Design For top performance place indexes in separate tablespace Also facilitates separate bufferpool placement Which facilitates monitoring and tuning of bufferpools and index tablespaces Access patterns Synchronous/Asynchronous NOTES: Synchronous I/O – used by DB2 to read or write pages. Used when a page or small number of pages are involved. OLTP environments should have high percentage of synchronous I/O indicating index access, few rows returned. Asynchronous I/O – use by DB2 to prefetch pages into the bufferpool in advance of application needing them. Determined as part of optimization or during execution via sequential detection. BI/DW or mixed environments have high percentage of asynchronous I/O due to sequential nature of the data and amounts of data (table scans, large amounts of data). Use “db2 get snapshot for tablespaces on sample” to get tablespace performance data.

DB2_STRIPED_ _CONTAINERS
With this variable set, Pre V8.1 caused DMS container tag to be allocated in an extent and when the extent size is aligned with RAID stripe size provided improved I/O performance Now default behavior in V8.1 Could use DB2 registry variable DB2_USE_PAGE_CONTAINER_TAG to revert to former behavior, not recommended NOTES:

DB2 Striping NOTES: DB2 writes extents in a round robin fashion as illustrated above. Thus DB2 implements striping to improve use of parallel I/O when multiple containers are used or when containers are on RAID-5, with the DB2_PARALLEL_IO environmental variable enabled.

Container Placement Rules of thumb:
Spread data over multiple containers over multiple physical disks Multiple adapters and paths for redundancy/performance NOTES:

Tablespace Design “db2 list tablespace”
“db2 list tablespace containers for 0” NOTES: Refer to List tablespace example output.

DB2_PARALLEL_IO DB2 environmental variable
Can set to a range of containers or for all “db2set db2_parallel_io=*” Set this variable when using RAID-5 storage and single container Enables DB2 to issue parallel read requests NOTES: When not enabled, the number of prefetchers requested will be based on the number of containers in the tablespace.

RAW DEVICES RAW devices still outperform all other setups Logs
Will not change for short term but things are in the works Concurrent I/O! NOTES:

Storage – The Past Enterprise Database servers with onboard disk
Limited capacity Limited Reliability Limited Availability NOTES: This is how storage looked in the past. You may still have this setup for many group or departmental databases, but the field has changed.

RAID-1 Disk mirroring Writes data in two places
With dual adapters, high availability Either disk can fail, data is still accessible With duplexing, disk controller can also fail High Performance Twice the usual number of disks NOTES: Use for top write performance and for DB2 logs (write intensive)

RAID-5 Collection of disks in an array (typically 6-7) with parity striping Parity data is striped across all disks in the array High-end storage solutions use hardware RAID-5 and large cache which reduces write penalty unless cache overrun Provides high availability unless 2 disks fail at exact same time NOTES: RAID-5 is in widespread use for database storage because of points above which reduced write penalty. However, severe degradation when cache overrun.

RAID-10 Implemented as a striped array whose segments are RAID-1arrays
Same fault tolerance as RAID-1 High I/O rates achieved by striping RAID-1 segments Has same overhead for fault tolerance as with mirroring alone NOTES: RAID-10 provides high performance. Very expensive and high overhead. Has limitations at sustained performance rates. (all drives must move in parallel to proper track lowering sustained performance)

The Present Enterprise Storage Direct Attached SAN NAS NOTES:

NOTES: This is an example of 1 tablespace with 8 containers spread over 2 sharks, using multiple adapters and arrays. This makes maximum use of available drives and enables high degree of I/O parallelism as all containers can be read simultaneously.

Balanced I/O Key for good performance Balance I/O across
Storage devices (cabinets) Clusters within a cabinet Span disk adapters Engage as many arrays as possible NOTES: Balanced I/O as illustrated in previous SHARK example is key to good performance.

Present/Future SAN NAS RAID?
NOTES: Will RAID-5 or RAID 10 handle our current and future throughput requirements?

NOTES: Subsystem Device Driver (SDD) provides multi-pathing (VPATHS) capability for use with ESS SHARK. Symmetrix has similar driver requirement.

DIRECT I/O On AIX, limited Direct I/O support added in DB2 V8.1.4 for all SMS tablespaces except the following: temporary tablespaces Long fields LOBs Not yet available for DMS, possibly future release or fixpak Previously available on Windows™ for SMS and DMS using the DB2NTNOCACHE registry variable NOTES: Limited support for Direct I/O added to DB2 on AIX with v This is only for SMS tablespaces at this time. The exceptions are for long files, LOB, and temporary table spaces. Direct I/O bypasses OS file system cache, thereby reducing CPU consumption by eliminating the need to copy the data from the file system cache to the bufferpool, for both reads and writes. Direct I/O can be enabled two ways: Using the mount option, mount –o dio <fs name> or by enabling the DB2 Registry variable, DB2_DIRECT_IO using the db2set DB2_DIRECT_IO=ON command.

DIRECT I/O Direct I/O improves performance but INODE locking still a serious detriment to performance Concurrent I/O introduced in AIX 5L V (ML1) May 27, 2003 in the enhanced Journaling File System (JFS2) DB2 UDB for Linux, UNIX, and Windows does not support Concurrent I/O as of V8.1.4 NOTES:

DIRECT I/O Direct I/O does improve performance by reducing CPU consumption But Concurrent I/O is needed to reduce INODE lock contention NOTES:

CONCURRENT I/O Concurrent I/O not yet available for DB2 as of V8.1.4 on AIX Is needed to reduce I/O bottleneck Uses Direct I/O implicity, and is enabled using similar commands Already supported by other relational databases NOTES:

CONCURRENT I/O Reduces INODE lock problems by taking a read-shared lock NOTES:

ASYNCHRONOUS I/O Also know as AIO, enabled by default in DB2 V8.1
AIO improves the performance of page cleaners Also, when using RAW devices, AIO server threads (which are also kernal threads) do not require a context switch as with files and use a “fast path” to the LVM that avoids the use of AIO server threads NOTES: AIO is enabled automatically at installation time in DB2 V8.1. It can also be enabled via SMIT or similar interface.

Storage Provisioning Storage provisioning is concerned with easing the storage administration burden for DBAs and Sytem Administrators Future plan Based on similar concepts used on mainframe for managing storage System Managed Storage (SMS) Don’t confuse this with System Managed Space (SMS)! NOTES:

Storage Provisioning Designers envision that storage provisioning will generally consist of the following components: Storage templates Dedicated storage pools Quality of Service based on priority of the data Archiving NOTES:

Storage Provisioning Automatic movement of tablespaces based on activity, less important data moved to slower devices and vice versa Automatic growing of pools and disk additions NOTES:

Table Design for Performance

Table Design Determine the different kinds of activity that will go against the table What are the business priorities? Selects (OLTP response) Updates? Inserts? Then tune to meet business objectives NOTES: DBAs cannot ensure that business requirements (rules or priorities) are met unless they are aware of them. Too many times business requirements are not known until after the fact.

Table Design Clustering Data not stored in any particular order
Regular or MDC? Freespace Data access patterns Consider tradeoffs of not clustering Data not stored in any particular order If APPEND ON used, can’t have a clustering index More reorgs may be required NOTES:

Table Design Consider reducing number of freespace control records (FSCRs) to be searched when looking for freespace for inserts Use registry variable to reduce If inserts still not fast enough, use APPEND ON Again, this must be based on business priorities NOTES: Use ALTER TABLE APPEND ON DB2MAXFSCRSEARCH (DB2 performance variable) can be used to decrease or increase the number of FSCR that DB2 will search. The default is 5. Consider reducing the number prior to using APPEND ON.

Range Clustered Tables (RCTs)
DB2 V8.1.4 introduces Range Clustered Tables (RCTs) Implements range partitioning Uses offset or displacement Easier for Oracle™ users to migrate to DB2! NOTES: Enables real range partitioning for first time in DB2. Prior to this it could be done via views and UNION ALL, but not really seen as true range partitioning. Oracle had used lack of range partitioning in anti-DB2 marketing……

Module 2 Snapshot Monitoring

Objectives After completion of this unit you should be able to:
Identify snapshot monitoring facilities available and be able to enable snapshot monitoring at the instance or application level Understand when, why and how to use snapshot monitoring NOTES:

Objectives Understand how DBAs use the output from snapshot monitoring to identify, investigate, and solve performance problems Develop a snapshot monitoring strategy for databases you support NOTES:

Monitoring In order to make sure that resources are being used efficiently and to ensure that business requirements are met, continuous monitoring must be practiced Use Health Center, Health Monitor, Memory tracker, Snapshot Monitoring and Event Monitoring Third Party Vendor Tools DBM and DB CFG parameters must be monitored on a regular basis. As workloads and environments change, these parameters will need to tuned accordingly. DB2 provides snapshot and event monitoring capabilities with the product along with additional tools introduced in DB2 UDB V8.1. Third party vendor tools are also available to assist in this area.

Snapshot Monitoring Architecture
Controlled by a series of monitoring switches Off by default (except timestamp switch) Can be enabled at the instance level or application level Each switch adds overhead on database manager NOTES: DB2 records a certain amount of information, regardless of whether switches are enabled or not. For example, Database Manager and database snapshots do not require any switches to be enabled. We’ll see this later in this section.

Snapshot Monitoring Architecture
Controlled by a series of monitoring switches Use of snapshot monitoring is DBA “first line of defense” Very difficult to support DB2 without using them Kind of like investigating a crime scene, they show evidence of wrong doing but more investigation/analysis required NOTES:

Snapshot Monitoring Authorization Required
SYSADM, SYSCNTL, SYSMAINT New registry variable in DB2 V8.1.4 DB2_SNAPSHOT_NOAUTH “db2set DB2_SNAPSHOT_NOAUTH=ON” Allows users without above authority to take snapshot NOTES:

Snapshot Monitoring Monitor Recording Switches
Switch list for db partition number 0 Buffer Pool Activity Information (BUFFERPOOL) = ON :02: Lock Information (LOCK) = ON :02: Sorting Information (SORT) = ON :02: SQL Statement Information (STATEMENT) = ON :02: Table Activity Information (TABLE) = ON :02: Take Timestamp Information (TIMESTAMP) = ON :02: Unit of Work Information (UOW) = ON :02: NOTES: Before we go into the details of snapshot monitoring, we need to understand how to review and set Database Manager monitor switches. We can see the current settings of monitor switches by issuing “GET DBM MONITOR SWITCHES” and reviewing the output. The output of the command shows the status of the seven monitor switches at the database manager level. By default all switches are turned off (with the exception of the timestamp switch new in V8.1). A switch that is on at this level has been turned on by a monitoring application. The date and time that the monitoring switch was enabled is also shown. Monitor switches can be turned on by a DBA with the appropriate authority by issuing the appropriate command from a command line, or through a program using the DB2 Administrative Application Programming Interface (API), or by a vendor tool. It indicates that the database system monitor component of DB2 is collecting data for any monitoring application. Furthermore, each person or application monitoring DB2 has its own logical view of monitor switches. Snapshot monitoring adds overhead to the database monitored. However it is difficult to understand what is happening without monitoring. Monitor switches can be turned on and off at the instance level or at the application level using the UPDATE MONITOR SWITCHES command. Setting the instance configuration parameter for monitor switches affects all databases within the instance. Every application connecting to a database inherits the default switches set at the instance level.

Snapshot Monitoring This page intentionally left blank
NOTES Continued: It indicates that the database system monitor component of DB2 is collecting data for any monitoring application. Furthermore, each person or application monitoring DB2 has its own logical view of monitor switches. Snapshot monitoring adds overhead to the database monitored. However it is difficult to understand what is happening without monitoring. Monitor switches can be turned on and off at the instance level or at the application level using the UPDATE MONITOR SWITCHES command. Setting the instance configuration parameter for monitor switches affects all databases within the instance. Every application connecting to a database inherits the default switches set at the instance level.

Snapshot Monitoring NOTES: This table contains the list of Database Manager and Database snapshot monitoring switches and resource activity collected.

Snapshot Monitoring Snapshot monitor switches can be turned on and off as needed. Use the UPDATE DBM CONFIGURATION USING DFT_ON_TABLE ON or UPDATE MONITOR SWITCHES USING TABLE ON command. NOTES: When a monitor switch is turned on, the monitor data starts being collected. There is a minor performance degradation when the data is recorded and taking snapshots incur additional performance overhead as previously discussed. It is important to note that monitor switches must be enabled before issuing a snapshot command. If the appropriate monitor switches are not enabled at either the instance or database level, DBM and DB information is available but not much is available for other resources as shown in the Monitoring Switch matrix.

Snapshot Monitoring It is important to note that monitor switches must be enabled before issuing a snapshot command If the appropriate monitor switches are not enabled at either the instance or database level, DBM and database information is available but not much other information is available Refer to the Monitoring Matrix for complete details NOTES:

NOTES: Refer to Example 43
NOTES: Refer to Example 43. This is a useful snapshot monitor switch matrix that depicts what information is and is not returned depending on the switches enabled.

Snapshot Monitoring There are eight snapshot levels available as follows: Database – Records information at the database level Database Manager – Records information at the instance level Application – Records application information Bufferpools – Records bufferpool activity Tablespace – Records tablespace activity Table – Records table activity Lock – Records lock information for locks held by applications Dynamic SQL cache – Records point-in-time statement information from the SQL statement cache NOTES: NOTE: DB2 v8 introduced a new TIMESTAMP monitor switch. By default this switch is enabled. If timestamp information is not required, this switch can be turned off resulting in improved database performance. This switch was introduced to reduce the number of context switches taking place.

Snapshot Monitoring NOTES: This page includes a table of frequently used snapshot commands. It does not include the complete syntax but is designed to be used as a quick reference. Refer to the Command Reference and System Monitor Guide and Reference for complete details.

Database Manager Snapshot
A DBM snapshot can be taken by issuing the following command from the CLP: db2 get snapshot for database manager NOTES: See Database Manager Snapshot. (Refer to Example 1 in the Example Appendix)

Snapshot Monitoring The Database Manager Snapshot output has changed significantly in v8. Heap memory areas are now separately reported for the various heaps. These are identified by the element “Memory Pool Type” followed by the heap being reported. From the previous example,we can see that the package and catalog cache entries have changed. NOTES: Refer to Example 1.

Snapshot Monitoring Key Database Manager snapshot elements to review and to monitor on a regular basis are: Post threshold sorts Pipe sort requests requested Pipe sort requests rejected Agents waiting for a token Agents stolen from other applications Max agents overflow Hash joins after heap threshold exceeded Gateway connection pool agents stolen Size of package cache Size of catalog cache Size of lock managers heap Size of database heap NOTES: Refer to Database Manager Snapshot (Example 1).

Database Snapshot Key Database snapshot elements to review, monitor, and track on a regular basis are: Lock waits Time database waited on locks Deadlocks detected Lock escalations Lock timeouts Sort overflows Bufferpool data physical reads Bufferpool data writes Bufferpool index physical reads Dirty page steal cleaner triggers NOTES: Refer to Example 2.

Database Snapshot (continued)
Key Database snapshot elements to review, monitor, and track on a regular basis are: Dirty page threshold cleaner triggers Direct reads Direct writes Database files closed Failed statement operations Internal rollbacks Internal rollbacks due to deadlocks All row metrics Package cache overflows Catalog cache overflows Number of hash join overflows NOTES: Refer to Example 2.

DBM and Database Performance Indicators
NOTES: Refer to Example 1 and 2.

DBM and DatabasePerformance Indicators
NOTES:

Application Snapshots
Use application snapshots to monitor details of application activity. This snapshot enables you to identify applications consuming large amounts of CPU and memory. Locking, bufferpool, and agent activity are provided for each active application. This information can be used in conjunction with the “db2 list applications show detail” command to identify problems. NOTES: See application snapshot. (Example 3) and Example 4. “db2 list applications show detail” followed by application snapshots as follows: “db2 get snapshot for applications on sample” “db2 get snapshot for applications agentid 16” Output from “list applications show details” command. Application snapshots are useful for determining the following: Number of locks held by an application. Lock waits and time application waited on locks. Sorts and sort overflows, bufferpool activity, row activity, type of dml activity, unread prefetch pages. The snapshot gives an overall wealth of information regarding application activity, but it is only a point in time and if “classic” is not saved anywhere. Typically, a third party vendor tool is required in order for DBAs to make use of application snapshots. New SQL snapshot function can be used to capture this data to a DB2 table!

Tablespace Snapshot Tablespace snapshots are very useful for identifying categories of tablespace activity such as asynchronous and synchronous read and write activity, bufferpool activity, logical and physical reads from which a tablespace bufferpool hit ratio can be computed, and direct reads and writes Top tablespaces in the database can be identified and targeted for tuning “db2 get snapshot for tablespaces on sample” NOTES: Refer to Example 5 for an example of a Tablespace snapshot. Top tablespace performance indicators are: Asynchronous Pages Read Per Request (APPR) (Asynchronous pool data page reads + Asynchronous pool index page reads) / (Asynchronous read requests) Using the values found in the tablespace snapshot, Example 5., APPR = ( ) / 17 = 7.6 APPR is a measure of preftech effectiveness. It should be equal or close to the EXTENTSIZE of the tablespace under review. Asynchronous Read Percentage (ARP) ((Asynchronous pool data page reads + Asynchronous pool index page reads) * 100) / (Buffer pool data physical reads + Buffer pool index physical reads) Using the values found in Example 5 for table space SYSCATSPACE, ARP = ((108+21)*100)/( ))= 12% Asynchronous Read Percentage (ARP), together with Synchronous Read Percentage (SRP = 100 – ARP), indicates whether an object, buffer pool or table space, is predominantly randomly read or asynchronously read (prefetched). Simply stated, the Asynchronous and Synchronous Read Percentages will effectively help you assign table space objects to appropriate buffer pools. Table spaces that are randomly read (high SRP) should be assigned to buffer pools with random objectives. Table spaces that are asynchronously read (high ARP) should be assigned to buffer pools with asynchronous objectives.

Table Snapshot Table snapshots are useful for identifying the most active tables via rows read and rows written, and tables with page reorgs and overflows This snapshot can be used in conjunction with the tablespace snapshot to identify active tables per tablespace NOTES: Refer to Example 6 for an example of a table snapshot. Performance indicators of interest from the snapshot on tables are: Page reorgs and overflows. Page reorgs – occur when updates to rows cause row lengths to increase, the page may have enough space to accommodate the new row, but a page reorg may be required to defragment the space. Overflows – occur when rows with varying length columns are updated, which causes an overflow record to be created which causes overflow accesses during reads.

Bufferpools Accessing memory is extremely fast, as most memory chips commercially available deliver performance that is measured in nanoseconds, often 70ns or faster. Contrast this to the time typically required to access disk storage devices, which is commonly measured in milliseconds. Good disk performance is typically measured in the 3ms to 7ms range, suggesting that accessing memory for data is about 1000 times faster than accessing disk storage NOTES:

Bufferpool Snapshot For demonstration purposes this is an example of a bufferpool snapshot without the bufferpool snapshot switch enabled As you can tell from this page and the next, basically no data is available without the monitoring switch enabled NOTES: Bufferpool Snapshot. Bufferpool name = IBMDEFAULTBP Database name = SAMPLE Database path = /home/db2inst1/db2inst1/NODE0000/SQL00001/ Input database alias = SAMPLE Buffer pool data logical reads = Not Collected Buffer pool data physical reads = Not Collected Buffer pool data writes = Not Collected Buffer pool index logical reads = Not Collected Buffer pool index physical reads = Not Collected Total buffer pool read time (ms) = Not Collected Total buffer pool write time (ms) = Not Collected Asynchronous pool data page reads = Not Collected Asynchronous pool data page writes = Not Collected Buffer pool index writes = Not Collected Asynchronous pool index page reads = Not Collected Asynchronous pool index page writes = Not Collected Total elapsed asynchronous read time = Not Collected Total elapsed asynchronous write time = Not Collected Asynchronous read requests = Not Collected Direct reads = Not Collected Direct writes = Not Collected

Bufferpool Snapshot continued
This page intentionally left blank NOTES: Direct read requests = Not Collected Direct write requests = Not Collected Direct reads elapsed time (ms) = Not Collected Direct write elapsed time (ms) = Not Collected Database files closed = Not Collected Data pages copied to extended storage = Not Collected Index pages copied to extended storage = Not Collected Data pages copied from extended storage = Not Collected Index pages copied from extended storage = Not Collected Direct reads = Not Collected Vectored IOs = Not Collected Pages from vectored IOs = Not Collected Block IOs = Not Collected Pages from block IOs = Not Collected Physical page maps = Not Collected Node number = 0 Tablespaces using bufferpool = 3 Alter bufferpool information: Pages left to remove = 0 Current size = 1000 Post-alter size = 1000

Bufferpool Snapshot Issued “db2 get monitor switches” after not receiving any output from previous snapshot Status of monitoring switches: Monitor Recording Switches Switch list for db partition number 0 Buffer Pool Activity Information (BUFFERPOOL) = OFF Lock Information (LOCK) = OFF Sorting Information (SORT) = OFF SQL Statement Information (STATEMENT) = OFF Table Activity Information (TABLE) = OFF Take Timestamp Information (TIMESTAMP) = ON :04: Unit of Work Information (UOW) = OFF NOTES:

Bufferpool Snapshot Monitor Recording Switches
Issued the command to enable the bufferpool monitoring switch “db2 update monitor switches using bufferpool on” Monitor Recording Switches Switch list for db partition number 0 Buffer Pool Activity Information (BUFFERPOOL) = ON :48: Lock Information (LOCK) = OFF Sorting Information (SORT) = OFF SQL Statement Information (STATEMENT) = OFF Table Activity Information (TABLE) = OFF Take Timestamp Information (TIMESTAMP) = ON :04: Unit of Work Information (UOW) = OFF Then issued the “db2 get snapshot for bufferpools on sample” snapshot command (Refer to Example 7) NOTES: Refer to Example 7 and 7A for an example bufferpool snapshot. Bufferpool snapshots are good for evaluating bufferpool activity and performance. Note previous APPR formula is applicable to tuning bufferpools. Tablespace and bufferpool snapshots are similar, however, bufferpool snapshots give us a picture of overall bufferpool activity for all tablespaces defined to that particular bufferpool. Key performance indicators for bufferpools are: Index Hit Rate – (Buffer pool index logical reads * 100) / (Buffer pool index logical reads + Buffer pool index physical reads) Using the values found in Example 7, IHR = ((14345 * 100)/( )) = 96% A page that was already in the buffer pool satisfied 96 out of 100 logical read requests. The higher the hit ratio, the less frequently DB2 needs to access disk devices to bring pages into the buffer pool. A high index hit ratio is especially important to transactional OLTP applications, including E-commerce web sites.

Bufferpool Snapshot Key Performance Indicators Continued NOTES:
Bufferpool hit ratio (Overall): ((Buffer pool data logical reads + Buffer pool index logical reads) * 100) / (Buffer pool index logical reads + Buffer pool index physical reads + Buffer pool data logical reads + Buffer pool data physical reads) Using the values found in Example 7, OHR = (( ) * 100) / ( ) = 96% The overall bufferpool hit ratio is usually lower than the IHR as there are usually many more data pages than index pages and data pages may be accessed more often. Physical Pages Read Per Minute (PPRPM) – (Buffer pool data physical reads + Buffer pool index physical reads) / Number of elapsed minutes since monitor switches activated or reset PPRPM is useful for identifying tablespaces that are more active than others, indicating that I/O may not be spread evenly across all tablespaces. Or there may be unusual activity on certain tablespaces cuasing more I/O than other tablespaces.

Bufferpool Snapshot Key Performance Indicators Continued NOTES:
Average Read Time is a measure of how long DB2 agents wait for a page or pages to be read into the bufferpool (application waits while the agent waits) ART= (Total bufferpool read time) / (Data physical reads + Index physical reads)

Bufferpool Snapshot Block-based bufferpools
New in DB2 UDB V8.1 Causes non-contiguous pages to be prefetched into contiguous area of memory BLOCKSIZE and NUMBLOCKPAGES specified on create or alter bufferpool command NUMBLOCKPAGES used to specify amount of bufferpool to be reserved for sequential blocks Cannot exceed 98% of bufferpool size Use for bufferpools with sequential “characteristics” NOTES: New in V8.1, block-based bufferpools improve the performance of data that is sequential in nature (BI/DW) by prefetching blocks of non-contiguous data into bufferpool pages that are contiguous. The BLOCKSIZE should equal the EXTENTSIZE of the tablespace so that space is not wasted. The largest value for NUMBLOCKPAGES can be found using the following formula: NUMBLOCKPAGES= (Floor(Floor(BP #Pages x 98/100)/ Blocksize #Pages Since not all I/O is sequential in a BI/DW database, consider reserving 60% of the bufferpool size for sequential I/O by substituting 60% for 98% in previous formula.

Lock Snapshot The snapshot for locks shows all locks held in the database by application handle and application id, along with the number of locks held and the status of the application Used in conjunction with “list applications show detail” command may enable you to identify and solve locking problems More than likely, a deadlock event monitor with details will be more helpful NOTES: Refer to Example 8 for a lock snapshot. “db2 get snapshot for locks on sample”. Combined with list applications show detail and application snapshots can be used to identify and troubleshoot current locking problems. Difficult to use without third party vendor tool. Although new SQL snapshot functions help.

Dynamic SQL Snapshot Dynamic SQL snapshots add overhead to the database manager and should be used only as needed. Output should be written to a file for detailed analysis. Upi can use Dynamic SQL snapshots to find and investigate SQL statements with high costs, high number of rows read, and sorts. NOTES: See Dynamic SQL snapshot. (Example 9)

Resetting Monitoring Switches
An application can reset monitor switches, which in effect resets the counters to 0 for the application issuing the reset command. Note: An application in this respect could be the CLP, Command Center, Third Party Vendor Tool, or a user-written application. This can be accomplished by an application issuing the RESET MONITOR FOR DATABASE <database name> command. The GET MONITOR SWITCH command can be used to display the current status of monitoring switches. The RESET MONITOR ALL command can be used to reset the monitor switches for all databases in an instance. NOTES:

Snapshot Monitoring Every application has its own copy of the snapshot monitor values. Resetting the monitor switches only effects the counters of the application that issues the reset. NOTES:

Snapshot Monitoring via New SQL Functions in V8.1
New in v8 is the ability to issue snapshot commands via SQL functions. These 15 new functions make it easy for you to write programs that issue snapshot commands and process the snapshot data. They can also be issued via the CLP. Previously this was only available via the administrative API. This is a real breathrough in monitoring capability in DB2 and along with write-to-table event monitors provides a historical repository for use in reviewing performance problems, trend and historical analysis NOTES:

SQL Snapshot Functions
Below is the syntax and examples can be found in the next few pages: SNAPSHOT_BP(<dbname>, <partition>) Returns bufferpool information SELECT DIRECT_READS FROM TABLE(SNAPSHOT_BP('GUNNDB', -2)) AS S; NOTES:

NOTES: The table on this page contains a list of new SQL snapshot functions in DB2 UDB V8.1. Refer to Example 10, SQL Snapshot Function Table in the Exercises and Example Appendix.

SQL snapshot functions provide DBAs with a way to integrate snapshot history into overall monitoring strategy Can be included in a script and run periodically throughout the day, recording snapshot output in DB2 tables Can be used for problem determination and for trending and historical purposes NOTES:

SQL Snapshot Table DDL NOTES: Sample DDL to create a table to store SQL snapshot output for bufferpool snapshot. Refer to the Snapshot table DDL in the Exercises and Example Appendix.

Sample insert statement for using an SQL snapshot function and inserting the output into a DB2 table: INSERT INTO BP_SNAP SELECT * FROM TABLE( SNAPSHOT_BP( ‘sample', -1 )) as SNAPSHOT_BP; NOTES: The format of SQL snapshot function tables can be found in the DB2 UDB System Monitor Guide and Reference. They can also be constructed by reviewing the SQL snapshot function definitions stored in SYSCAT.ROUTINEPARMS.

NOTES:

Snapshot Monitoring Summary
Snapshot monitoring shows us performance data at a point-in-time. With snapshots, we may or may not capture the information of interest depending on the time we take the snapshot, and whether or not the event we are trying to capture is running or has completed and the frequency of the snapshot taken. Many times snapshot data gives you enough data to identify a suspected problem and an event monitor is required to capture the complete picture. Hence event monitors should be used when snapshot data is inconclusive and further data capture is required.

Snapshot Monitoring Summary
As demonstrated, Tablespace and Bufferpool activity is better analyzed using snapshots which provide adequate data to properly monitor and tune bufferpools and tablespace activity. However, for other data such as SQL Statements and connection and application activity, event monitoring is requried in addition to snapshot monitoring

Snapshot Monitoring Workshop
Refer to your student workbook and examples for the Snapshot Monitoring Workshop. You have 45 minutes to complete the workshop.

Module 3 Event Monitoring

Understand DB2 Event Monitoring and the different types of event monitors Be able to create, run and stop event monitors Be able to format and interpret event monitor output Be able to discern when to use snapshot monitoring, event monitoring, and the correct type of event monitor NOTES:

Event Monitoring Event monitoring is used in DB2 to capture complete data regarding DB2 events. Unlike snapshots, which are capturing data at a point-in-time, event monitors are like a trace and record data for all event types at event completion. This has some implications that you need to be aware of such as: More overhead than snapshot monitoring Data captured to files or pipes is not in a viewable format Has to be formatted using db2eva utility Write to table data is viewable and requires no formatting besides your user queries Event monitoring is needed in many cases to accurately identify performance problems NOTES: Most commonly used event monitors for problem solving are connection and statement event monitors which capture activity at the connection and application level.

Event Monitoring Create Event Monitor Options
Options can be specified on the CREATE EVENT MONITOR statement to control the amount of information collected, to determine how the event monitor stops and starts, to specify the location and size of output files or pipes, types and size of buffers, and scope of the event monitor (local or global). NOTES:

Event Monitoring Authorization Required SYSADM or DBADM NOTES:

Event Monitoring Event monitors can be created that write to three possible types of destinations Write to pipe (requires a process to be reading the pipe) but low overhead Write to file(s) – number and size of files specified on create event monitor command Write to table – powerful and flexibile but these tables are logged just like any other DB2 persistent table NOTES: As noted, there are three basic types of event monitors. Write to table event monitors provide similar capability as Oracle™ system tables, but in DB2 write to table event monitors are under DBA control.

Event Monitoring Write to table – powerful and flexible but these tables are logged just like any other DB2 persistent table! However, write-to-table event monitors much easier to use than other types NOTES:

Write-to-table Event Monitors
Can use to support development efforts and once you have developed “canned” SQL to query the tables, it is very easy to review and asses performance and adequacy of SQL and resources Third party vendor query tools also can be used to speed analysis of data NOTES:

Event Monitoring The following command can be used to create a connection event monitor that uses default values and writes-to tables: CREATE EVENT MONITOR PGUNN FOR CONNECTIONS WRITE TO TABLE; NOTES: Event monitors are started and stopped by setting the STATE to 1 or 0 respectively. SET EVENT MONITOR <event monitor name> STATE = 1 SET EVENT MONITOR PGUN2 STATE = 0

Event Monitoring Event monitors capture and record data as events complete. DB2 provides eight types of event monitors. They are: Database Tables Deadlocks Tablespaces Bufferpools Connections Statements Transactions NOTES: New in v8, event monitors can write output directly to event monitor tables. Use the DB2EVTBL tool to create and customize event monitor output and tables.

Event Monitoring Tablespace Events Bufferpools Events
Records an event record for each active tablespace when the last application disconnects from the database. Bufferpools Events Bufferpool event monitor record an event record for bufferpools when the last application disconnects from the database. NOTES: Database event monitors record an event record when the last application disconnects from the database. NOTE: Event monitors can be customized to capture events based on APPL_ID, AUTH_ID, and APPL_NAME event conditions. Table Events Table event monitors record an event record for each active table when the last application disconnects from the database. An active table is a table that has changed since the first connection to the database. Deadlocks Events Deadlock event monitors record an event record for each deadlock event. When the WITH DETAILS option is used, the event monitor will generate a more complete deadlock connection event for each application involved. Additional details include: The statement text from the application that was executing when the deadlock occurred The locks held by the application when the deadlock occurred. In a partitioned database environment, the locks are only included for the database partition where the application was waiting for its lock when the deadlock occurred.

Event Monitoring Connection Events Statement Events
Connection event monitors record an event record for each database connection event when an application disconnects for the database. Statement Events Statement event monitors record an event for every SQL statement issued by an application (for both dynamic and static SQL). Transactions Events Transaction event monitors record an event record for every transaction when it completes (indicated by a COMMIT or ROLLBACK statement). NOTES: Event Monitor Creation Event monitors are created via the CLP or Control Center as a database event object. Event monitor can write event records to pipes, files, or tables. The type of output mechanism is indicated in the Create Event monitor SQL statement. Use the following criteria to determine what type of output mechanism you should use: Transaction volume Planned use for the event data Throughput required CPU capacity Disk storage available Use of Third Party Vendor tools To create an event monitor, you must have either DBADM or SYSADM authority.

SQL Events 25-33% of tuning lies in database, tablespace, and bufferpool tuning 67-75% or more tuning opportunities lie within individual SQL statements Reduce the cost of most expensive SQL, and entire application runs faster. It doesn’t matter how good a job you do on the database infrastructure. If some badly performing SQL gets into the application, the application and possibly the entire database will begin to perform beneath expectations. Transactional SQL has to be “short and sweet” with efficient, low cost, access plans.

Statement Event Monitor
create event monitor STMNTS for statements write to file ‘/home/db2inst1/event/out’ maxfiles 5 maxfilesize 1000 blocked append manualstart; set event monitor STMNTS state = 1; To find SQL statement costs, create a statement event monitor, then turn it on. The sample create event monitor statement provided on this page will capture up to 20MB of raw SQL performance trace data. Depending on Transaction Rates, and SQL per Transaction, this monitor could run for many hours or only a few minutes before automatically shutting itself off.

Statement Events set event monitor STMNTS state = 0;
Now that we have captured SQL Event Data: Use DB2EVMON to format it Use Control Center tool to look at it Third party vendor tool Shut the event monitor off, if it hasn’t already shut off automatically… this flushes the buffers. Next, use IBM supplied program DB2EVMON to format the Statement event monitor data.

db2evmon Format the data as follows:
db2evmon –path ‘/home/db2inst1/event/out’ > stmtrpt.txt - or - db2evmon –db SAMPLE –evm STMNTS > stmtrpt.txt edit/view/more stmtrpt.txt db2eva

db2evmon output 3541) Statement Event ... Appl Handle: 297
Appl Id: A02CCD85.042B Appl Seq number: 0001 Type : Dynamic Operation: Close Section : 297 Creator : NULLID Package : SQLL1B0N Cursor : SQLCURCS297 Text : SELECT MERCHANT.MECNTRY FROM MCUSTINFO MCUSTINFO, MERCHANT MERCHANT, SHOPPER SHOPPER WHERE SHOPPER.SHRFNBR = MCUSTINFO.MCSHNBR and MERCHANT.MERFNBR = MCUSTINFO.MCMENBR and SHOPPER.SHRFNBR = Start Time: :43: Stop Time: :43: Number of Agents created: 1 User CPU: seconds System CPU: seconds Fetch Count: 1 Sorts: 2 Total sort time: 0 Sort overflows: 0 Rows read: 2 Rows written: 0 Internal rows deleted: 0 Internal rows updated: 0 Internal rows inserted: 0

Formatting Event Monitor Output
NOTES: db2eva – Event Analyzer utility can be used to format event monitor data written to tables.

Statement Event Monitor
The statement event output will show a great deal of cost information including CPU, Sort, Fetches, Rows Read/Written, and internal rows DML. Elapsed time can be computed by finding the difference between Start and Stop times. Bufferpool detail (logical, physical, async reads/writes) is not available at statement event level. It is also not readily apparent which tablespaces are accessed.

Connection Event Monitors
Connection events provide true cost of sort Use in conjunction with statement event monitors to identify suboptimal SQL and sort problems Use during periods of suspected activity to identify problems, then deactivate monitors Can be run more often but volume of data produced and overhead can be prohibitive NOTES:

Deadlocks Event Monitor With Details
New Deadlock event monitor with details Activated by default at DB2 installation DB2DETAILDEADLOCK Deactivate this right away as it imposes significant overhead Use only as needed New “with details” option provides finer granularity for offending SQL, object, and lock details NOTES:

When a write-to-table event monitor is created, by default the following thirteen event monitor tables are created: CONNHEADER DEADLOCK DLCONN CONTROL DLLOCK STMT SUBSECTION XACT CONN DB BUFFERPOOL TABLESPACE TABLE NOTES: If only a subset of events are specified on the CREATE EVENT MONITOR statement, then DB2 will only create tables needed for those events. DB2 uses the following naming convention when creating event monitor tables: schemaname.table name_event_monitor_name for example, if a DBA with a user id of PRODBA issued the following create event monitor statement: CREATE EVENT MONITOR PGUN2 FOR CONNECTIONS WRITE TO TABLE then the event monitor tables will be created with a schema of PRODDBA and with a table suffix of PGUN2 appended to the event monitor table name. The event monitor tables would be created as follows: proddba.conn_pgun2 proddba.connheader_pgun2 proddba.control_pgun2 If not specified, the tables will be created in tablespaces as specified by the IN tablespaceName clause. If specified, the tablespaces must already exist. THE CREATE EVENT MONITOR statement does not create tablespaces.

NOTES: New write-to-table event monitor tables in DB2 V8.1. For each event monitor created, the associated tables are created as part of the create event monitor statement. Note that if the event monitor is dropped, the tables remain. They must be dropped manually. Also, there is no automatic purge of data stored in these tables, so you’ll want to purge these based on your criteria. Nonetheless, this new feature is a very welcome and powerful feature that, along with SQL snapshot functions, provides DB2 with a true historical capability. This has been sorely needed in past releases and has been high on most DBAs list of things to add to DB2. Now you have a capability to store data for trending and historical purposes. Plus, it can be used to solve routine problems. For example, if the help desk calls you with a DB2 problem that occurred 5 days ago, if you are using these features you can query the event monitor or snapshot tables for the data of interest. This is truly a breakthrough capability that is now in DB2.

General Consideration for Write-to-Table Event Monitors When the CREATE EVENT MONITOR statement is issued, all event monitor target tables are created. If the creation of a table fails for any reason, an error is passed back to the application program and the CREATE EVENT MONITOR statement fails. During CREATE EVENT MONITOR processing, if a table already exists, but is not defined for use by another event monitor, no table is created, and processing continues. A warning is passed back to the application program. NOTES: For write-to-table event monitor, the LOCAL and GLOBAL keywords are ignored because an event monitor output process or thread is started on each database partition in the instance, and each of these processes reports data only for the database partition on which it is running. db2evtbl tool can be used to create specific write-to-table event monitors. A target table can only be used by one event monitor. During CREATE EVENT MONITOR processing if a target table is found to have already been defined for use by another event monitor, the CREATE EVENT MONITOR statement fails, and an error is passed back to the application program. You can tell if a table is already defined for use by another event monitor if the table name matches a value found in the SYSCAT.EVENTTABLES catalog view. NOTE: DB2 does not drop target tables as part of the DROP EVENT MONITOR statement.

Can also use “filters” with event monitors NOTES: While an event monitor is running it inserts event records into target tables. If an insert fails, uncommitted changes are rolled back, a message is written to the administration notification log, and the event monitor is deactivated. TIP: Make sure the tablespaces being used for your event monitor target tables have enough space to handle your requirements. For DMS tablespaces, the PCTDEACTIVATE option can specify as a percentage how full the table space can be before the event monitor automatically deactivates.

Event Monitoring The following command can be used to create a connection event monitor that uses default values and writes-to tables: CREATE EVENT MONITOR PGUNN FOR CONNECTIONS WRITE TO TABLE; NOTES: $ db2 create event monitor pgunn for connections,statements write to table DB20000I The SQL command completed successfully. $ db2 set event monitor pgunn state=1 Event monitors are started and stopped by setting the STATE to 1 or 0 respectively. SET EVENT MONITOR <event monitor name> STATE = 1 SET EVENT MONITOR PGUN2 STATE = 0

Creating Event Monitors
create event monitor BADONE for statements write to file ‘/home/db2inst1/events/eventout’ maxfiles 10 maxfilesize 1000 blocked append manualstart; Activate the event monitor by setting its state to 1 set event monitor BADONE state = 1; Deactivate the event monitor by setting state to 0 NOTES: Commands to activate an event monitor that writes to files. Event monitors are objects in DB2 and are accessed through the syscat.eventmonitors DB2 catalog view.

Can create via command line as previously outlined or via Control Center NOTES:

NOTES:

NOTES: <evmGroup> ( [(TABLE <tableName>,] [IN <tablespaceName>,] [PCTDEACTIVATE <integer>,] [TRUNC] [INCLUDES|EXCLUDES (elem1, elem2, ..)] ), ....

NOTES:

This page intentionally left blank NOTES:

Module 4 Tuning Sorts

Identify and eliminate sorts Understand sort terminology Use event monitors to identify SQL statements causing sort problems Understand sort overflows and learn how to prevent them from occurring NOTES:

Sorts We humans need data returned in some type of order
Sorts occur to accomplish this when data is not in order, indexes are not available to return the data in order, or distinct is used to eliminate duplicates Max or Min functions not supported by index Sorts are usually a major problem in all environments NOTES:

Key Sort Indicators SORT VOCABULARY Sort Size Sort Heap Sort Overflows
Sort Heap Threshold Sort Capacity Post Threshold Sorts Private Sort Shared Sort SHEAPTHRES SHEAPTHRES_SHR Sort Size: the row width/bytes multiplied by the number of rows to be sorted Sort Heap: That special memory place where sorts happen without physical I/O Sort Overflow: Should the sort size exceed the Sort Heap, the data to be sorted spills over (overflows) into TEMPSPACE tablespace via the bufferpool Sort Heap Threshold: DB2’s allowance for total sort memory within an instance Sort Heap Threshold / Sort Heap Size = Sort Capacity = Number of Eligible sorts prior to post threshold sort Post Threshold Sorts - What happens when DB2 exceeds its sort memory “allowance” SHEAPTHRES – controls amount of shared and private sorts that can be performed before post threshold sorts occur SHEAPTHRES_SHR – If intra_parallel enabled, can be used to control shared sorts at database level (New in V8.1)

Configuration Settings that affect your SORT physical memory
INTRA_PARALLEL = YES, Shared Memory Decision Support Databases Large Data Volume Queries NO, Private Memory OLTP/Web based/Mixed Applications (DEGREE = 1) DB2MEMDISCLAIM (AIX) = YES DB2MEMMAXFREE = SHEAPTHRES and SHEAPTHRES_SHR BUFFERPOOL SIZES

NonOverflowed Sort (Piped)
Agent Bufferpool Disk Sortheap NOTES: This page depicts graphically how piped sorts perform. They do not overflow to disk. System Temporary Space A piped sort completes within sortheap without overflowing to bufferpool and disk

Overflowed Sort Bufferpool Sortheap Disk System Temporary Space NOTES:
This page depicts graphically how non-piped sorts perform. They overflow through the bufferpool to disk. This can happen multiple times during merge phase. Note, temporary tables are allocated completely. Multipage file allocation does not effect sort temporary tables. System Temporary Space

Sampling SORT Performance
Sources of SORT Data Database Snapshots Application Snapshots Connection Events SQL Events DB2 “GET SNAPSHOT FOR DATABASE ON DBNAME” DB2 “GET SNAPSHOT FOR DATABASE MANAGER” Database Snapshot Total sort heap allocated = 0 Total sorts = 237 Total sort time (ms) = 10190 Sort overflows = 29 Active sorts = 0 Number of hash joins = 0 Number of hash loops = 0 Number of hash join overflows = 0 Number of small hash join overflows = 0 Database Manager Snapshot Sort heap allocated = 0 Post threshold sorts = 0 Piped sorts requested = 181 Piped sorts accepted = 181

SORT Formulas % SHEAPTHRES Allocated = Sort Heap Allocated * 100 / SHEAPTHRES If > 90%, larger SHEAPTHRES or smaller SORTHEAP If Post Threshold Sorts > 0 (rule above violated), same remedies Average Sort Time = Total Sort Time(ms)/Total Sorts % SORT Overflows = # Overflows * 100 / Total Sorts If OLTP & If > 3%, cure sorts or increase SORTHEAP Sorts / TX, Sorts / SQL, Sort Time / TX, “TCA” High Water Sort Heap Used, Average Sort Heap Used SORT RQMNT Per Connection = Average Sort Heap Used / Average Number Connections Executing Connection Events show true costs of sorts.

Measuring SORT Performance
Connection Events provide true cost of Sort activity for an Application create event monitor GUNCONN for connections write to file 'e:\tmp\dbaudit\conn\' maxfiles 1 maxfilesize 1024 blocked replace manualstart; Set event monitor GUNCONN state = 1; Set event monitor GUNCONN state = 0; SQL Events provide true cost of Sort activity for an individual statement create event monitor GUNSQL for statements write to file 'e:\tmp\dbaudit\sql\' maxfiles 1 maxfilesize 2048 blocked replace manualstart; set event monitor GUNSQL state = 1; set event monitor GUNSQL state = 0; db2evmon -path e:\tmp\dbaudit\DIRNAME

Attendee Notes File - Pipes
Blocked - ensures no data loss at the expense of possible performance delays, should the buffer become full Nonblocked - Fastest capture of event data, but event data could be lost if buffers become full Pipes Memory address used to pass event data from buffers to application program reading pipe, extremely fast. “nonblocked” is only option Write-to-table event monitors new in V8.1

Sort Consumption dynexpln -g -d DBNAME -q “Select * from … order by …”
| Insert Into Sorted Temp Table ID = t1 | | #Columns = 31 | | #Sort Key Columns = 1 | | | Key 1: NAME (Ascending) | | Sortheap Allocation Parameters: | | | #Rows = 1981 | | | Row Width = 260 | | Piped ((260+8) x 1981) = 530,908 bytes / 4096 = K Pages = 530,908 / 1,048,576 = .51 MB …. 50% of default 256 4K SORTHEAP The SORTHEAP requirement for an SQL statement can be learned from Explain. Some SQL statements perform several sorts of varying sizes to arrive at the result set. If Rows x Row Width > SORTHEAP size, an overflow will occur to TEMPSPACE via bufferpool Sorts don’t overflow “just a little bit”, they overflow in their entirety. Hash Joins overflow only to the extent necessary. If > 5% Hash Joins have small overflows, increase SORTHEAP size.

Attendee Notes Sorts that cannot be completed within a SORTHEAP will overflow to TEMPSPACE via the bufferpool. Recognizing that some number of Overflow sorts will invariably occur, there are several tuning opportunities to optimize overflow behavior. Dramatic sort performance improvements can be achieved by adhering to essential principles: Container Placement Number of Containers Prefetch Sizes TEMPSPACE Bufferpool

Design Advisor db2advis -d peiprd -I wildsortsql.txt > wildsqlixadvout.txt execution started at timestamp found [1] SQL statements from the input file Calculating initial cost (without recommmended indexes) [ ] timerons Initial set of proposed indexes is ready. Found maximum set of [1] recommended indexes Cost of workload with all indexes included [ ] timerons total disk space needed for initial set [ ] MB total disk space constrained to [ ] MB 1 indexes in current solution [ ] timerons (without indexes) [ ] timerons (with current solution) [%99.69] improvement Trying variations of the solution set.-- -- execution finished at timestamp -- LIST OF RECOMMENDED INDEXES -- =========================== -- index[1], MB CREATE INDEX WIZ1 ON "PUSER "."T_FILE" ("FILE_STATUS_NUM" DESC) ; -- ===========================-- Design Workload tool is finished. 2

Tuning Sort Summary Tuning sorts is a major part of your job as a DBA. Queries and reports require data to be in order to support business rules and objectives SQL requires sorts to be performed if indexes or clustering are not being used to return data in the order required Sorts that aren’t tuned can overflow to disk and significantly degrade response time and consume excessive CPU NOTES:

Tuning Sort Summary OLTP and Web-based applications should not use sorts that overflow due the stringent response time requirements that must be met Sort overflows should be 0 or close to 0 in this case BI/DW and Mixed environments perform larger sorts due to the complexity of the SQL and amount of data, hence, sort overflows are likely to occur Use the same tuning techniques to eliminate sorts as in OLTP but also tune the I/O subsystem to improve sorts that overflow NOTES:

Event Monitoring and Sort Tuning Workshop
Refer to your student workbook and examples for the Event Monitoring and Sort Tuning Workshop. You have 1 hr to complete the workshop. NOTES:

Module 5 DB2 Architecture

OBJECTIVES DB2 Architecture Why Tune? How is memory used?
Configuration Parameters Agent Related Parameters Summary NOTES:

Introduction Successful tuning requires knowledge of DB2 processing and available monitoring facilities NOTES:

Application Shared Library Communication Laver
Client Application Shared Library Communication Laver Network: NETBIOS TCP/IP named pipes Communication Laver BSU Base Support Utilities Process Model <sqe> db2lock db2bm db2agents db2fcmds db2loggr db2lfrm RDS rational data services catalogue services <sqrl> Complier Q G M APM Access Plan Manager <sqra> -parser -semantic -optimizer -code- generator Explain <sqnx> NOTES: This chart depicts the DB2 Architecture and associate key components. Utilities: -import, export <sps> -load <squ> -backup, restore, rollforward <sql>

RTI run-time interpreter <sqri>
Relational operations -aggregrations -group by -hash join -nested-loop join -math -UDF -table queue -bloom filter DMS data management services Sort <sqs> -index manager <sqx> -long,large object Manager <sqdl, sqdx> -table manager <sqd> Insert, update, delete DPS data protection services <sqb> -locks -logging -transaction management NOTES: This chart depicts the DB2 Architecture and associate key components, continued.

BPM buffer pool manager <sqb>
Common services: -system monitor <sqm> -config. services <sqf> -data services <sv> Prefetchers db2pfchr Page cleaner db21frm OSS operation system services <sqo> -code page conversions –file I/O –latching -memory management –message queues -semaphores –trace –wait post OS AIX Linux Solaris HP-UX NOTES: This chart depicts the DB2 Architecture and associate key components, continued. Windows Hard Disk

DB2 Process Model NOTES: Query flow with connection concentration enabled.

DB2 Process Model NOTES: Query flow without connection concentration enabled.

DB2 UDB Process Model Client Prefetchers Coordinating Agents Memory
Logger Notes: Applications IO Cleaners

DB2 Process Model The DB2 Process Model consists of clients running applications, with agents doing work on behalf of applications. Data is prefetched into the bufferpool by prefetchers. As the bufferpool fills with dirty pages, IO Cleaners write dirty pages asnchronously to disk. The logger process writes undo/redo information to the log, and commits it from the log buffer at commit. Notes:

DB2 Prefetchers DB2 Prefetchers prefetch data into the bufferpool, in advance of the application needing it, when sequential access is being used. This prevents the application from waiting while data is retrieved from disk. The DB CFG NUM_IOSERVERS parameter specifies how many prefetchers will be available to DB2. Notes:

DB2 Prefetchers As agents request that pages be brought into the bufferpool, prefetch requests are placed on a common prefetch queue. Requests are processed from the queue using FIFO. At times, a prefetcher may not be available. In this case the agent does the prefetch requests which causes the application to wait until the request completes. This is not good! Notes:

DB2 Prefetchers As a rule of thumb, configure as many prefetchers as the number of disks. DB2 attempts to create a prefetcher for each container to obtain parallel I/O. Notes:

DB2 IOCLEANERS The DB CFG NUM_IOCLEANERS controls how many IOCLEANERS are available for a database. I/O cleaners are used to write dirty pages to disk. The CHNGPGS_THRES controls when IOCLEANERS are activated or woken up. The default threshold is 60%. That is if 60% of pages in the bufferpool are dirty, IOCLEANERS will write dirty pages to disk asynchronously. Notes: num_iocleaners should be set to in an OLTP and mixed environment. This is so we eliminate I/O spikes and gives us “trickle I/O”.

DB2 IOCLEANERS This behavior is good, as we want asynchronous writes to occur. If the bufferpool fills up with dirty pages and an IOCLEANER is not available, the agent will have to cause the synchronous write to disk to occur which results in applications having to wait till the write completes. THIS IS NOT GOOD. Configure the NUM_IOCLEANERS parameter to the number of CPUs and monitor over time Notes:

Coordinating Agents A Coordinating agent is created for each application connected to the database. The coordinator agent does work on behalf of the application. The coordinator agent may create subagents if parallelism is enabled and if work can be done in parallel. Notes:

Bufferpools Bufferpools are used by DB2 to cache frequently accessed pages in memory. For a dedicated database server, up to 75% of memory should be dedicated to bufferpools as properly sized and configured bufferpools can have a significant effect on performance Notes:

DB2 Logger The DB2 Logger is responsible to log redo/undo records and to write committed work to the log buffer and disk. The DB2 Logger works in conjunction with the Bufferpool Manager to ensure that updated pages are written to disk. In V8, we have true dual logging at the database level. This should be enabled at all times. Notes:

Module 6 Memory Model and Configuration Parameters

Understand the DB2 Memory Model Understand the heaps and parameters that affect how memory is used Understand DB2 agent parameters Be able to determine when and why parameters require tuning NOTES:

Instance Shared Memory
database_memory Bufferpools locklist pckcachesz shared sorts dbheap logbufsz catalogcache_sz util_heap_sz Update with latest V8 changes Shared Sorts Instance Shared Memory fcm buffers Database (Shared) Global Memory (numdb) Database (Shared) Global Memory (1) … appgroup_mem_sz appgroup_mem_sz Application Group Shared Memory Application Shared Memory (maxappls) (only if DPF or Intra_parallel enabled) Application Shared Memory (app_ctl_heap_sz) Application Shared Memory (1) NOTES: Successful tuning of DB2 UDB requires an understanding of various memory areas in DB2 and how they are used. Instance Manager Shared memory is used by the Database Manager to manage activity across all database connections. Database Shared memory is allocated per database. appgroup_mem_sz is new in V8.1. appgroup_mem_sz is Application Shared memory is only allocated if intra_parallel is set to yes or if running in a partitioned database enviroment (DPF). Agent private memory is allocated per agent, up to the value of maxagents. If these memory areas are under or over allocated, DB2 will not provide optimal performance. New in V8.1 is Application Group Shared Memory. .. Agent Private Memory (1) Agent Private Memory (maxagents) ..

Application Groups See notes
NOTES:An 'application' is the logical representation of a client connection to a database on the DBMS, and is created when a user initiates a connection to a particular database. Applications always belong to a particular 'application group', which is the logical representation of a set of shared resources (shared that is between apps within that app-group). A 'coordinator agent' is a physical agent process created by the server to perform a unit of work on behalf of an application/client connection. In general, coordinator agents are created by a dispatcher process when it detects a request on a client connection, and a suitable pooled coordinator agent cannot be found to service the request. If a suitable pooled coordinator agent is found, it will be used instead rather than creating a new coordinator agent process. After obtaining a coordinator agent process, the dispatcher will then pass the active client connection to the coordinator agent which is then associated with that client connection, and it's application group. The agent process will then serve requests on behalf of that application for the duration of a single transaction. The transaction ends when the client issues a commit or a rollback, at which point the coordinator agent will pass the client-connection back to the dispatcher process, and disassociate itself from that application. The coordinator agent will usually then be placed in a pool on the application's app-group for reuse by other applications performing work. When pooled, the coordinator agent remains associated with the app-group in question to improve performance of subsequent transactions.

Application Groups See notes
NOTES: In V8.1 there is a database level idle agent pool in The database level idle pool is actually comprised one or more pools for particular groups of applications. Initially, when a system starts up, there is only one Application Group. As more connections are added, DB2 creates additional Application Groups and creates an idle pool for each new Application Group — this is done automatically and transparently by DB2. There are typically applications per Application Group. Application Groups help spread out access to critical resources such as shared SQL work space across agents. This can help reduce contention in very high user throughput environments with hundreds to thousands of users.

Connection Concentrator
See notes NOTES: For Web-based or OLTP applications with many simultaneous user connections, the connection concentrator may improve performance by allowing many more client connections to be processed efficiently. It also reduces memory use for each connection, and decreases the number of operating system level context switches. This feature is intended for short running web applications, and is not intended for long running applications such as the production of big reports. When a client application connects to a database, an application control block is associated with it.

Dynamic Configuration Parameters
Deferred Get DBM or DB CFG Immediate Transaction boundary NOTES:

Autonomous Computing Self-managing and Resource Tuning (SMART)
Configuration parameters that can be set to automatic NOTES:

Database Manager Shared Memory Database Global Memory
Monitor heap (mon_heap_sz) Audit buffer (audit_buf_sz) Global Control Block FCM Connection entries (fcm_num_connect) FCM Message Anchors (fcm_num_anchors) FCM Request block (fcm_num_rqb) NOTES: Database Manager Shared Memory -- Allocated when Database Manager is started and remains allocated until the Database Manager is stopped. Fast Communication Manager is used for communications between partitions on DPF and for communications between agents in ESE when intra_parallel is enabled (set to YES(1)) in the DBM CFG. This picture depicts memory for an EE environment. DPF allocation of FCM control blocks and buffers is slightly different. This area can get quite large based on the value of maxagents and the number of FCM entries and amount of parallelism used. Monitor heap (mon_heap_sz) DBM CFG Default of 56, 34, or 12 4k pages depending on the OS. Range of 0 – 60,000 4k pages. A value of 0 disables monitoring. Monitor heap is used by snapshot and event monitors. It determines the amount of memory allocated for use by the Database System Monitor. Allocated at db2start and freed at db2stop. It is used when performing such activities as updating monitor switches, turning on event monitors, and taking a snapshot. If the memory allocated for monitor heap fills up errors will be set to the db2alert and db2diag.log along with error messages to the application. Audit Buffer (audit_buf_sz) Default of 0. Range of 0 – 65,000 4k pages. Used for buffering audit record asynchronously when written by the audit facility when auditing is active. If left at 0 and auditing is used, records are written synchronously. FCM Buffer Pool (one for each host) FCM Buffers (fcm_num_buffers) FCM Control Block Database Global Memory

Database Manager Shared Memory
Database Global Memory Sort Heap – Shared Sort (sortheap) Package Cache (pckcachesz) Catalog Cache (catalogcache_sz) Lock List (locklist) Restore Buffer (restbufsz) Log Buffer (logbufsz) Extended Memory Cache Backup Buffer (backbufsz) Database Heap (dbheap) Buffer Pools (buffpage) Utility Heap (util_heap_sz) NOTES: Database Global Memory is allocated at “activate” or “first connect” time. It is used across all applications that connect to a database and contains many important memory areas. Database Global memory is allocated per database.

Database Global Memory Package Cache (pckcachesz) Catalog Cache (catalogcache_sz) Lock List (locklist) Restore Buffer (restbufsz) Log Buffer (logbufsz) Extended Memory Cache Backup Buffer (backbufsz) Database Heap (dbheap) Buffer Pools (buffpage) Utility Heap (util_heap_sz) 5000 4k pages with range of 16 – 524,288 4k pages Low Allocated when needed by backup, restore, and load utility & freed when no longer needed NOTES: Utility Heap (util_heap_sz) (DB CFG) specifies the maximum amount of memory that can be concurrently used by the backup, restore, load recovery, and load utility. Default is k pages with a range of 16 – 524,288 4k pages. Utility heap is allocated when needed by the utility and freed when no longer needed by the utility. If utility heap is not large enough, error messages will be generated when running utilities and you may not be able to run the utility. Start with the default and increase if errors are encountered.

Database Global Memory Package Cache (pckcachesz) Catalog Cache (catalogcache_sz) Lock List (locklist) Restore Buffer (restbufsz) Log Buffer (logbufsz) Extended Memory Cache Backup Buffer (backbufsz) Database Heap (dbheap) Buffer Pools (buffpage) Utility Heap (util_heap_sz) 1,024 4k pages with range of 8 – 524,288 4k pages Low Allocated when backup utility is called & freed when the utility completes processing. NOTES: Backup buffer (backbufsz) (DBM CFG) specifies the size of the backup buffer when backing up the database if the size is not specified on the backup utility call. Default is k pages with a range of 8 — 524,288 4k pages. It is allocated when the backup utility is called and freed when the utility completes processing. If set too low backups will take longer or not run and if set too high memory is wasted. Adjust it to what works in your environment.

Database Global Memory Package Cache (pckcachesz) Catalog Cache (catalogcache_sz) Lock List (locklist) Restore Buffer (restbufsz) Log Buffer (logbufsz) Extended Memory Cache Backup Buffer (backbufsz) Database Heap (dbheap) Buffer Pools (buffpage) Utility Heap (util_heap_sz) Allocated when the utility is called and freed when utility completes 1,024 4k pages with range of 16 – 524,288 4k pages Med NOTES: Restore Buffer(restbufsz) (DBM CFG) specifies the size of the buffer used when restoring the database if not specified on the restore command. DB2 copies the data to an internal buffer specified by restbufsz and the data is then written to the disk when this buffer is full. Allocated when the restore utility is called and freed when the utility completes. Default of k pages with a range of 16 – 524,288 4k pages. Adjust to your environment. Generally, you want to be liberal with the allocation as you want your restores to complete in the shortest amount of time.

Database Global Memory Package Cache (pckcachesz) Catalog Cache (catalogcache_sz) Lock List (locklist) Restore Buffer (restbufsz) Log Buffer (logbufsz) Extended Memory Cache Backup Buffer (backbufsz) Database Heap (dbheap) Buffer Pools (buffpage) Utility Heap (util_heap_sz) 8x maxappls or 32, whichever is largest with upper limit of 64,000 (32 bit) or 524,288 (64 bit) 4k pages depending on OS Must be large enough to hold all SQL statements that are executing concurrently. Package cache reduces overhead by eliminating the need to access catalog and by eliminating a prepare or the load of a package Allocated when the database is initialized and when the database is shutdown High NOTES: Package Cache (pckcachesz) (DB CFG) specifies the amount of memory to be used for caching dynamic and static SQL. Allocated at database startup and freed at shutdown. Default of 8x maxappls or 32, whichever is largest with an upper limit of 64,000 (32-bit) or 524,288 (64 bit) 4k pages depending on the operating system. In a DPF environment there is one package cache per partition. Package caching reduces overhead by eliminating the need to access the catalog to load a package or eliminating a compile (prepare) for dynamic SQL. Sections are kept in the cache until 1) the database is shutdown 2) The package or dynamic SQL statement is invalidated or 3) The cache runs out of space! Caching of sections is more important in OLTP systems as statements tend to be used over and over. Be aware of those sections that are used initially and then only a few are run repeatedly as the initial ones will remain in the cache and waste memory that may be better spent on bufferpools. Note that Package Cache is required and it must be large enough to hold all SQL statements that are executing concurrently. If there is more memory than needed, then sections are cached. Benchmarking should be used to initially size your package cache.

Database Global Memory Good use of bufferpools can give you the biggest bang for the buck. Can offset bad design to some extent Package Cache (pckcachesz) Catalog Cache (catalogcache_sz) Lock List (locklist) Restore Buffer (restbufsz) Log Buffer (logbufsz) Extended Memory Cache Backup Buffer (backbufsz) Database Heap (dbheap) Bufferpools (buffpage) Utility Heap (util_heap_sz) High 75% of physical memory can be devoted to bufferpools if dedicated database server Use of extended memory cache can be beneficial when the amount of real memory available exceeds the addressability of the OS, workload is mostly read only, and when the workload is IO bound. NOTES: Bufferpools (buffpage) (DB CFG) specifies the number of pages for bufferpool creation if NPAGES –1, otherwise buffpage is ingnored and bufferpool will be created based on the NPAGES setting in the CREATE or ALTER BUFFERPOOL statement. Default is 1,000 pages on UNIX and 250 on NT and OS/2. Range is to 2 – 524,288 on 32 bit UNIX and 2 – 2,147,483,647 on 64 bit UNIX. Allocated at “First Connect” or “Activate Database” issued. ONE OF THE MOST CRITICAL AREAS THAT HAS A MAJOR EFFECT ON PERFORMANCE. Allocate bufferpools as large as possible without causing paging. If data can be accessed in the bufferpool it will be accessed many times faster than if an I/O to disk has to be done. Place large randomly accessed objects into separate bufferpools so that activity against these objects does not force pages for other objects out of the bufferpool. Most large randomly accessed objects cannot be contained in a bufferpool and because of the random access patterns cause regular disk access and will not get good bufferpool hits anyway. Prioritize the objects in your bufferpools based on business priorities, if you don’t have priorities from the business side, prioritize based on your knowledge of access pattern and frequency. Some memory is used in DBHEAP for each bufferpool created so DBHEAP might have to be increased. Extended Memory Cache can be used when the amount of real storage available exceeds the usable addressable virtual memory (which is 2GB or 4GB) on 32-bit OS’s. Extended storage memory segments are associated with a bufferpool and are used as a second level of caching. The Database Manager moves data from extended storage as pages, but it is still much faster than accessing disk. Num_estore_segs and estore_seg_sz define the amount of extended memory available along with the appropriate bufferpool option. Recent presentation indicated one of the top reasons for poor performance is using default buffpage

Database Global Memory Package Cache (pckcachesz) Catalog Cache (catalogcache_sz) Lock List (locklist) Restore Buffer (restbufsz) Log Buffer (logbufsz) Extended Memory Cache Backup Buffer (backbufsz) Database Heap (dbheap) Buffer Pools (buffpage) Utility Heap (util_heap_sz) Allocated at first connect and freed when last application disconnects from database Default depends on OS with range of 4 – 60,000 4k pages High NOTES: Lock List (locklist) (DB CFG) specifies how much memory is allocated for storing locks. The default is 100, 50, or 25 4k pages depending on the OS and/or local and remote clients. The range is 4 – 60,000 4k pages. It is allocated at “First Connect” and freed when last application disconnects from the database. There is one lock list per database and it is related to the maxlocks DB CFG parameter. It should be set as a result of stress testing or benchmarking. It must be tailored to your environment. When the amount of lock list used by one application reaches maxlocks, lock escalation will occur for the locks being held, from row to table which results in decreased concurrency as a result of table locks and could affect many more applications trying to access the same object. Note, the use of Repeatable Read (RR) isolation may result in an immediate table lock. In most situations, it is best to avoid lock escalation. Escalation also takes place if locklist runs out of space. Used in conjunction with maxlocks. Maxlocks specifies percent of locklist any one application can use before escalation takes place

Database Global Memory Package Cache (pckcachesz) Catalog Cache (catalogcache_sz) Lock List (locklist) Restore Buffer (restbufsz) Log Buffer (logbufsz) Extended Memory Cache Backup Buffer (backbufsz) Database Heap (dbheap) Buffer Pools (buffpage) Utility Heap (util_heap_sz) High Allocated at first connect and freed when last application disconnects from the database. Default depends on the OS with a range of 32 – 524,288 4k pages NOTES: Database Heap (dbheap) (DB CFG) specifies the amount of memory that is allocated and contains control blocks for tablespaces, tables, and indexes. Bufferpool control blocks moved out in V8.1 The log buffer and catalog cache are also contained in dbheap. The default is 1,200, 600, and 300 4k pages depending on the OS and if local and remote clients are used. The range is 32 – 524,288 4k pages. The amount of dbheap that you need should be established as a result of stress testing or benchmarking and should be monitored continuously. Dbheap is allocated at “First Connect” and freed when the last application has disconnected from the database. Monitor via snapshot monitoring. Take into consideration the amount of log buffer and catalog cache that is required when specifying dbheap. Log Buffer and Catalog Cache are allocated from dbheap

Database Global Memory Allocated as part of dbheap Package Cache (pckcachesz) Catalog Cache (catalogcache_sz) Lock List (locklist) Restore Buffer (restbufsz) Log Buffer (logbufsz) Extended Memory Cache Backup Buffer (backbufsz) Database Heap (dbheap) Buffer Pools (buffpage) Utility Heap (util_heap_sz) 8 4k pages with range 4 – 4,096 4k pages (32 bit) & 4 – 65,535 4k pages (64 bit) High NOTES: The Log Buffer (logbufsz) (DB CFG) specifies the amount of dbheap for use for writing log records before they are written to disk. Allocated as part of dbheap. The default is 8 4k pages with a range of 4 – 4,096 4k pages for 32 bit systems and 4 – 65,535 4k pages for 64 bit UNIX systems. Log records are written to disk when 1) A transaction commits or a group of transactions commit as defined by the mincommit DB CFG parameter 2) The log buffer is full 3) As a result of some other internal database manager event. It is generally efficient to specify a mincommit value of 1 (the default). For very high transactional systems, monitor and adjust as necessary.

Database Global Memory Monitor using elements: cat_cache_lookups, cat_cache_inserts, cat_cache_overflows, cat_cache_heap_full Package Cache (pckcachesz) Catalog Cache (catalogcache_sz) Lock List (locklist) Restore Buffer (restbufsz) Log Buffer (logbufsz) Extended Memory Cache Backup Buffer (backbufsz) Database Heap (dbheap) Buffer Pools (buffpage) Utility Heap (util_heap_sz) Default depends on OS with range of 16 – 60,000 4k pages If not large enough increase a few pages at a time Med Stores table descriptor info used when table, view, or alias referenced during compilation of an SQL statement NOTES: Catalog Cache (catalogcache) (DB CFG) specifies the maximum amount of memory that can be used for caching table descriptor information in dbheap. The default is 64, 32, or 16 4k pages depending on the OS and whether or not remote and local clients are used. It is allocated when dbheap is allocated. Catalog Cache stores table descriptor information that is used when a table, view, or alias is referenced during compilation of an SQL statement. Use of cache helps improve performance of SQL statements when the same tables, views, or aliases have been referenced in previous statements. Descriptor information for declared temporary tables is not stored in catalog cache, it is stored in application control heap. Running DDL against a table will purge that table’s entry from the cache. Otherwise, it is kept in the cache until room is needed for another table, but not until all units of work have been completed. Start with the deault and monitor with snapshot monitoring.

Database Global Memory Sort Heap – Shared Sort (sortheap) Package Cache (pckcachesz) Catalog Cache (catalogcache_sz) Lock List (locklist) Restore Buffer (restbufsz) Log Buffer (logbufsz) Extended Memory Cache Backup Buffer (backbufsz) Database Heap (dbheap) Buffer Pools (buffpage) Utility Heap (util_heap_sz) SHEAPTHRES is an instance wide soft limit for private sorts SHEAPTHRES for shared sorts is an instance wide hard limit on the on total amount of memory used by shared sorts at any given time High NOTES: Shared sort memory size is statistically determined by DB2 based on the size of SHEAPTHRES and it is not allocated until needed. Shared sort is used when intra-partition parallelism is used, as sort operations can be processed in parallel. SHEAPTHRES is a DBM CFG parameter that controls the total amount of memory that can be allocated for sorting. It should be set to the maximum amount of memory for all sortheaps that should be allocated at any given time. SHEAPTHRES_SHR, new in V8.1 controls shared sorts at the database level if intra-partition parallelism enabled.

Application Global Memory (app_ctl_heap_sz)
Med Application Global Memory (app_ctl_heap_sz) Only allocated if if you are using DPF or ESE with intra_parallel enabled Used to store Declared Temporary Tables in DPF Agent Private Memory Agent Stack (agent_stack_sz) Statistics Heap (stat_heap_sz) Sort Heap (sortheap) Application Heap (applheapsz) DRDA Heap (obsolee in V8) UDF Memory (obsolete in V8) Statement Heap (stmtheap) Query Heap (query_heap_sz) Client I/O Block (rqrioblk) NOTES: Application Global Memory (app_ctl_heap_sz) (DB CFG) is only allocated if using DPF or ESE with intra_parallel enabled. Default is 128, 64, 256, and 128 4k pages depending on the OS and if local and/or remote clients are used. Application Global Memory specifies how much application control heap is allocated for each application at the database where the applications is active (in the case of DPF, heap is allocated at each database partition where the application is active). The heap is allocated during connect processing by the first agent to receive a request for the application at the database (or database partition). The heap is used by agents working for the application to share information (in DPF sharing occurs at the database partition level). The heap is also used to store descriptor information for declared temporary tables, which is kept until the declared temporary table is explicitly dropped. In a DPF environment, app_ctl_heap_sz is used to store executing copies of SQL statements for agents and subagents. Note that SMP subagents use applheapsz instead. Allocation only occurs if intra_parallel is set on and the CURRENT DEGREE special register is set greater than 1. Start with the default value and increase it if you use lots of declared temporary tables, run complex SQL or have many database partitions.

Agent Private Memory Allocated when agent initialized and freed when agent completes work for an application. Stores copies of executing SQL statements Agent Stack (agent_stack_sz) Statistics Heap (stat_heap_sz) Sort Heap (sortheap) Med Application Heap (applheapsz) DRDA Heap (drda_heap_sz) UDF Memory (udf_mem_sz) Statement Heap (stmtheap) Default of 128 or 64 4k pages depending on EEE or not with a range of 16 – 60,000 4k pages Query Heap (query_heap_sz) Client I/O Block (rqrioblk) NOTES: Application Heap (applheapsz) (DB CFG) specifies the amount of private memory to be used by the Database Manager on behalf of a specific agent or subagent. The default is 128 or 64 4k pages depending on whether DPF is used or not. The range is 16 – 60,000 4k pages. It is allocated at agent initialization and freed when the agent completes the work to be done for an application. The initial amount allocated will be the minimum required to do the work. As more heap space is required it will be allocated by the Database Manager as needed, up to the maximum specified. Start with the default and monitor. Increase if errors are received indicating a shortage in application heap.

Agent Private Memory Low Low unless over-allocated then OS paging may occur Agent Stack (agent_stack_sz) Statistics Heap (stat_heap_sz) Sort Heap (sortheap) Application Heap (applheapsz) DRDA Heap (drda_heap_sz) UDF Memory (udf_mem_sz) Statement Heap (stmtheap) Query Heap (query_heap_sz) Client I/O Block (rqrioblk) NOTES: Agent Stack Size (agent_stack_sz) (DB CFG) specifies the amount of virtual memory that is allocated for each agent. This does not apply to UNIX platforms. The default is 64 and 16 4k pages (Windows NT) with a range of 8 – 1,000 4k pages. The memory is used when needed to process an SQL statement. More complex queries require more stack space, and simple queries require less. You should start with the default and only increase it if you are using very complex queries. If there is not enough agent stack available, an error message will be returned and an error recorded in the db2diag.log.

Agent Private Memory Low Agent Stack (agent_stack_sz) Statistics Heap (stat_heap_sz) Sort Heap (sortheap) Application Heap (applheapsz) DRDA Heap (drda_heap_sz) UDF Memory (udf_mem_sz) Statement Heap (stmtheap) Query Heap (query_heap_sz) Client I/O Block (rqrioblk) NOTES: Statistics Heap (stat_heap_sz) (DB CFG) specifies the maximum size of heap to be obtained while running runstats. The default is k pages with a range of 1096 – 524,288 4k pages. The default value is only good for small narrow tables. If you are collecting distribution statistics you will need to increase statistics heap, especially with wide tables.

Agent Private Memory High Allocated when needed and freed when sorting is completed High Agent Stack (agent_stack_sz) Statistics Heap (stat_heap_sz) Sort Heap (sortheap) Application Heap (applheapsz) Piped sorts can return results without requiring a temp table. Non-piped sorts require temp tables via buffer pool to return the result. DRDA Heap (drda_heap_sz) Monitor using the following elements: Sort_heap_allocated Post_threshold_sorts Piped_sorts_requested Piped_sorts_accepted Total_sorts Total_sort_time Sort_overflows Active_sorts UDF Memory (udf_mem_sz) Statement Heap (stmtheap) Non-overflowed sorts can complete within the sortheap.Overflowed sorts cannot complete in sortheap and require temp tables. Query Heap (query_heap_sz) Client I/O Block (rqrioblk) NOTES: Sort Heap (sortheap) (DB CFG) specifies the maximum number of private memory pages to be used for each private sort or the maximum number of shared memory pages to be used for shared sorts. The default is 256 4k pages with a range of 16 – 524,288 4k pages. It is allocated as needed to perform sorts and freed when sorting is complete. DB2 uses two types of memory for sorts. Private memory is used for non-parallel sort techniques (using ESE with intra_parallel set to no). Private sorts are allocated out of agent private memory as needed by the Database Manager. Data is sorted in the sort heap. A smaller sort heap can be used if determined by the optimizer. Sort heap is used by the optimizer in determining access paths and packages should be rebound if the value of sort heap is changed. If the sort heap is not big enough, sorting can overflow to temporary tables in Tempspace1 and this should be avoided in OLTP systems if possible. In large database environments or OLAP/BI, sort overflows can probably not be avoided and then effort should be placed on tuning the I/O paths (disks, containers, bufferpools). Of course, the best sort is NO SORT but we humans like things in order so it is difficult to avoid. The most effective method for eliminating sorts is proper indexing on the sorted columns. Sorting is very costly and probably makes up more than 70% of most queries elapsed times, so there can be big payoffs by tuning sorts and sort memory areas. Be sensitive to OS paging when sizing sortheap and SHEAPTHRES and do not size them so big as to cause paging. Piped sort not freed until cursor closed.

Agent Private Memory Workspace for dynamic SQL (when executing) and static SQL (during bind) Agent Stack (agent_stack_sz) Statistics Heap (stat_heap_sz) Sort Heap (sortheap) Application Heap (applheapsz) Med DRDA Heap (drda_heap_sz) UDF Memory (udf_mem_sz) Statement Heap (stmtheap) Default of 2,048 4k pages with range of 128 – 60,000 4k pages Query Heap (query_heap_sz) Client I/O Block (rqrioblk) NOTES: Statement Heap (stmtheap) (DB CFG) specifies the amount of memory to be used as a workspace to be used by the SQL compiler when compiling an SQL statement. For dynamic SQL this area will be used during execution of the dynamic SQL statement, whereas for static SQL it is only used during the bind process. The default is k pages with a range of 128 – 60,000 4k pages. It is allocated for each statement during precompiling or binding. It is freed when precompiling or binding of each statement is complete. The default should be adequate. If it is not, an error indicating that the statement is “too complex” will be returned. Increase the size in regular increments until it is resolved. In DPF, the app_ctl_heap_sz is used instead.

Use to store each query in the agents private memory. SQLDA, statement, SQLCA, package name, creator, section number and consistency token Memory for blocking cursors, cursor control blocks, and fully resolved SQLDA Agent Private Memory Agent Stack (agent_stack_sz) Statistics Heap (stat_heap_sz) Sort Heap (sortheap) Application Heap (applheapsz) Default of 1,000 4k pages with range of 2 – 524,288 4k pages DRDA Heap (drda_heap_sz) UDF Memory (udf_mem_sz) Statement Heap (stmtheap) Med Query Heap (query_heap_sz) Client I/O Block (rqrioblk) NOTES: Query Heap Size (query_heap_sz) (DBM CFG) Specifies the maximum amount of memory that can be allocated for the query heap. A query heap is used to store each query in the agent’s private memory. Information stored for each query consists of the SQLDA, statement, SQLCA, package name, creator, section number, and consistency token. Memory for blocking cursors is also allocated from query heap along with cursor control blocks and fully resolved output SQLDA. Query Heap Size defaults to the size of Application Support Layer Heap (aslheapsz). Interestingly, the query heap size must be greater than or equal to two (2), and must be greater than or equal to the aslheapsz parameter. If it is not large enough to handle a given request, it will be reallocated to the size required by the request (not exceeding query heap size). If this new query heap size is more than 1.5 times larger than aslheapsz, the query heap will be reallocated to the size of aslheapsz when the query ends. The default should be sufficient. As a minimum it should be 5x the aslheapsz. This will provide enough memory for about 3 or 4 blocking cursors to be open at a given time. If using LOBs, you may need to increase this. ASLHEAPSZ is not depicted here. It is used for communication between agents and local applications. Tuning priority is high for local clients or batch applications is similar to recommendations for applheapsz. Default is 15 4k pages.

Specifies size of the communications buffer between agents and remote applications in bytes Agent Private Memory Agent Stack (agent_stack_sz) Statistics Heap (stat_heap_sz) Sort Heap (sortheap) Determines size of IO block at database client when blocking cursor is opened, Default of 32,767 bytes should be ok Application Heap (applheapsz) DRDA Heap (drda_heap_sz) UDF Memory (udf_mem_sz) Statement Heap (stmtheap) High Optimize for N rows can be used to control the number of rows returned per block Query Heap (query_heap_sz) Client I/O Block (rqrioblk) NOTES: Client I/O Block (rqrioblk) ( DBM CFG) specifies the size of the communication buffer between remote applications and their database agents on the database server. The default is 32,737 bytes with a range of 4,096 – 65,535 bytes. It is allocated when a remote client application issues a connection request for a server database, and when a blocking cursor is opened, additional blocks are opened at the client. It is freed when the remote application disconnects from the server database and when the blocking cursor is closed. When a remote client requests a connection to remote database, this communication buffer is allocated on the client. On the database server, a communication buffer of 32,767 is initially allocated, until a connection is established and the database server can determine the value of rqrioblk at the client. Once the server knows this value it will reallocate its buffer if the clients is not 32,767. Rqrioblk is also used to determine the size of the I/O block at the database client when a blocking cursor is opened. The default is generally sufficient. If more fetch requests are generated than required by the application, OPTIMIZE FOR N ROWS can be used to control the number of fetch requests. OPTIMIZE FOR N ROWS also limits the number of rows returned per I/O Block.

Monitoring IBM Self – Managing and Resource Tuning (SMART) Project
Will make DBAs more productive and job easier DBA will always be in-the-loop Compete better with SQL Server and to a lesser extent Oracle IBM Announced the SMART program at IDUG 2001 during Janet Perna’s keynote speech. The Configure Performance Wizard and Index Advisor are steps in this direction. Things to come include automatic detection of parameter threshold breeches and automatic action. Additional wizards will include additional health – checking, detection and action. The goal is to make our jobs easier and compete with SQL Server which is well known for it’s ease of administration and to a lesser extent Oracle. Many DBAs will fear this SMART initiative as they will look upon it as threatening job security but I believe we will always need the DBA to make those decisions and take actions that just can’t be automated to work in every environment. Example, a tool can recommend an index but tools don’t know the workload mix, like how will this effect other programs accessing this object doing inserts or updates?

Automatic Configuration
Autoconfigure command can be used to set DBM and DB CFG parameters initially until the workload can be better defined Syntax -- AUTOCONFIGURE [USING config-keyword value [{,config-keyword value}...]] [APPLY {DB ONLY | DB AND DBM | NONE}] config-keyword: MEM_PERCENT, WORKLOAD_TYPE, NUM_STMTS, TPM, ADMIN_PRIORITY, IS_POPULATED NUM_LOCAL_APPS, NUM_REMOTE_APPS, ISOLATION, BP_RESIZEABLE. NOTES: Refer to Example 12, autoconfig example for additional details.

Automatic Configuration
“db2 autoconfigure apply none” command can be used to see DB2 recommendations but not apply them. Refer to autoconfigure example output. NOTES:

Monitoring In order to make sure that resources are being used efficiently and to ensure that business requirements are met, continuous monitoring must be practiced Use Snapshot Monitoring and Event Monitoring Third Party Vendor Tools DBM and DB CFG parameters must be monitored on a regular basis. As workloads and environments change, these parameters will need to tuned accordingly. DB2 provides snapshot and event monitoring capabilities with the product. Third party vendor tools are available to assist in this area.

NUM_IOCLEANERS DB CFG Default 1 Range(0 –255)
Specifies number of asynchronous page cleaners for a database. Write changed pages from Bufferpool to disk Triggered by CHNGPGS_THRESH which specifies a percentage of used pages at which asynchronous page cleaners will start writing out pages Set to number of CPUs

Page Cleaning Enhancements
New registry variable in DB2 V8.1.4 changes page cleaning algorithm DB2_USE_ALTERNATE_PAGE_CLEANING registry variable Page cleaners write pages proactively instead of waiting to be triggered

NUM_IOSERVERS DB CFG Default 3 Range(1 –255)
Used to specify the number of prefetchers that work on behalf of database agents to perform prefetch IO and asynchronous IO for utilities such as backup and restore. Set to at least the number of physical disks available to increase opportunity for parallel IO Increase if errors are received indicating that “there are no prefetchers available.”

Agent Parameters Coordinator Agent – Each application has one which does work on its behalf and in a parallel environment distributes work to subagents Upon disconnect or detach from an instance the coordinating agent is freed and marked as idle if max number of pool agents not reached else it is terminated and storage freed if max number of pool agents reached DBM CFG parameter max_coordagents Max_coordagents parameter specifies the number of coordinating agents that can exist at one time. One coordinating agent is required for each local or remote application. The default is the value of maxagents. If intra_parallel is set to yes or running DPF, the default is maxagents – num_initagents.

Agent Parameters Maximum Number of Agents (maxagents) — specifies the maximum number of database manager agents, whether coordinating agents or subagents, available at any given time to accept application requests Can be used in resource constrained systems to limit memory usage Maxagents –(DBM CFG) Default is 200 or 400 (DPF) or 10 (Satellite) with a range of 1 – 64,000. Recommended setting is the sum of the value for maxappls for each database allowed to be accessed concurrently.

Agent Parameters Maximum Number of Active Applications (maxappls)
Specifies the maximum number of concurrent applications that can be connected to a database When reached, an error is returned to the application and connection is not allowed Can be used to throttle applications in a resource constrained system Maxappls – (DB CFG) Default is 40, 30 or 10 depending on the OS. Range is ,000. In a DPF environment, maxappls controls the maximum number of applications that can be concurrently active against a database partition. To a certain extent, the maximum number of applications is also governed by maxagents. An application can only connect to the database, if there is an available connection (maxappls) as well as an available agent (maxagents). In addition, the maximum number of applications is also controlled by the max_coordagents configuration parameter, because no new applications (that is, coordinator agents) can be started if max_coordagents has been reached.

Agent Parameters Maximum Number of Concurrent Agents (maxcagents)
Specifies the max number of database manager coordinating agents that can be concurrently executing a database manager transaction Does not limit the number of applications connected but limits the number of database manager agents that can be processed concurrently Can be used to throttle applications if resource constrained Maxcagents (DBM CFG) default of –1, max_coordagents. If the default is not working in your environment, use benchmarking to determine the appropriate setting. Controls the number of processing tokens available from the database manager. All coordinating agents must get a processing token from the database manager in order to process the transaction. If not available, the agent will wait until one is available.

Agent Parameters Initial Number of Agents in Pool (Num_initagents)
Specifies the initial number of idle agents that are created in the agent pool at DB2START By specifying a value, agents are available in the pool for initial requests and the overhead of repeated agent creation is avoided Num_initagents (DBM CFG) Default is 0 with a range of 0 to num_poolagents.

Agent Parameters Agent Pool Size (num_poolagents)
Specifies how large the agent pool can get Contains subagents and idle agents Idle agents can be used as coordinating agents or subagents If more agents created than this parameter they will be terminated when the current request is completed rather than returned to the pool Num_poolagents (DBM CFG) specifies the maximum numbe of agents that can be created in the agent pool. Decision-support environments should set num_poolagents to a small value to avoid having an agent pool that is full of idle agents. In an OLTP or Web environment in which many applications are concurrently connected, increase the value of num_poolagents to avoid the costs associated with the frequent creation and termination of agents.

Conclusion Successful system tuning requires knowledge of:
DB2 processing Available monitoring facilities Instance Configuration Parameters Database Configuration Parameters Cause and Effect of parameters to processing *Available References *Successful tuning and understanding of DB2 UDB requires that DBAs have access to these resources.

References Database Performance on AIX in DB2 UDB and Oracle Environments, SG URL:

References SC09-4821, Administration Guide: Performance
SC , Administration Guide: Planning SC , Administration Guide: Implementation SC , What’s New, DB2 UDB V8

REFERENCES DB2 UDB v8 Announcement, , DB2 Connect v8 Announcement, http:// SC , Command Reference

DB2 Architecture Workshop
Refer to your student workbook and examples for the DB2 Architecture Workshop. You have 1 hr to complete the workshop. NOTES:

Module 7 DB2 Optimizer Explain & Design Advisor

OBJECTIVES Upon completion of this module you will understand:
Optimization Classes and selection of classes based on type of environment (OLTP,WEB, Mixed, BI/DW) How to use DB2 Explain to tune SQL What the optimizer uses to determine access paths How to use Design Advisor Types of predicates NOTES:

Basic SQL Coding Rules Limit the amount of data returned
Avoid “SELECT *” Use predicates with good selectivity Use Range delimiting and Index Sargable predicates wherever possible When joining tables, specify predicates that are indexed Analyze local, order by, and join predicates for index access (index only access, elimination of sorts, cartesian products. NOTES:

TYPE-2 INDEXES Type-2 indexes improve performance by eliminating most next-key-share locks, as entries are marked deleted instead of physically deleted from the page. Type-2 indexes are required for online load, online reorganization, and MDC. A table cannot have a mix of type-1 and type-2 indexes. Tables can be migrated to type-2 indexes via index reorganization. Type-2 indexes let you create an index on a column that is longer than 255 bytes. NOTES:

DB2 Optimizer NOTES:

DB2 Optimizer - Predicates
Range delimiting (Index Manager) Used to bracket an index scan Provide start and stop keys for the index search NOTES:

DB2 Optimizer Index Sargable Not used to bracket a search
But are evaluated from the index Columns involved in the predicates are part of the index Evaluate by Index Manager NOTES:

DB2 Optimizer -Predicates
Data Sargable Cannot be evaluated by index manager Evaluated by Data Management Services Typically require data access DMS will retrieve the columns to evaluate the predicate as well as any other to satisfy columns in the select list that could not be obtained via an index NOTES:

DB2 Optimizer - Predicates
Residual predicates require I/O beyond just accessing the base table Example: Quantified subqueries ANY, ALL, IN, SOME, Long VARCHAR, LOBS Are evaluated by RDS and are the most costly of the 4 categories of predicates NOTES:

DB2 Optimizer NOTES:

DB2 Optimizer - Dynamic SQL
Flexibility Most packaged applications and web based applications use dynamic SQL Provides flexibility and uses the most current statistics NOTES:

DB2 Optimizer - Dynamic SQL
Caution! Growth in data Number of indexes available Can result in changing access paths NOTES: One customer experienced changing access paths from day to day or even during the same day. All SQL was dynamic SQL. Resolved by eliminating redundant indexes and running RUNSTATS on a regular basis. Refer to unit on Predicates for more information.

DB2 Optimizer - Inputs NOTES:

DB2 Optimizer – Distribution Statistics
NOTES: DETAILED INDEXES STATISTICS -- Calculates extended index statistics. These are the PAGE_FETCH_PAIRS and CLUSTERFACTOR statistics. This information enables the optimizer to better estimate the cost of accessing a table through an index. The optimizer considers both buffer pool size and degree of clustering when it estimates the cost of accessing a table through and index. WITH DISTRIBUTION -- Specifies that both basic statistics and distribution statistics are collected on columns. Distribution statistics are collected on all columns of the table. Frequency statistics provide information about the column and the data value with the highest number of duplicates, the next highest number of duplicate values up to the number specified by the num_freqvalues DB CFG parameter. Quantile statistics may also be collected. Distribution statistics should be collected for a table when dynamic SQL is being used and when static SQL is being used without host variables. Distribution statistics should be collected when at least one column in a table has a highly “non-uniform” distribution of data and the column occurs frequently in equality or range predicates, and when duplicate data values occur often.

RUNSTATS See Notes NOTES:
When the RUNSTATS utility is run for a table or indexes the following statistical information is stored in the DB2 Catalog tables: For a table and index: The number of pages in use The number of pages that contain rows The number of rows that overflow The number of rows in the table (cardinality) For each column in the table and the first column in the index key: The cardinality of the column The average length of the column The second highest and second lowest values in the columns The number of NULLS in the column For Indexes only: The number of leaf pages The number of index levels The degree of clustering of the table data to the index The ratio of leaf pages on disk in index key order to the number of pages in the range of pages occupied by the index The number of distinct values in the first column of the index The number of distinct values in the first two, three, and four columns of the index The number of distinct values in all columns of the index The number of leaf pages located on disk in index key order, with few or no large gaps between them The number of pages on which all RIDS are marked deleted (V8 only) The number of RIDS marked deleted on pages on which not all RIDS are marked deleted (V8 only)

Optimization Class There are 7 optimization classes available for use
The optimization class (dft_queryopt) Database Configuration (DB CFG) parameter specifies how much optimization the query optimizer should use for queries by default it is set to 5 There are 7 optimization classes available for use On the next slide we’ll discuss them in detail NOTES:

Optimization Class 0 -- provides for minimal query optimization
1 -- small amount of query optimization 2 -- slight amount of query optimization 3 -- moderate amount of query optimization 4 -- reserved for future use 5 -- significant query optimization with heuristics to limit the effort expended on selecting an access plan 6 -- reserved for future use 7 -- significant query optimization 9 -- maximal query optimization NOTES: 0 - This class directs the optimizer to use minimal optimization. Non-uniform distribution statistics are not considered. Only basic query rewrite rules are applied. Greedy join enumeration occurs. Only nested loop joins and index scan access methods are enabled. List prefetch and index ANDing are not considered. Star join strategy is not considered.This class should only be used when very simple SQL is being used with well indexed tables. 1 - Non-uniformdistribution statistics are not considered. Only a subset of query rewrite rules are used. Greedy join enumeration occurs. List prefetch and index ANDing are not used. Index ANDing may be used for semijoins in star joins. Merge scan joins and table scans may be used. 2 - Uses a higher degree of optimization but not as much as Class 3. All available statistics, including both frequency and quantile non-uniform and distribution statistics are used. All query rewrite rules are applied, except computational intensive rules (which are very infrequent). Including routing to ASTs/MQTs. Greedy join enumeration. Wide range of access methods are used as well as routing to MQTs and use of Star joins. Class 2 is recommended for complex queries in DSS or OLAP environment where such queries are rarely repeated exactly.

Optimization Class This page intentionally left blank NOTES:
3 - This class requests a moderate amount of optimization. Closest class to that used on DB2 for z/OS. Non-uniform distribution statistics, which track frequently occurring values are used. Most query rewrite rules are applied, including sub-query-to-join transformations. Dynamic programming join enumeration. Limited use of composite inner tables, and limited use of cartesian products for star-schema using lookup tables. Wide range of access methods, including list prefetch, index ANDing and Star Joins. This class improves access paths for queries with more than 4 joins. 5- This class directs the optimizer to use a significant amount of optimization to generate an access plan. All available statistics are used. All query rewrite rules are applied. Dynamic programming join enumeration. Wide range of access methods. When the optimizer detects that additional resources and processing time are not warranted for complex dynamic SQL queries, optimization is reduced. The extent or size of the reduction depends on the size of the machine and number of predicates. When the query optimizer reduces the amount of optimization, it continues to apply all the query rewrite rules that would normally be applied. However it uses the Greedy Join enumeration method and reduces the number of access plan combinations considered. Class 5 is ideal for a hybrid environment, where OLTP and DW queries are used.

Optimization Class Use the SET CURRENT QUERY OPTIMIZATION command in dynamic SQL to set the special register to the desired optimization class Use explain to verify if an improved access plan is generated Test and verify new plan NOTES: 7 - This class is the same as 5 except it does not reduce the amount of optimization done for complex dynamic SQL queries. 9 - This class directs the optimizer to use all available optimization techniques. These include: -- all available statistics -- all query rewrite rules -- all possibilities for join enumeration, including Cartesian products and unlimited composite inners. -- all access methods This class can greatly expand the number of access plans considered by the optimizer. You might use this class to find out whether more comprehensive optimization would generate a better access plan for very complex and very long running queries that use large tables. Consider using this class for DW and BI queries that meet the above criteria. Use explain and verify that a better plan has been developed. Use the SET CURRENT QUERY OPTIMIZATION = 1 (or class desired) command to set the special register for dynamic SQL statements that you want to use a different value than the dft_queryopt parameter setting.

Optimization Class Set dft_queryopt through trial and error process with your applications during application development process For OLTP and Web-based applications, set dft_queryopt to 1 or 3 (keep restrictions in mind) OLTP/Web applications should be only returning a row or few rows with index only or indexed access Few reads involved Not very complex SQL For Mixed environments set dft_queryopt to 5 Usually a mixture of simple and complex SQL NOTES:

Optimization Class For BI/DW environments set dft_queryopt to 7 or 9
Evaluate differences between optimization class 7 and 9 in your environment, with your applications Can be evaluated easily using Visual Explain NOTES: Since each environment and application is different, blanket recommendations for optimization class setting is difficult to give. What we have found is that many environments to not fit nicely into standard OLTP and BI/DW categories, i.e.; OLTP only returning a row or two and BI/DW using complex SQL and large table scans. ERP/CRM/SCM are somewhere in between these categories. Many environments are a mixture of the two.

SQL Compiler NOTES:

Explain Tables NOTES: Refer to Example 16 for a listing of DB2 Explain Tables. Before an explain can be done, explain tables must be created. They are also used for storing Design Advisor information and are required for Design Advisor to work properly.

DB2 Explain DB2 offers three types of Explains
dynexpln which is used for explaining dynamic SQL is still available but deprecated db2exfmt is a “complete explain” and produces explain information in text format DB2 Visual Explain DB2 Visual Explain is a powerful tool which visually displays access paths and statistics associated with the SQL statement being analyzed. It can be launched from the Control Center DB2 Visual Explain has a Tutorial available NOTES: Explain is the primary tool for evaluating access plans developed by the DB2 Optimizer. Explain should be used by application developers when developing SQL and periodically when evaluating the performance of SQL once it is in production. It is used by DBAs in support of application development efforts, such as walkthroughs, acceptance or regression testing, and to troubleshoot performance problems in the production environment. Explain information is covered in detail in the Administration Guide: Performance.

Optimization Parameters
Finally, Visual Explain has a window that shows the system parameters that affect access path selection. Any changes could affect the access path chosen.

Visual Explain NOTES:

Visual Explain NOTES:Visual Explain starts as a graph of the access path. The SQL text is shown on the next slide. Visual Explain also shows selectivity of predicates. Developers should use predicates with selectivity of .10 or less if possible. This means that for a table of 50,000 rows, 5,000 rows would be returned by using a predicate with selectivity of Developers should take selectivity into consideration when writing SQL queries. The golden rule of SQL is only retrieve what you need!

Optimized SQL Text In the above 2 windows, it is possible to compare the original and optimized SQL text. In some cases, these could be quite different if the query was rewritten by the optimizer. An example using a subselect could have shown the query rewritten to be a join.

Visual Explain NOTES:

db2exfmt Formats rows in explain tables
db2exfmt –d dbname -e schema -f O –n name –s schema –o outfile -# sectnbr -h -d name of database -e schema of explain table -s schema of table being explained -n source name -o outfile: name of output file Wild cards can be used: % and _ The db2exfmt command is entered at the command prompt of the DB2 Command Line Processor. The five parameters can follow on the same line, or if they are not specified, the command will prompt for them. The OUTFILE can be preceded by a drive and directory specification. Use db2exfmt -h to get help information for the input parameters. db2exfmt requires rows in the explain tables. The db2exfmt tool works from the existing output of the bind process. There is no plan table or bind parameters to specify. The access path is interpreted from SYSSECTION, along with English explanations from related Catalog tables.

db2exfmt See below example Refer to example 41
DB2 Universal Database Version 8.1, (c) Copyright IBM Corp. 1991, 2002 Licensed Material - Program Property of IBM IBM DATABASE 2 Explain Table Format Tool ******************** EXPLAIN INSTANCE ******************** DB2_VERSION: SOURCE_NAME: SYSSH100 SOURCE_SCHEMA: NULLID SOURCE_VERSION: EXPLAIN_TIME: EXPLAIN_REQUESTER: DB2INST1 Database Context: Parallelism: None CPU Speed: e-06 Comm Speed: 2 Buffer Pool size: 1000 Sort Heap size: 256 Database Heap size: 1200 Lock List size: 100 Maximum Lock List: 10 Average Applications: 1 Locks Available: 1130 Package Context: SQL Type: Dynamic Optimization Level: 5 Blocking: Block All Cursors Isolation Level: Uncommitted Read

db2exfmt See below example NOTES: Continued.
STATEMENT 1 SECTION QUERYNO: 1 QUERYTAG: Statement Type: Select Updatable: No Deletable: No Query Degree: 1 Original Statement: WITH DEPT_MGR AS ( SELECT DEPTNO, DEPTNAME, EMPNO, LASTNAME, FIRSTNME, PHONENO FROM DEPARTMENT D, EMPLOYEE E WHERE D.MGRNO=E.EMPNO AND E.JOB='MANAGER' ), DEPT_NO_MGR AS ( SELECT DEPTNO, DEPTNAME, MGRNO AS EMPNO FROM DEPARTMENT EXCEPT ALL SELECT DEPTNO, DEPTNAME, EMPNO FROM DEPT_MGR ), MGR_NO_DEPT (DEPTNO, EMPNO, LASTNAME, FIRSTNME, PHONENO) AS (

Line Command Explains db2expln Dynexpln (deprecated in V8) db2exfmt
Use for static when not explained Builds output from syssection Dynexpln (deprecated in V8) Use for a file of SQL or interactive “quickie” db2exfmt To print contents of explain tables The explain commands entered through the command line are most useful when the output is put in a file for review. These commands provide options for whether the explain option was used at bind time, for a file of dynamic SQL, or for formatting the explain tables. Of course, it may be desirable to develop a set of your own queries to find exceptions within the explain tables.

db2expln Command DB2 describes the access plan selection for *dynamic and static SQL statements stored in the DB2 catalog See below NOTES: The syntax is: V | >>--db2expln----<option>--+-->< where <option> and <parameter> are taken from the list below. Each <option> may appear only once, and they may be specified in any order. Connection Options: -database <db> = Connect to the database named <db>. -d <db> -user <name> <pw> = Connect as user <name> with password <pw>. -u <name> <pw> A database name must be specified. Output Options: -terminal = Send output to the terminal. -t -output <file> = Write output to the file named <file>. -o <file> Either terminal or file output must be specified. Help Options: -help = Display this help text. -h -? Package Options: -schema <pattern> = The package creator must match <pattern>. -c <pattern> -package <pattern> = The package name must match <pattern>. -p <pattern> -version <pattern> = The package version must match <pattern>. If not specified, then the package with the version '' (the empty string) will be explained. -section <number> = The section number is <number>. Use 0 (zero) for -s <number> all sections in the package. -escape <charater> = Use <character> as the escape character when -e <character> matching patterns. -noupper = Do not upper case creator, package and version -l before matching. The creator and package information must be specified unless dynamic SQL is being explained. If the section information is not specified, then all sections will be displayed. The <pattern> for creator, package, and version is in LIKE predicate form, which allows the percent sign (%) and underscore (_) as pattern matching characters. This allows multiple packages to be explained with one invocation of db2expln. The escape character can be used to force the % and _ characters to be treated literally. (See the SQL Reference for more information on the LIKE predicate.) If multiple packages may be matched, the section number is automatically set to 0 (all sections). Dynamic Statement Options: -statement <statement> = The dynamic statement <statement> will be -q <sql> explained. -stmtfile <file> = The dynamic statements contained in the file -f <file> <file> will be explained. <File> must exist at the client. -terminator <character> = Each dynamic statement ends at <character>. If -z <character> this option is not specified, then each statement is assumed to be one line long. -noenv = By default, db2expln will invoke each dynamic SET statement after it has been explained. This option prevents the execution of these statements. Explain Options: -graph = Reconstruct the original optimizer plan graph (as -g presented by Visual Explain). Note that the reconstructed graph may not exactly match the original plan. -opids = Show the operator ID numbers. -i The specific options available may vary by database server. Use "db2expln -help -database <db>" to get the options available for a specific server.

db2expln db2expln Notes: Package Options:
-schema <pattern> = The package creator must match <pattern>. -c <pattern> -package <pattern> = The package name must match <pattern>. -p <pattern> -version <pattern> = The package version must match <pattern>. If not specified, then the package with the version '' (the empty string) will be explained. -section <number> = The section number is <number>. Use 0 (zero) for -s <number> all sections in the package. -escape <charater> = Use <character> as the escape character when -e <character> matching patterns. -noupper = Do not upper case creator, package and version -l before matching. The creator and package information must be specified unless dynamic SQL is being explained. If the section information is not specified, then all sections will be displayed. The <pattern> for creator, package, and version is in LIKE predicate form, which allows the percent sign (%) and underscore (_) as pattern matching characters. This allows multiple packages to be explained with one invocation of db2expln. The escape character can be used to force the %

db2expln db2expln Notes:
and _ characters to be treated literally. (See the SQL Reference for more information on the LIKE predicate.) If multiple packages may be matched, the section number is automatically set to 0 (all sections). Dynamic Statement Options: -statement <statement> = The dynamic statement <statement> will be -q <sql> explained. -stmtfile <file> = The dynamic statements contained in the file -f <file> <file> will be explained. <File> must exist at the client. -terminator <character> = Each dynamic statement ends at <character>. If -z <character> this option is not specified, then each statement is assumed to be one line long. -noenv = By default, db2expln will invoke each dynamic SET statement after it has been explained. This option prevents the execution of these statements. Explain Options: -graph = Reconstruct the original optimizer plan graph (as -g presented by Visual Explain). Note that the reconstructed graph may not exactly match the original plan. -opids = Show the operator ID numbers. -i

db2expln Sample DB2 Universal Database Version 7.1, (c) Copyright IBM Corp. 1991, 2000 Licensed Material - Program Property of IBM IBM DATABASE 2 SQL Explain Tool ******************** PACKAGE *************************************** Package Name = NULLID.SQLUJD03 Prep Date = 1999/10/19 Prep Time = 00:00:00 Bind Timestamp = Isolation Level = Cursor Stability Blocking = Block All Cursors Query Optimization Class = 5 Partition Parallel = No Intra-Partition Parallel = No Function Path = "SYSIBM", "SYSFUN", "DB2ADMIN" SECTION This is an sample of db2expln output for an SQL statement. For screen display purposes, some blank lines have been removed.

db2expln Sample SECTION Section = 12 SQL Statement: SELECT COUNT(NAME) INTO :H00037 FROM SYSIBM.SYSCOLUMNS WHERE (TBNAME = :H00029 AND TBCREATOR = :H00030 ) AND(GENERATED != '') Estimated Cost = 149 Estimated Cardinality = 1 Access Table Name = SYSIBM.SYSCOLUMNS ID = 0,3 | #Columns = 3 | Index Scan: Name = SYSIBM.IBM01 ID = 1 | | Index Columns: | | | 1: TBCREATOR (Ascending) | | | 2: TBNAME (Ascending) | | | 3: NAME (Ascending) | | #Key Columns = 2 | | | Start Key: Inclusive Value | | | | 1: ? | | | | 2: ? The statement is shown followed by a fairly clear explanation of the table accessed, the columns returned, the index and the number of key columns used to limit the search, the locking and filter used. The SYSIBM.SYSCOLUMNS table is accessed and 3 of its columns ARE used. An index scan is done using the SYSIBM.IBM01 index (ID=1). The #Key Columns value of 2 means that this number of columns are used.

Design Advisor Design Advisor is a tool used to evaluate SQL statements and workloads and to recommend indexes to improve performance Can accept single SQL statement or a workload as input Ouputs create index statements that can be used to create recommended indexes Should not blindly apply indexes especially for single SQL statements Take entire workload and business priorities into consideration Future enhancements will include recommendations for MQTs and MDC NOTES: DB2 Design Advisor is a very good tool and it can be used from the Control Center or command line. It should be one of the primary tools in a DBAs toolkit. Significant performance improvements can be made to existing production systems using this tool.

Design Advisor NOTES: Design Advisor can be launched from the Control Center or from a command line using the db2advis command. We will use the db2advis command from the command line.

Design Advisor db2advis -d gunprd -I wildsortsql.txt > wildsqlixadvout.txt execution started at timestamp found [1] SQL statements from the input file Calculating initial cost (without recommmended indexes) [ ] timerons Initial set of proposed indexes is ready. Found maximum set of [1] recommended indexes Cost of workload with all indexes included [ ] timerons total disk space needed for initial set [ ] MB total disk space constrained to [ ] MB 1 indexes in current solution [ ] timerons (without indexes) [ ] timerons (with current solution) [%99.69] improvement Trying variations of the solution set.-- -- execution finished at timestamp -- LIST OF RECOMMENDED INDEXES -- =========================== -- index[1], MB CREATE INDEX WIZ1 ON "PUSER "."T_FILE" ("FILE_STATUS_NUM" DESC) ; -- ===========================-- Design Workload tool is finished. NOTES: This is an actual example of an SQL statement from a very active mixed workload system. The index was created and significant reduction in CPU consumption and improved response times were observed. 2

Automatic Summary Tables (ASTs)
ASTs are a special kind of Materialized Query Table (New in V8.1). Powerful way to improve response time for complex queries of the following type: Aggregated data over one or more dimensions Joins and aggregated data over a group of tables Data from a commonly accessed subset of data, that is, from a “hot” horizontal or vertical partition

Repartitioned data from a table, or part of a table, in a partitioned database environment MQTs are new in V8.1. ASTs are now a special kind of MQT. A Summary Table is a table whose definition is based on the results of a query. Summary tables typically contain precomputed results based on data existing in the base table or tables on which its definition is based.

Can provide drastic performance improvements An example is overnight queries now run in minutes DB2 optimizer may determine that the query may run more efficiently against a summary table than the base table(s) Accomplished via DB2 Optimizer query rewrite option “Drastic” is rather a strong term and it is used in this slide to “get your attention”. If you are not using ASTs and in the future, MQTs, then you a losing out on rather significant performance improvements.

How are they created? CREATE TABLE with SUMMARY keyword or if SUMMARY keyword omitted and AS followed by fullselect, DB2 recognizes this as a summary table

Example: CREATE SUMMARY TABLE CLASS_SKU AS (SELECT STOR_SKU, SUM(QTY) AS QUANTITY, SUM(AMT) AS AMOUNT FROM SKU_MASTER GROUP BY STOR_SKU ) DATA INITIALLY DEFERRED REFRESH DEFERRED; A summary table is created with the CREATE TABLE statement using the SUMMARY keyword. IF summary is omitted, DB2 still recognizes a summary table by the keyword AS followed by a fullselect.

The fullselect of a summary table definition cannot contain the following: References to a view Expressions that are a reference type or DATALINK (or a distinct type based on these types) Functions that have an external action Functions that depend on physical characteristics (NODENUMBER, PARTITION)

Table or view references to system objects (explain tables included) Cannot use IMPORT or LOAD utility Cannot create a unique index Limited use of ALTER statement There are several restrictions when using ASTs. The ALTER TABLE statement options that can be used are activating NOT LOGGED INITIALLY, PCTFREE, LOCKSIZE, APPEND or VOLATILE.

Tables created with the REFRESH IMMEDIATE option are subject to the following restrictions: The fullselect in the summary table definition must be a sub-select and cannot include: functions that are not deterministic scalar fullselects predicates with fullselects special registers

A Group By clause must be included in the sub-select The select list must have a COUNT(*) function (or COUNT_BIG) and no DISTINCT Only SUM (if not nullable columns), COUNT, or COUNT_BIG column functions are allowed in the select list (without DISTINCT) and other SELECT list items must be included in the GROUP BY clause.

ALL Group By items must be included in the Select list No grouping sets are allowed (including CUBE and ROLLUP) or GROUPING on constants A HAVING clause is not allowed Although ASTs are a good thing, there are some limitations that you need to be aware of. But, if thought out properly, they can be very useful in a DW environment. When in doubt, use them!

USER-Maintained Materialized Query Tables (MQTs)
V8.1 features that ALLOWS USERS to migrate from ORACLE, which already has this capability. Useful for users already generating summary data through other means, such as a nightly batch job. With a user-maintained summary table, users can load this summarized data. USER-maintained summary tables are no different from system summary tables with the exception that the creation and loading of the summary table is under user control.

USER-Maintained MQTs DB2 does not refresh the data
Responsibility of the user to update data as needed DB2 will assume that data in these tables is correct and may produce incorrect results if the data is not valid. Created by specifying the MAINTAINED BY USER option of the CREATE SUMMARY TABLE statement.

USER-Maintained MQTs Example CREATE SUMMARY TABLE UMST_SALES AS
(Select Region COUNT(*) AS SKU_COUNT, SUM(RSALES) AS REG_SALES SUM(TSALES) AS SORT_TOT From SALES GROUP BY REGION ) DATA INTIALLY DEFERRED REFRESH DEFERRED MAINTAINED BY USER; Here is a sample create table statement. The user will have to load the data into this table. Before loading the data, the summary table has to be taken out of check pending state. Example set integrity command: SET INTEGRITY FOR UMST_SALES ALL IMMEDIATE UNCHECKED;

Automatic Summary Tables
Setting the register to NONE will prevent any summary tables from being used during creation. Set back on if you want to take advantage of the newly created summary table This special register can contain one of four values ALL-specifies that all possible types of maintained tables controlled by this special register, now and in the future, are to be considered when optimizing the processing of dynamic SQL queries.

LOADING the USER-MAINTAINED SUMMARY TABLE INSERT INTO UMST_SALES Select * From (Select REGION, Count(*), SUM(RSALES), SUM (CORP_TOT) From SALES GROUP by REGION ) AST; Statement to use to load the summary table. You may also have to set the special optimization register to prevent other summary tables being used when loading this table. Prior to running the inset statement, issue the following command SET CURRENT MAINTAINED TABLE TYPES FOR OPTIMIZATION=NONE; OPTIMIZATION SETTING

NONE- specifies that none of the object types of maintained tables controlled by this special register, now and in the future, are to be considered when optimizing the processing of dynamic SQL queries.

System- specifies that system maintained refresh-deferred MQTs can be considered to optimize the processing of dynamic SQL queries (Immediate MQTs are always available) User- specifies that user-maintained refresh-deferred MQTs can be considered to optimize the processing of dynamic SQL queries. In addition to this optimization register, you will have to set the CURRENT REFRESH AGE register to ensure that DB2 will use these user-maintained summary table: SET CURRENT REFRESH AGE ANY If this is set to 0, DB2 will only consider summary tables that are automatically maintained by the system.

CURRENT REFRESH AGE special REGISTER Specifies a timestamp duration value with a data type of DECIMAL (20,6) This duration is the maximum duration, since a REFRESH TABLE statement has been processd on a deferred refresh summary table. It determines if a summary table can be used to optimize the processing of a query The set CURRENT REFRESH AGE statement enables you to change the value of the of the CURRENT REFRESH ARE special register. The value must be 0 or (9999 years, 99 months, 99 days, 99 hrs, 99 minutes, and 99 seconds) Zero(0)-means that only summary tables defined with REFRESH DEFERRED or REFRESH IMMEDIATE may be used to optimize the processing of a query.

ASTs defined as REFRESH IMMEDIATE are immediately updated as inserts, updates, and deletes occur on the base tables Changes are cascaded to AST ASTs defined as REFRESH DEFERRED are refreshed as a result of the REFRESH TABLE statement being issued In V8.1 staging tables are used to accomplish a delta refresh of the table

Staging tables are created as part of the CREATE TABLE statement with PROPAGATION IMMEDIATE specified The staging tables contain some “control columns” to assist DB2 in managing the deltas

The keyword ANY is shorthand for Initial value of CURRENT REFRESH AGE is 0. MUST SET to ANY or for DB2 to be able for DB2 to consider using deferred refresh summary tables SET CURRENT REFRESH AGE ANY

REFRESHED DEFERRED Data in the table is refreshed when a REFRESH TABLE statement is executed. The data in the summary table reflects the results of the query at the time the REFRESH TABLE statement is executed.

REFRESH IMMEDIATE Means that the changes made to the underlying tables as part of a DELETE, INSERT, or UPDATE statement are cascaded to the summary table immediately.

Additional CREATE TABLE OPTIONS ENABLE QUERY OPTIMIZATION Means table can be used DISABLE QUERY OPTIMIZATION Means the table (MQT) will not be used for query optimization. But, THE TABLE CAN STILL BE QUERIED DIRECTLY. Other Rules When MAINTAINED BY USER is used, the user is allowed to perform update, delete, or insert operation against user maintained MQTs. The REFRESH TABLE STATEMENT, used for system maintained MQTs, cannot be invoked against USER-maintained MQTs. Only a REFRESH DEFERRED MQT can be defined as MAINTAINED BY USER.

Materialized Query Table (MQT)
An MQT is a table whose definition is based on the result of a query, and whose data is in the form of precomputed results that are taken from one or more tables on which the MQT definition is based. The definition of an MQT contain joins, functions, and other SQL elements that are not allowed in ASTS!

Materialized Query Table (MQT)
MQT Example: CREATE SUMMARY TABLE SMQT_CGS As (SELECT RAWM, DIRECT_COSTS, OVHD, REGION R where C region= R.region) Data initially deferred REFRESH DEFERRED; MQTs can be used to pre-join tables together. This allows the optimizer to bypass joining tables together if the MQT contains the columns and rows that it needs.

MQTs LIST TABLES command can be used to query the system catalog to produce a list of summary tables. The value S in the type column indicates that this table is a summary table.

AST Summary Restrictions Not Allowed: Allowed: Create indexes
Cannot create unique index Limited ALTER CAPABILITY Cannot insert, update or delete unless it is a user-maintained query Allowed: Create indexes Use RUNSTATS Use Explain information to see if summary table is used

Replicated MQT Tables in DPF
Replicated MQTs improve performance of frequent executed joins in a partitioned database environment by allowing the database to manage precomputed values if the table data Designate Replicated MQT on the CREATE TABLE command using the REPLICATED keyword Once created, run RUNSTATS on replicated table Replicated MQTs improve performance in DPF environment as the data does not have to be moved across the network to each database partition. Replicated tables combined with collocated joins are a powerful combination.

AST and MQT Summary ASTs and MQTs are very powerful mechanisms to improve query performance in DW environments. Many queries can be satisfied by MQTs resulting in improved performance. Summary tables are usually much smaller then the base (fact) tables The optimizer can generally elect to use summary tables if the following conditions exist The optimizer chooses to do so via query rewrite Setting of the current REFRESH AGE special register Zero (0) means only summary tables defined with refresh immediate may be used to optimize the processing of a query Any or 9(14) means either summary table with refresh deferred or refresh immediate may be used A good detailed example can be found on pg. 209 of the Administration Guide: Performance.

User-Maintained MQT Summary
The Current Maintained Table Types for Optimization Special Register, must be set to USER to enable user-maintained refresh deferred MQTs to be considered for optimizing the processing of dynamic SQL queries. Note Current Refresh AGE must be set to ANY else if it is 0, DB2 will only consider summary tables that are automatically maintained by the system.

Replicated MQTs Summary
Replicated MQTs combined with collocated joins are a powerful combination to assist you in optimizing the performance of your Data Warehouse.

CLUSTERING Every table should have a clustering index specified
Clustering keeps index and data pages entries in clustering order. Improves insert performance and prefetching of data as requested data is in order. Over time clustering degrades as a result of inserts Only one clustering index is allowed. REORGCHK showed be run on a regular basis to identify candidates for reorganization. Clustering can significantly improve performance. Specify a clustering index on the REORG command will cause the data rows to be placed in the same physical sequence of the index. The clustering index should be the most frequently accessed index. Note, usually only 1 index will be highly clustered. Formula F4 from REORGCHK shows the clusteratio or cluster factor. This ratio shows the percentage of data rows that are stored in the same physical sequence as the index. The clusterratio of an index should be greater then 90%*.

SQL Tuning Workshop Refer to your student workbook and examples for the SQL Tuning Workshop. You have 1 hr to complete the workshop. NOTES:

Module 8 Health Monitor and Health Center

Objectives Upon completion of this module you should understand:
Difference between point and exception monitoring Health center and Health monitor operation Health Center components Health Indicators How to setup up alarms and exception actions NOTES:

Types of Monitoring Point Based Exception Based Visual “Eyes On”
Visual Cues and Alerts Dedicated resource required Real time monitoring Drill down capability Exception Based Server side lightweight agent Alerts based on predefined or default thresholds s and text messages Visual cues/alerts NOTES:

Health Monitor The health monitor is a lightweight server-side agent implemented in the DAS process The Health Center interfaces with the Health Monitor and can be used to configure the Health Monitor The Health Monitor implements exception based monitoring Can operate 24x7 Can generate alerts and actions based on predefined or default threshold breaches Can generate s and send text messages to pagers NOTES: The health monitor uses new monitoring technology that does not impose a performance penalty and does not require or use any monitoring switches.

Health Monitor Default instance level health_mon switch is off by default Must be enabled by issuing “db2 update dbm cfg using health_mon on” NOTES:$ db2 get dbm monitor switches DBM System Monitor Information Collected Switch list for db partition number 0 Buffer Pool Activity Information (BUFFERPOOL) = ON :32: Lock Information (LOCK) = ON :23: Sorting Information (SORT) = ON :32: SQL Statement Information (STATEMENT) = ON :34: Table Activity Information (TABLE) = ON :34: Take Timestamp Information (TIMESTAMP) = ON :04: Unit of Work Information (UOW) = ON :34:

Health Monitor Is not displayed by “db2 get dbm monitor switches” or “db2 get monitor switches” Issue $ db2 get dbm cfg | grep health* Monitor health of instance and databases (HEALTH_MON) = ON to see if switch is enabled or not. NOTES: $ db2 get monitor switches Monitor Recording Switches Switch list for db partition number 0 Buffer Pool Activity Information (BUFFERPOOL) = ON :32: Lock Information (LOCK) = ON :32: Sorting Information (SORT) = ON :32: SQL Statement Information (STATEMENT) = ON :34: Table Activity Information (TABLE) = ON :34: Take Timestamp Information (TIMESTAMP) = ON :04: Unit of Work Information (UOW) = ON :34:

Health Monitor Administration
Refer to Example 19 for an example of a Health Monitor alert and associated recommendations Can ignore, review recommendations or take action “db2 get recommendations for health indicator <health-indicator-short-name>” “db2 get description for health indicator <health-indicator-short-name>” NOTES: “db2 get recommendations for health indicator “db.log_fs_util”. Refer to Example 19 for sample output for this command and associated recommendations.

Health Center Launched from the Control Center
Presents instances and databases using in a tree like format Database status is reported by Health Monitor and beacons display green, amber or red depending on the severity of the problem DBM and DB parameters can be monitored and recommendations provided for parameter setting changes based on recommendations provided NOTES:

Health Center NOTES:

Health Center This page intentionally left blank NOTES:

Health Center NOTES: This is an example of the Health Center receiving an alert for a low catalog cache hit ratio. We can observe that the beacon has changed to Red. We can change view a history plot or change threshold settings. We can also select the recommendations tab as seen on the next page.

Health Center NOTES: From the recommendations dialogue you can launch the memory visualizer to plot current memory usage and to plot it over time.

Memory Visualizer NOTES: Memory Visualizer was launched from the previous dialogue from the Health Center.

DBM Health Indicators DBM health indicators for an instance can be obtained by issuing the following command: “db2 get alert cfg for dbm” NOTES:

DBM Health Indicators Indicator Name = db2.db2_op_status
Type = State-based Sensitivity = 0 Formula = db2.db2_status; Actions = Disabled Threshold or State checking = Enabled NOTES:

DBM Health Indicators Indicator Name = db2.sort_privmem_util
Type = Threshold-based Warning = 90 Alarm = 100 Sensitivity = 0 Formula = ((db2.sort_heap_allocated/sheapthres)*100); Actions = Disabled Threshold or State checking = Enabled NOTES:

DBM Health Indicators Indicator Name = db2.max_sort_privmem_util
Type = Threshold-based Warning = 60 Alarm = 30 Sensitivity = 0 Formula = ((db2.max_priv_sort_mem/sheapthres)*100); Actions = Disabled Threshold or State checking = Enabled NOTES:

DBM Health Indicators Indicator Name = db2.mon_heap_util
Type = Threshold-based Warning = 85 Alarm = 95 Sensitivity = 0 Formula = ((db2.mon_heap_cur_size/db2.mon_heap_max_size)*100); Actions = Disabled Threshold or State checking = Enabled NOTES:

Database Health Indicators
Database Configuration health indicators for a database can be obtained by issuing the following command: “db2 get alert cfg for databases” NOTES:

Indicator Name = db.sort_shrmem_util Type = Threshold-based Warning = 70 Alarm = 85 Sensitivity = 0 Formula = ((db.sort_shrheap_allocated/sheapthres_shr)*100); Actions = Disabled Threshold or State checking = Enabled NOTES:

Indicator Name = db.spilled_sorts Type = Threshold-based Warning = 30 Alarm = 50 Sensitivity = 0 Formula = (db.sort_overflows/db.total_sorts)*100; Actions = Disabled Threshold or State checking = Enabled NOTES:

Indicator Name = db.max_sort_shrmem_util Type = Threshold-based Warning = 60 Alarm = 30 Sensitivity = 0 Formula = ((db.max_shr_sort_mem/sheapthres_shr)*100); Actions = Disabled Threshold or State checking = Enabled NOTES:

Indicator Name = db.log_util Type = Threshold-based Warning = 75 Alarm = 85 Sensitivity = 0 Formula = (db.total_log_used/(db.total_log_used+db.total_log_available))*100; Actions = Disabled Threshold or State checking = Enabled NOTES:

Indicator Name = db.log_fs_util Type = Threshold-based Warning = 75 Alarm = 85 Sensitivity = 0 Formula = ((os.fs_used/os.fs_total)*100); Actions = Disabled Threshold or State checking = Enabled NOTES:

Indicator Name = db.deadlock_rate Type = Threshold-based Warning = 5 Alarm = 10 Sensitivity = 0 Formula = delta(db.deadlocks); Actions = Disabled Threshold or State checking = Enabled NOTES:

Indicator Name = db.locklist_util Type = Threshold-based Warning = 75 Alarm = 85 Sensitivity = 0 Formula = (db.lock_list_in_use/(locklist*4096))*100; Actions = Disabled Threshold or State checking = Enabled NOTES:

Indicator Name = db.lock_escal_rate Type = Threshold-based Warning = 5 Alarm = 10 Sensitivity = 0 Formula = delta(db.lock_escals); Actions = Disabled Threshold or State checking = Enabled NOTES:

Indicator Name = db.apps_waiting_locks Type = Threshold-based Warning = 50 Alarm = 70 Sensitivity = 0 Formula = (db.locks_waiting/db.appls_cur_cons)*100; Actions = Disabled Threshold or State checking = Enabled NOTES:

Indicator Name = db.pkgcache_hitratio Type = Threshold-based Warning = 80 Alarm = 70 Sensitivity = 0 Formula = (1-(db.pkg_cache_inserts/db.pkg_cache_lookups))*100; Actions = Disabled Threshold or State checking = Disabled NOTES:

Indicator Name = db.catcache_hitratio Type = Threshold-based Warning = 80 Alarm = 70 Sensitivity = 0 Formula = (1-(db.cat_cache_inserts/db.cat_cache_lookups))*100; Actions = Disabled Threshold or State checking = Disabled NOTES:

Indicator Name = db.shrworkspace_hitratio Type = Threshold-based Warning = 80 Alarm = 70 Sensitivity = 0 Formula = ((1-(db.shr_workspace_section_inserts/db.shr_workspace_section_lookups))*100); Actions = Disabled Threshold or State checking = Disabled NOTES:

Indicator Name = db.db_heap_util Type = Threshold-based Warning = 85 Alarm = 95 Sensitivity = 0 Formula = ((db.db_heap_cur_size/db.db_heap_max_size)*100); Actions = Disabled Threshold or State checking = Enabled NOTES:

Module 9 Memory Tracker and DB2BATCH

Monitor a database using memory tracker Understand basic memory tracker usage and capabilities Understand basics of db2batch Be able to use db2batch to perform benchmark tests Improve performance of db2 utilities using paralellism NOTES:

Memory Tracker DB2 Memory Tracker is a new tool available in DB2 V8.1
Similar usage to UNIX commands such as VMSTAT, IOSTAT Run from DB2 command line Command and options follow: db2mtrk -i | -d | -p [-m | -w] [-v] [-r interval [count]] [-h] Handy for tracking memory usage over time interval Can be used for point based monitoring and could be scripted and data stored in DB2 tables to show DB2 memory/heap activity over time NOTES:Usage: db2mtrk -i | -d | -p [-m | -w] [-v] [-r interval [count]] [-h] -i Display instance level memory usage -d Display database level memory usage -p Display agent private memory usage -m Display maximum usage information -w Display watermark usage information -v Display verbose memory usage information -r Run in repeat mode interval Amount of seconds to wait between reports count Number of reports to generate before quitting -h Display this help screen Notes: 1. One of -i -d -p must be specified. 2. The -w and -m flags are optional. An invocation of the application is invalid if both flags are specified. 3. The -m flag reports the maximum allowable size for a given heap while the -w flag reports the largest amount of memory allocated from a given heap at some point in its history.

Memory Tracker Output NOTES: Usage scenarios: db2mtrk -i -d
Report current memory usage for instance and all databases db2mtrk -i -p -m Report maximum allowable size for instance and agent private memory db2mtrk -p -r 1 5 Report agent private memory five times at one second intervals Heap Legend: When running in normal mode (i.e. -v flag not specified) heaps are named using the following codes: appctlh - Application Control Heap apph Application Heap bph Buffer Pool Heap catcacheh - Catalog Cache Heap dbh Database Heap dlfmh DFM Heap fcmbp FCMBP Heap ip Import Pool lockh Lock Manager Heap monh Database Monitor Heap other Other Memory pckcacheh - Package Cache queryh - Query Heap stath Statistics Heap stmth Statement Heap utilh Backup/Restore/Util Heap

Memory Tracker Memory Tracker provides a complete report of memory status for instance, databases, and agents The following memory pool allocations are displayed: Current size and maximum size Largest size and type Agent that allocated the pool (if private) NOTES:

Memory Tracker Memory tracker is a good tool that should be integrated into your total monitoring strategy NOTES:

Health Monitoring Workshop
Refer to your student workbook and examples for the Health Monitoring Workshop. You have 1 hr to complete the workshop. NOTES:

Db2batch and Utilities

DB2BATCH db2batch is a benchmarking tool provided with DB2
Reads SQL statements from either a flat file or standard input and dynamically prepares and describes statements and returns an answer set NOTES:

DB2BATCH The number of rows to be fetched, sent to the output file or standard out can be controlled and the level of performance returned SQL statements can be included in a block and information is collected for all of the statements at once, instead of one at a time NOTES:

DB2BATCH Command parameters
ROWS_FETCH – number of rows to be fetched from the answer set ROWS_OUT – Number of rows to be sent to output PERF_DETAIL – Level of performance info to be returned 0 – no timing done 1 – return elapsed time only 2 – return elapsed time and cpu time 3 – return a summary of monitoring information 4 – return a snapshot for DBM, DB, application, statement (single statement) 5 – similar to level 4 but return bufferpool, tablespace, and FCM NOTES:

DB2BATCH Command parameters continued:
DELIMITER SLEEP – Number of seconds to sleep PAUSE – Prompts the user to continue TIMESTAMP – Generates a timestamp Similar parameters for output and the following: o query_optimization_class e explain_mode v Verbose s summary table p ESE DPF only Cli Run db2batch in CLI mode cache_size – size of the statement memory, expressed as number of statements NOTES: db2batch is very useful for benchmarking workloads. It can be used with different SQL statements, tablespace and disk layouts, and varying amounts of data to verify alternative designs.

Utilities Throttling of utilities became available in DB2 V8.1.2
With the introduction of utility throttling, you can regulate the performance impact of maintenance utilities, so that they can be run concurrently with production periods. You can develop a throttling policy that will run the utilities aggressively when the production workload is light, but will run them more conservatively as production demands increase. NOTES: Eliminate performance impact as a consideration when monitoring a utility and configuring its parameters (for example, setting the PARALLELISM parameter for a backup invocation). Once a throttling policy is established, it is the system's responsibility to ensure that the policy is obeyed (to the extent possible). Throttling is controlled by the UTIL_IMPACT_PRIORITY priority option of the backup utility. The priority can be between 1 – 100. The util_lim_impact configuration parameter also has to be set.

Utilities Use parallelism to speed up utilities
Either CPU or I/O parallelism Backup multiple tablespaces in parallel NOTES:

Utilities Online Load Other tables in same talbespace no longer locked out Provides additional flexibility Improves availability NOTES:

DB2 UDB V8.1 References References:
IBM DB2 Universal Database SC Command Reference IBM DB2 Universal Database SC What’s New IBM DB2 Universal Database SC Administration Guide: Planning IBM DB2 Universal Database SC Implementation IBM DB2 Universal Database SC Performance IBM DB2 Universal Database SC SQL Reference, Volume 1 IBM DB2 Universal Database SC SQL Reference, Volume 2 IBM DB2 Universal Database SC System Monitor Guide and Reference NOTES:

DB2 UDB for Linux, UNIX, and Windows Performance and Tuning

DB2 UDB for Linux, UNIX, and Windows Performance and Tuning

Similar presentations

Presentation on theme: "DB2 UDB for Linux, UNIX, and Windows Performance and Tuning"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

DB2 UDB for Linux, UNIX, and Windows Performance and Tuning

Similar presentations

Presentation on theme: "DB2 UDB for Linux, UNIX, and Windows Performance and Tuning"— Presentation transcript:

Similar presentations

About project

Feedback