Rui Ribeiro MCITP 2010/10/19 Index optimization … …on low peak periods V ENCONTRO DA COMUNIDADE SQLPORT

Topics to be Adressed: –Problem Description; –Implementation Overview: - Pessimistic scenario; -Optimistic scenario. -Schedule Index Maintenance; -Demo

1) Problem Description

1)Problem Description __________________________________________ OLTP SQL Server environment Some DBs > 2 TB Some tables > 500 MB Public Access –[20 hours in a row] x 7 days a week. Daily Processing Batchs

Weekly activity

As always, and as everyone else, I was running short in time, and I had to get it all working A.S.A.P.. In the search of the “wheel” that someone had already invented and which wasn´t a complete answer to my problem…

... I´ve found several Index Optimization solutions … The base of the process I am about to describe here it’s available on:

I stated that it wasn´t a complete answer to my problem because it was impossible to: -Exclude indexes from being optimized on a specific week day; -Perform index maintenance for a short period, and then again start on another short period, starting from the point where it ended before; -It didn´t had a way to help DBA to decide which indexes should(‘nt) be maintained during weekday/weekend

All I needed was a solution to support Index maintenance like this: 1) Imagine I want to start a Index Maintenance plan (IMP) IMPSTEPS CHECKEDACTIONS Has Performs Id Info StartDate EndDate Id Id_CHECKED Id_IMP Id_STEPS StartDate EndDate DBNAME TableName IndexName CurrentFragmentationLevel CurrentPageCountl StartDate EndDate Checked ActiontDate StartDate EndDate Command Comment Result IndexChecked 1N N ) IT will have as many Steps as those required to be performed accordingly time spend in each Job execution 3) N Indexes will be checked 4) In each Index Check, if required, a SQL statement will be used to handle each different Action

- Possibility to configure the maintenance of indexes of a particular database, to be performed automatically or manually - Ability to manage the maintenance of the indexes, on a daily basis; - Ability to take advantage of all the periods of low activity for the maintenance of such indexes. On the final solution reached, I highlight the following characteristics:

Implementation Overview - I am going to consider two scenarios in terms of the implementation of this Index Maintenance, namely: Pessimistic Optimistic

Implementation Overview : - Pessimistic scenario You are just the DBA and you don´t know nothing about the business behind the environment you are responsible for. You can also say that you don´t receive any help from the person which is supposed to know !...

Implementation Overview : - Optimistic scenario You understand the business behind the environment you are responsible for. You have all the help you need from the owner of the business.

Pessimistic scenario Collect information and configure/tune Index Maintenance Exclusion –At least once per month you have to (should!) collect information about each table regarding number of rows and data size on each one. –With the previous information, it is possible, using Pareto’s Principle (80-20 rule) to choose 20% most significant tables, that are going to be excluded from regular Index Maintenance, on regular days. Making those 20% only available to maintenance on Saturdays.

Pessimistic scenario Notes: –Why should we exclude the 20% most significant tables ? You can be sure that, in the least, the overhead caused by the smallest maintenance over the index of those 20% most significant tables will/could be significant. And the more complex the environment is, the probability of having locks, deadlocks, and related problems, increases ! So the point is: Why not just avoid any problem generated by this issue, during week days (critical ones!) ?

Optimistic scenario Manually configure Index Defrag Exclusion –Lets take a look on the table which is going to be used: - … to configure when is possible to optimize a specific index; - … to exclude a specific table or database from the optimization process. –If there isn’t any reference to a specific table/database in this table, that means that there isn’t any exclusion at all on that object !!!

Optimistic scenario Manually configure Index Defrag Exclusion Create Table dbo.DBA_indexDefragExclusion ( databaseID int Not Null, databaseName nvarchar(128) Not Null, objectID int Not Null, objectName nvarchar(128) Not Null, indexID int Not Null, indexName nvarchar(128) Not Null, exclusionMask int Not Null /* 1=Sunday, 2=Monday, 4=Tuesday, 8=Wednesday, 16=Thursday, 32=Friday, 64=Saturday Take a SUM of the values for the days that you want excluded. So if you want an index to only be defragged on weekends, you would add up Monday through Friday ( ) and use a value of 62 for the exclusionMask column. */ Constraint PK_indexDefragExclusion_v40 Primary Key Clustered (databaseID, objectID, indexID) );

Optimistic scenario Manually configure Index Defrag Exclusion At this point there are 3 possible actions: –Exclude all indexes from a specific database; –Configure some tables to be optimized whenever is suitable to business; –You don´t need /want to exclude anything !

Optimistic scenario: Manually configure Index Defrag Exclusion Action: Exclude all indexes from a specific database; USE DBNAME_Of_DB_2_Exclude go insert [DBA].dbo.DBA_indexDefragExclusion selectdb_id() as 'databaseID', db_name(db_id()) as 'databaseName', a.id as 'objectID', a.name as 'objectName', b.index_id as 'indexID', b.name as 'indexName', 127 as 'exclusionMask' from sys.sysobjects a join sys.indexes b on b.object_id = a.id and b.type > 0 where a.xtype like 'U'

Optimistic scenario: Manually configure Index Defrag Exclusion Action: Configure some tables to be optimized whenever is suitable to business; USE DBNAME_Of_DB_Selected go -- Only to be "worked" on weekend ! insert [DBA].dbo.DBA_indexDefragExclusion selectdb_id() as 'databaseID', db_name(db_id()) as 'databaseName', a.id as 'objectID', a.name as 'objectName', b.index_id as 'indexID', b.name as 'indexName', 62 as 'exclusionMask' from sys.sysobjects a join sys.indexes b on b.object_id = a.id and b.type > 0 where a.xtype like 'U' and a.name in ('Table1','Table2','TableA','TableZ')

Optimistic scenario: Manually configure Index Defrag Exclusion Action: You don´t need /want to exclude anything ! Great !!!

Schedule Index Maintenance After having configured which table should be ex/included on regular Index Maintenance in an automatic way (or after a certain manual tuning), we should have on table DBA_indexDefragExclusion all information required to make sure that no action is going to be performed over any Index in all critical tables, except in weekends (or whenever we wish !).

Schedule Index Maintenance: Schedules 1)Regular days (Monday to Friday) In my scenario, I had a non peak period between 20:00 and 22:00. Just to be sure that the Index Maintenance operation was not going to be extended too much from the limit time, I have used a time limit of 60 minutes. This way, if the Index Maintenance operation started a new index Maintenance on the 59 minute, I would have another 60 minutes of the non peak period … So, the job would ran on the regular days (from Monday to Friday), from 20:00 to 22:00.

Schedule Index Maintenance: Code 1)Regular days (Monday to Friday) EXECUTE = = = = = = = = = = min x 60 Secs

Schedule Index Maintenance: Schedules 2)Weekend (Saturday) Once again in my scenario, I have considered a low peak period on Saturday from 06:00 to 18:00, leaving time for the Systems Team to update system security patches or something else that could be required. So, the job would ran only during Saturday, from 06:00 to 18:00. Note: Regarding schedules, the main idea is that you can/should grab each non peak period and use it to perform this action of Index Optimization

Schedule Index Maintenance: Code 2)Weekend (Saturday) EXECUTE = = = = = = = = = = -- 10h x 60 min x 60 Secs = = 10h – Give 2 extra hours if Index Maintenance takes too long, considering that we are dealing with Indexes related with heavy/critical tables.

Last minute... More details... Questions... Other... And that’s all !