Tuna Helper Proven Process for SQL Tuning Janis Griffin Senior DBA, Confio Software 1

Who Am I? Senior DBA for Confio Software JanisGriffin@confio.com Twitter - @DoBoutAnything Current – 20+ Years in SQL Server & Oracle DBA and Developer Specialize in Performance Tuning Review Performance of 100’s of Databases for Customers and Prospects Common Thread – Paralyzed by Tuning

Agenda Introduction Challenges Identify - Which SQL and Why Gather – Details about SQL Tune – Case Study Monitor – Make sure it stays tuned

Tuna Helper – Proven Process for SQL Tuning Give a man a fish and you feed him for a day. Teach a man to fish and you feed him for a lifetime. Chinese Proverb

Introduction SQL Tuning is Hard This Presentation is an Introduction 3-5 day detailed classes are typical Providing a Framework Helps develop your own processes There is no magic tool Tools cannot reliably tune SQL statements Tuning requires the involvement of you and other technical and functional members of team

Challenges SQL Tuning is Hard – did I mention that? Requires Expertise in Many Areas Technical – Plan, Data Access, SQL Design Business – What is the Purpose of SQL? Tuning Takes Time Large Number of SQL Statements Each Statement is Different Low Priority in Some Companies Vendor Applications Focus on Hardware or System Issues Never Ending

Identify – Which SQL Highest Wait Times (Wait Types) Tracing a Session / Process User / Batch Job Complaints Queries Performing Most I/O (LIO, PIO) Queries Consuming CPU Queries Doing Full Table Scans Known Poorly Performing SQL

Identify – End-to-End Understand the business aspects of statement Who registered yesterday for SQL Tuning Why does the business need to know this How often is the information needed Who uses this information Understand the technical aspects Review ERD Understand tables and the data (at a high level) Understand application architecture Understand the entire process What portion of the total time is database Where is it called from in the application

Identify – End-to-End Time 9

Execution Model (cont) CPU SPID 60 – Running (Needs to perform IO) SPID 51 - Running Waiter List SPID 52 – OLEDB SPID 53 – WRITELOG SPID 54 – PAGEIOLATCH_SH SPID 57 – LCK_M_X SPID 59 – WAITFOR SPID 60 – PAGEIOLATCH_SH CPU Queue SPID 51 – Runnable SPID 61 – Runnable SPID 59 – Runnable

Wait Time Tables (SQL 2000) http://support.microsoft.com/kb/822101 sysprocesses loginame hostname programname spid dbid waittype waittime lastwaittype waitresource sql_handle stmt_start stmt_end cmd sysdatabases dbid name ::fn_get_sql text

Wait Time Tables (SQL 2005/8) http://msdn.microsoft.com/en-us/library/ms188754.aspx http://msdn.microsoft.com/en-us/library/ms188068.aspx sysprocesses loginame hostname programname spid dbid waittype waittime Lastwaittype waitresource sql_handle stmt_start stmt_end cmd dm_exec_sessions login_time login_name host_name program_name session_id dm_exec_requests start_time status sql_handle plan_handle start/stop offset database_id user_id blocking_session wait_type wait_time dm_os_wait_stats wait_type waiting_tasks_count wait_time_ms dm_exec_query_stats execution_count total_logical_writes total_physical_reads total_logical_reads dm_exec_query_plan query_plan dm_exec_sql_text text

Problems with DMVs Can give an overall performance view Can find costly queries and other problems. Contents are from instance startup No way to tie some information to a timeframe. Are these Query stats from today or last week? Need to sample data periodically Be able to go back to 3:00 pm yesterday when problem occurred. How often and when does this query execute?

Measure Database Wait Time Understand the total time a Query spends in Database Measure time while Query executes SQL Server helps by providing Wait Types 14

SQLs – Wait Types - Wait Time Almost 10 hrs on CPU 3.5 hrs on CXPACKET Over 6.5 hours of Total time

Wait Time Scenario Which scenario is worse? SQL Statement 1 Executed 1000 times Caused 10 minutes of wait time for end user Waited 90% of time on “PAGEIOLATCH_SH” SQL Statement 2 Executed 1 time Waited 90% on “LCK_M_X”

Identify – Simplification Break Down SQL Into Simplest Forms Complex SQL becomes multiple SQL Sub-Queries Should be Tuned Separately Tuned SQL in Stored Procedures Separately Get the definition of views Understand Distributed Queries

Identify – Summary Determine the SQL Understand End-to-End Measure Wait Time Simplify Statement

Gather - Metrics Get baseline metrics Collect Wait Type Information How long does it take now What is acceptable (10 sec, 2 min, 1 hour) Collect Wait Type Information Locking / Blocking (LCK) I/O problem (PAGEIOLATCH) Latch contention (LATCH) Network slowdown (NETWORK) May be multiple issues All have different resolutions Document everything in simple language

Gather – Execution Plan SQL Server Management Studio Estimated Execution Plan - can be wrong for many reasons Actual Execution Plan – must execute query, can be dangerous in production and also wrong in test SQL Server Profiler Tracing Event to collect: Performance: Execution Plan (2000) , Showplan XML (2005/8) Works when you know a problem will occur DM_EXEC_QUERY_PLAN Real execution plan of executed query

DM_EXEC_QUERY_PLAN

Example SQL Statement Who registered yesterday for SQL Tuning SELECT s.fname, s.lname, r.signup_date FROM student s INNER JOIN registration r ON s.student_id = r.student_id INNER JOIN class c ON r.class_id = c.class_id WHERE c.name = 'SQL TUNING' AND r.signup_date BETWEEN '2009-04-08 00:00' AND '2009-04-08 23:59' AND r.cancelled = 'N' Execution Time – 1:30 to execute Wait Types – Waits 90% on PAGEIOLATCH_SH

Execution Plan CREATE NONCLUSTERED INDEX [reg_alt] on [dbo].[REGISTRATION] ([CANCELLED],[SIGNUP_DATE]) 23

Database Tuning Advisor 2 Indexes & 4 Additional Statistics

Gather - Relationships CLASS class_id name class_level REGISTRATION class_id student_id signup_date cancelled STUDENT student_id fname lname

Gather – Table Information Table Definition Where does it physically reside Large columns? Data Profile Task / Viewer – Integration Services Existing Indexes Names of all existing indexes Columns those indexes contain Statistic Gathering & Rebuild Schedules

Gather – Summary Metrics Plan Table Relationships Table Information How long does it take currently What does the query wait for (wait types) Plan DM_EXEC_QUERY_PLAN Actual Execution Plan Do not use Estimated Plans unless necessary Table Relationships Table Information Columns, Existing Indexes & Triggers

Tune – Create SQL Diagram SQL Tuning – Dan Tow Great book that teaches SQL Diagramming http://www.singingsql.com registration .04 5 30 1 1 student class .002 select count(1) from registration where cancelled = 'N' and signup_date between '2009-04-08 00:00' and '2009-04-08 23:59' 3562 / 80000 = .0445 select count(1) from class where name = 'SQL TUNING' 2 / 1000 = .002

Tune – New Execution Plan create index IX_CLASS on CLASS(name) Metric – Takes 0:20 to execute Why would an Index Scan still occur on REGISTRATION?

Gather – Existing Indexes

Tune – New Execution Plan create index IX_REGISTRATION on REGISTRATION(class_id) Metric – Takes 0:02 to execute

Tune – Alternative from SSMS create index IX_ALT_REG on registration(cancelled, signup_date) Metric – Takes 0:08 to execute

Query 2 Who cancelled classes within the week SELECT s.lname, c.name, r.signup_date cancel_date FROM student s INNER JOIN registration r ON s.student_id = r.student_id INNER JOIN class c ON r.class_id = c.class_id WHERE r.signup_date BETWEEN '2009-04-02 00:00' AND '2009-04-08 23:59' AND r.cancelled = 'Y'

Tune – Execution Plan

Tune – Create SQL Diagram registration .004 5 30 1 1 student class select count(1) from registration where cancelled = 'Y' and signup_date between '2009-04-02 00:00' and '2009-04-08 23:59' 378 / 80000 = .0047 select count(1) from registration where signup_date between '2009-04-02 00:00' and '2009-04-08 23:59' 28797 / 80000 = .3599 select count(1) from registration where cancelled = 'Y' 523 / 80000 = .0065

Tune – New Execution Plan create index IX_ALT_REG on registration(cancelled, signup_date)

Better Execution Plan Execution Stats – 20 Logical Reads CREATE INDEX reg_alt ON registration(class_id) INCLUDE (signup_date, cancelled) Execution Stats – 20 Logical Reads Metric – Takes 0:00 (sub-second) to execute 37

Monitor Monitor the improvement Monitor for next tuning opportunity Be able to prove that tuning made a difference Take new metric measurements Compare them to initial readings Brag about the improvements – no one else will Monitor for next tuning opportunity Tuning is iterative There is always room for improvement Make sure you tune things that make a difference Shameless Product Pitch - Ignite

Summary Identify Gather Tune Monitor What is the Bottleneck End-to-End view of performance Simplify Gather Metrics – Current Performance Wait Time Execution Plan Object Definitions and Statistics Tune SQL Diagrams – Dan Tow Monitor New Metrics, Wait Time Profile, Execution Plan

Confio Software Wait-Based Performance Tools Igniter Suite Ignite for SQL Server, Oracle, DB2, Sybase Provides Help With Identify Gather Monitor Based in Colorado, worldwide customers Free trial at www.confio.com