to my madness! Janis Griffin Senior DBA, Confio Software Tuna Helper Proven Process for SQL Tuning Janis Griffin Senior DBA, Confio Software Query Tuning Get it Right the First Time Janis Griffin Senior DBA, Confio Software Looney Tuner? No, there IS a method to my madness! Janis Griffin Senior DBA, Confio Software 1 1

Who Am I? Senior DBA for Confio Software JanisGriffin@confio.com Twitter - @DoBoutAnything Current – 24+ Years in SQL Server & Oracle DBA and Developer Specialize in Performance Tuning Review Database Performance for Customers and Prospects Common Thread – Paralyzed by Tuning

Agenda Challenges Of Tuning Utilize Response Time Analysis (RTA) Who should tune Which SQLs to tune Utilize Response Time Analysis (RTA) Gather Details about SQL Example Query Execution Plans Use SQL or Query Diagramming Who registered yesterday for Tuning Class Lookup paycheck information Check order status Monitor – Make sure it stays tuned

Challenges Of Tuning SQL Tuning is Hard Requires Expertise in Many Areas Technical – Plan, Data Access, SQL Design Business – What is the Purpose of SQL? Tuning Takes Time Large Number of SQL Statements Each Statement is Different Low Priority in Some Companies Vendor Applications Focus on Hardware or System Issues Never Ending

Who Should Tune Developers? DBA? Need a team approach Developing applications is very difficult Typically focused on functionality Not much time left to tune SQL Don’t get enough practice or simply don’t know SQL runs differently in Production versus Dev/Test DBA? Do not know the code like developers do Focus on “Keep the Lights On” Very complex environment Need a team approach

Which SQL to Tune User / Batch Job Complaints Known Poorly Performing SQL Queries Performing Most I/O (LIO, PIO) Table or Index Scans Queries Consuming CPU Highest Response Times (Ignite8) SELECT sql_handle, statement_start_offset, statement_end_offset, plan_handle, execution_count, total_logical_reads, total_physical_reads, total_elapsed_time, st.text FROM sys.dm_exec_query_stats AS qs CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) AS st ORDER BY total_elapsed_time DESC

Response Time Analysis (RTA) Focus on Response Time Understand the total time a Query spends in Database Measure time while Query executes SQL Server helps by providing Wait Types 7

Wait Time Tables (SQL 2005/8) http://msdn.microsoft.com/en-us/library/ms188754.aspx http://msdn.microsoft.com/en-us/library/ms188068.aspx sysprocesses loginame hostname programname spid dbid waittype waittime Lastwaittype waitresource sql_handle stmt_start stmt_end cmd dm_exec_sessions login_time login_name host_name program_name session_id dm_exec_requests start_time status sql_handle plan_handle start/stop offset database_id user_id blocking_session wait_type wait_time dm_os_wait_stats wait_type waiting_tasks_count wait_time_ms dm_exec_query_stats execution_count total_logical_writes total_physical_reads total_logical_reads dm_exec_query_plan query_plan dm_exec_sql_text text

Problems with DMVs Can give an overall performance view Can find costly queries and other problems. Contents are from instance startup No way to tie some information to a timeframe. Are these Query stats from today or last week? Need to sample data periodically Be able to go back to 3:00 pm yesterday when problem occurred. How often and when does this query execute?

Base Monitoring Query INSERT INTO SessionWaitInfo SELECT r.session_id, r.sql_handle, r.statement_start_offset, r.statement_end_offset, r.plan_handle, r.database_id, r.blocking_session_id, r.wait_type, r.query_hash, s.host_name, s.program_name, s.host_process_id, s.login_name, CURRENT_TIMESTAMP cdt FROM sys.dm_exec_requests r INNER JOIN sys.dm_exec_sessions s ON s.session_id = r.session_id WHERE r.status <> 'background' AND r.command <> 'AWAITING COMMAND' AND s.session_id > 50 AND s.session_id <> @@SPID

RTA Tuning Data

RTA Benefits Before After

Wait Time Scenario Which scenario is worse? SQL Statement 1 Executed 1000 times Caused 10 minutes of wait time for end user Waited 90% of time on “PAGEIOLATCH_SH” SQL Statement 2 Executed 1 time Waited 90% on “LCK_M_X”

Gather Metrics Get baseline metrics Collect Wait Type Information How long does it take now What is acceptable (10 sec, 2 min, 1 hour) Collect Wait Type Information Locking / Blocking (LCK) I/O problem (PAGEIOLATCH) Latch contention (LATCH) Network slowdown (NETWORK) May be multiple issues All have different resolutions

Get Execution Plan SQL Server Management Studio Estimated Execution Plan - can be wrong for many reasons Actual Execution Plan – must execute query, can be dangerous in production and also wrong in test SQL Server Profiler Tracing Event to collect: Performance: Execution Plan (2000) , Showplan XML (2005/8) Works when you know a problem will occur DM_EXEC_QUERY_PLAN Real execution plan of executed query

DM_EXEC_QUERY_PLAN

Review Table Information Table Definition Where does it physically reside Large columns? Data Profile Task / Viewer – Integration Services Existing Indexes Names of all existing indexes Columns those indexes contain Statistic Gathering & Rebuild Schedules

Case Studies SQL Diagramming Who registered yesterday for Tuning Class Lookup paycheck information Check order status

SQL Statement 1 Who registered yesterday for SQL Tuning SELECT s.fname, s.lname, r.signup_date FROM student s INNER JOIN registration r ON s.student_id = r.student_id INNER JOIN class c ON r.class_id = c.class_id WHERE c.name = 'SQL TUNING' AND r.signup_date BETWEEN @BeginDate AND @EndDate AND r.cancelled = 'N' Execution Stats – 17,910 Logical Reads Execution Time – 1:30 to execute Wait Types – Waits 90% on PAGEIOLATCH_SH

Relationship Diagram

Execution Plan Recommendation from SSMS CREATE NONCLUSTERED INDEX [<Name of Missing Index>] ON [dbo].[registration] ([cancelled],[signup_date]) INCLUDE ([student_id],[class_id]) 17,910 Logical Reads

Database Tuning Advisor 2 Indexes & 4 Additional Statistics 22

SQL Diagramming Great Book “SQL Tuning” by Dan Tow registration Great book that teaches SQL Diagramming http://www.singingsql.com registration .04 5 30 1 1 student class .001 select count(1) from registration where cancelled = 'N' and signup_date between '2010-04-23 00:00' and '2010-04-24 00:00' 64,112 / 1,740,145 = 0.0368 select count(1) from class where name = 'SQL TUNING' 2 / 1,267 = .001

New Execution Plan Execution Stats – 17,152 Logical Reads CREATE INDEX cl_name ON class(name) Execution Stats – 17,152 Logical Reads Why would an Index Scan still occur on REGISTRATION?

Gather – Existing Indexes

New Execution Plan Execution Stats – 266 Logical Reads CREATE INDEX reg_alt ON registration(class_id) Execution Stats – 266 Logical Reads

Better Execution Plan Execution Stats – 20 Logical Reads CREATE INDEX reg_alt ON registration(class_id) INCLUDE (signup_date, cancelled) Execution Stats – 20 Logical Reads Metric – Takes 0:00 (sub-second) to execute 27

Alternative from SSMS CREATE INDEX reg_can ON registration(cancelled, signup_date) INCLUDE (class_id, student_id) CREATE NONCLUSTERED INDEX [<Name of Missing Index>] ON [dbo].[registration] ([class_id],[cancelled],[signup_date]) INCLUDE ([student_id])

SQL Statement 2 Current paychecks for specific employees SELECT e.first_name, e.last_name, l.description FROM emp e INNER JOIN loc l ON e.loc_id = l.loc_id WHERE (e.first_name = @fname OR e.last_name = @lname) AND EXISTS ( SELECT 1 FROM wage_pmt w WHERE w.emp_id = e.emp_id AND w.pay_date>= DATEADD(day,-31,CURRENT_TIMESTAMP) ) Execution Stats – 64,206 Logical Reads

Relationship Diagram

Execution Plan

SQL Diagramming wage_pmt emp loc select count(1) from wage_pmt .02 90 1 emp .0005 .0009 1000 1 loc select count(1) from wage_pmt where pay_date >= DATEADD(day,-31,CURRENT_TIMESTAMP) 40,760 / 1,915,088 = .02 select top 5 first_name, count(1) from emp group by first_name order by 2 desc 12 / 23,798 = .0005 – first_name 22 / 23,789 = .0009 – last_name

New Execution Plan CREATE INDEX ix1_last_name ON emp(last_name) CREATE INDEX ix2_fname ON emp(first_name)

Which Index? SSMS Recommendation CREATE INDEX wp_pay_date ON wage_pmt(pay_date) INCLUDE (emp_id) 50,000 Logical Reads or… Better Option CREATE INDEX wp_emp_pd ON wage_pmt(emp_id, pay_date) 46 Logical Reads

New Execution Plan CREATE INDEX wp_emp_pd ON wage_pmt(emp_id, pay_date)

SQL Statement 3 Lookup order status for caller SELECT o.OrderID, c.LastName, p.ProductID, p.Description, sd.ActualShipDate, sd.ShipStatus, sd.ExpectedShipDate FROM [Order] o INNER JOIN Item i ON i.OrderID = o.OrderID INNER JOIN Customer c ON c.CustomerID = o.CustomerID INNER JOIN ShipmentDetails sd ON sd.ShipmentID = i.ShipmentID LEFT OUTER JOIN Product p ON p.ProductID = i.ProductID LEFT OUTER JOIN Address a ON a.AddressID = sd.AddressID WHERE c.LastName LIKE ISNULL(@LastName,'') + '%' --AND c.FirstName LIKE ISNULL(@FirstName,'') + '%' AND o.OrderDate >= DATEADD(day, -30, CURRENT_TIMESTAMP) AND sd.ShipStatus <> 'C' Execution Stats – 10,159 Logical Reads

Database Diagram

Execution Plan

SQL Diagramming o i c sd p a .08 .005 i c .03 sd p a SELECT COUNT(1)*1.0/(SELECT COUNT(1) FROM Customer) FROM Customer WHERE LastName LIKE 'SMI%' .03 SELECT COUNT(1)*1.0/(SELECT COUNT(1) FROM [Order]) FROM [Order] WHERE OrderDate >= DATEADD(day, -30, CURRENT_TIMESTAMP) .08 WHERE OrderStatus <> 'C' .005 -- Combined

Data Skew Problems Only 0.5% of rows are <> ‘C’ SELECT OrderStatus, COUNT(1) FROM [Order] GROUP BY OrderStatus Only 0.5% of rows are <> ‘C’ How about changing the query? AND o.OrderStatus = 'I' Add an Index on ShipStatus

New Execution Plan Execution Stats – 3,052 Logical Reads CREATE INDEX IX2_OrderStatus ON [Order] (OrderStatus) INCLUDE (OrderID,CustomerID) Execution Stats – 3,052 Logical Reads

Monitor Monitor the improvement Monitor for next tuning opportunity Be able to prove that tuning made a difference Take new metric measurements Compare them to initial readings Brag about the improvements – no one else will Monitor for next tuning opportunity Tuning is iterative There is always room for improvement Make sure you tune things that make a difference Shameless Product Pitch - Ignite

Takeaway Points Tuning Queries gives more “bang for the buck” Make sure you are tuning the correct query Use Wait Types and Response Time Analysis Locking problems may not be a Query Tuning issue Wait types tell you where to start Use “Real Execution Plans” Get plan_handle from DM_EXEC_REQUESTS Pass plan_handle to DM_EXEC_QUERY_PLAN() SQL Diagramming Query Tuner Tools can mislead you Monitor For Next Tuning Opportunity

Confio Software Wait-Based Performance Tools Igniter Suite Ignite for SQL Server, Oracle, DB2, Sybase Helps show which SQL to tune Based in Colorado, worldwide customers Free trial at www.confio.com