Why Should I Care About … The Plan Cache? Tuning When Stakeholders Won’t Say Where It Hurts

Rick Lowe rick@data-flowe.com DataFLowe http://dataflowe.wordpress.com/

How We Wish Things Worked Hey Rick, server’s slow. What query are you running? SELECT Title, FirstName FROM Person.Person WHERE LastName = 'Anderson';

How We Wish Things Worked Continued Any chance “Title” is new? We might need to tweak our indexes to support it. Yeah, title is new. I didn’t think it’d make a difference, sorry. NP, I’ll test the change today. Should be in production on Monday.

How Things Actually Work (Or Don’t Work) Hey Rick, server’s slow. What query are you running? Uh, … What?

You Often Can’t Get the Query Users won’t remember what they were doing Users don’t actually type, or even see, query Documentation isn’t detailed enough to contain queries/procedure calls If it was, it’s not up to date Developers won’t have query memorized May vary based on parameters

And Might Get Weird Replies “Can’t you just make everything faster?” “It was fine yesterday, just fix it” “Ask that fancy monitoring tool you bought” “Disk is cheap, lets just index every column. Problem solved!” “Are you trying to blame my code for your database performance?” “So I’ve been reading about MongoDB…”

Tangent – Users Aren’t Stupid This just isn’t their area of expertise. We are surrounded by very complicated devices that have very simple interfaces They may think of the database like a car – complicated, but it either works or it doesn’t You wouldn’t tell your mechanic that you suspect the car broke because you set a gallon of milk on the passenger seat

But Wait, This Is Good News The bad news – not all queries are equal The good news – not all queries are equal! Tuning is only noticeable for queries that waste a lot of resources Queries that waste a lot of resources also use a lot of resources So if we could find out what queries use the most resources, we might work miracles

Outline of a Solution Keep track of queries run Keep track of resources used … aggregated across all runs Query plans too Running all the time Without using extended events 

My Best Friends sys.dm_exec_query_stats – query info Time created, last execution, # executions Elapsed time and worker time Read and write stats sys.dm_exec_sql_text() – query text, given the handle from exec_query_stats sys.dm_exec_query_plan() – query plan, given the handle from exec_query_stats

Requirements SQL Server 2005 or later For query_hash and query_plan_hash, SQL Server 2008 or later View Server State permission

Don’t Write This Down SELECT TOP 10 SUBSTRING(st.text, qs.statement_start_offset / 2 + 1, CASE qs.statement_end_offset WHEN -1 THEN LEN(st.text) ELSE qs.statement_end_offset / 2 END – qs.statement_start_offset / 2) AS stmt, qs.last_execution_time, qs.execution_count, qs.creation_time, qs.total_worker_time, qs.min_worker_time, qs.max_worker_time, qs.total_elapsed_time, qs.min_elapsed_time, qs.max_elapsed_time, qs.total_logical_reads, qs.max_logical_reads, qs.min_logical_reads, qs.total_logical_writes, qs.max_logical_writes, qs.min_logical_writes, qp.query_plan FROM sys.dm_exec_query_stats qs CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) st CROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qp ORDER BY qs.total_elapsed_time DESC

sql server expensive query But I Wanted That Query Tons of examples of the preceding query can be found by searching sql server expensive query For those who download this deck, a good one is http://solutioncenter.apexsql.com/monitor-sql-server-queries-find-poor-performers-dynamic-management-views-and-functions/

Breaking It Down - SELECT SELECT SUBSTRING(…) AS stmt, qs.last_execution_time, …, qp.query_plan The SUBSTRING pulls a single statement out of a stored procedure Most of the rest are goodies from query_stats Query_plan gives us a column we can actually click on to see the estimated plan.

Breaking It Down - FROM We already discussed the DMOs FROM sys.dm_exec_query_stats qs CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) st CROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qp We already discussed the DMOs CROSS APPLY – Call a function for each row, and join results We need to use APPLY instead of JOIN because the functions are taking arguments from query_stats.

Breaking It Down – ORDER BY ORDER BY qs.total_elapsed_time DESC Show me the queries that users had to wait the greatest amount of time for. Why elapsed time? Good estimate for how long a person had to wait for answers

I Do a Lot of Top Ten Lists By descending total elapsed time : How long has somebody been waiting for results By descending total worker time : How much CPU is this query burning By total_logical_writes or total_logical_reads : How much IO is the query doing By last_elapsed_time : How long did this query take during last night’s batch

Things to Remember Not all expensive queries are wasteful Plan cache is not persistent Estimated plans – no actual counts Different settings may mean different plans This means you, ANSI_NULLS Any change in query, even an extra space, can lead to a new plan

You Might Miss Actual Plans What do we do when we have an actual plan? Look for expensive operators. We can do this with estimated plans. Compare estimated vs actual – not in estimated plan. WITH RECOMPILE – not much visibility in actual plans.

No Actual Row Counts? Table scans (instead of clustered index scans) 1 estimated row Most expensive statement in stored procedure different than that in the estimated plan Extreme difference between maximum and minimum values in query_stats (perhaps order by descending max/min )

Tips sys.dm_exec_procedure_stats can be used to look for slow stored procedures DBCC FREEPROCCACHE can flush a single plan (instead of the entire cache)

Thank You Rick Lowe – rick@data-flowe.com DataFLowe http://dataflowe.wordpress.com/

Query Store 2016 Feature Very nice front end to this information Stores a history of the plans – this is a big deal Be careful about forcing plans Why is the plan changing? Would a query hint work? Is it possible a different plan will be appropriate in the future?

Don’t Write This Down Either WITH grp AS( SELECT query_hash, query_plan_hash, MIN(sql_handle) AS sql_handle, MIN(plan_handle) AS plan_handle, MAX(last_execution_time) AS last_execution_time, MIN(creation_time) AS creation_time, SUM(execution_count) AS execution_count, SUM(total_elapsed_time) AS total_elapsed_time, MIN(min_elapsed_time) AS min_elapsed_time, MAX(max_elapsed_time) AS max_elapsed_time FROM sys.dm_exec_query_stats GROUP BY query_hash, query_plan_hash )SELECT TOP 10 st.text, grp.last_execution_time, grp.execution_count, grp.creation_time, grp.total_elapsed_time, grp.min_elapsed_time, grp.max_elapsed_time, qp.query_plan FROM grp CROSS APPLY sys.dm_exec_sql_text(grp.sql_handle) st CROSS APPLY sys.dm_exec_query_plan(grp.plan_handle) qp ORDER BY grp.total_elapsed_time DESC

Question from the audience How can I see which stored procedure an expensive statement came from? Answer : dm_exec_sql_text includes an objectid column, which identifies the proc OBJECT_NAME function can retrieve the procedure name. But only if the correct database is selected (USE or dropdown in SSMS)

Question from the audience contd SELECT TOP 10 SUBSTRING(st.text, qs.statement_start_offset / 2 + 1, CASE qs.statement_end_offset WHEN -1 THEN LEN(st.text) ELSE qs.statement_end_offset / 2 END – qs.statement_start_offset / 2) AS stmt, qs.total_worker_time, qs.total_elapsed_time, qs.total_logical_reads, OBJECT_ID(st.objectid) AS sproc, qp.query_plan FROM sys.dm_exec_query_stats qs CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) st CROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qp ORDER BY qs.total_elapsed_time DESC Alternately, dm_exec_procedure_stats may be helpful