Welcome To the 2016 Query Store Janis Griffin Senior DBA / Performance Evangelist

Microsoft Data Amp WHERE DATA GETS TO WORK Put data, analytics and artificial intelligence into the heart of your solutions. Get the latest on big data and machine learning innovations. Join us online April 19, 2017 at 8AM PT microsoft.com/data-amp

Who am I Senior DBA / Performance Evangelist for Solarwinds JanisGriffin@solarwinds.com Twitter® - @DoBoutAnything Current – 25+ Years in Oracle®, SQL Server®, ASE, and now MySQL® DBA and Developer Specialize in Performance Tuning Review Database Performance for Customers and Prospects Common Question – How do I tune it?

Agenda What is Query Store? Why use Query Store? How Query Store Works In the Beginning: Tracing, DMVs & Extended Events Overview of Query Store Why use Query Store? Benefits & Gotchas How Query Store Works Enabling, configuring & managing it Best practice considerations – configuration & storage Performance Impact Reporting & Several Use Cases Demos - using both scripts & SSMS

In the Beginning… User complains query is slow What changed? DBA say nothing has changed No new releases No additional load What changed? Changes in data causing out-of-date statistics Changes in execution parameters Parameter sniffing and/or data skew System changes Troubleshooting plan changes difficult in previous versions Had to trace Had to know xQuery to decipher the plan cache XML

In the Beginning… <2005 had tracing High overhead Difficult to use (had to start & stop) 2005+ DMVs: dm_exec_cached_plans & query_stats Real time only or since last restart or clear Can’t see history of plan changes or regression overtime 2012+ Extended Events Easier to use than trace System Health session on by default No history of plan changes / regression

What is Query Store? New feature in 2016 Provides insight on Query Plans Useful in troubleshooting performance degradation Quickly find differences when query plan changes Can see history of plans and runtime statistics Data is persisted in database Not just stored in memory Shows workload and database usage patterns Makes upgrades easier Between versions of SQL Server Put in compatibility mode 100 (lowest) or keep at prior level Turn on query store Collect representative workload Force plans where needed

Benefits Of Using Query Store Simplifies performance tuning Can see plan changes & quickly compare their differences Can aggregate runtime statistics over different time intervals Can enable at database level You get to control when, where and how you use it Tracks both runtime and compile time metrics Has graphical interface in SSMS Supported on all SQL Server editions

Benefits – cont. Allows forcing or pinning of a better plan for a query Can be a quick temporary fix However, can be easily forgotten When data and/or schema changes Easy T-SQL for scripting Can quickly find queries: Which frequently fail Which use literals Which are resource intensive and/or most regressed over time Many SPs, QSD Wait types, & Extended Events For management purposes

How Query Store Works https://msdn.microsoft.com/en-us/library/mt631173.aspx

How Query Store Works – cont. https://msdn.microsoft.com/en-us/library/mt631173.aspx

Enabling the Query Store Query Store is local for each database Is disabled by default To enable in SSMS

Changing Query Store Settings SSMS T-SQL ALTER DATABASE AdventureWorks SET QUERY_STORE = ON ( OPERATION_MODE = READ_WRITE, CLEANUP_POLICY = ( STALE_QUERY_THRESHOLD_DAYS = 30 ), DATA_FLUSH_INTERVAL_SECONDS = 900, MAX_STORAGE_SIZE_MB = 100, INTERVAL_LENGTH_MINUTES = 10);

View Query Store Settings https://msdn.microsoft.com/en-us/library/dn818146.aspx Must monitor state and size! If full or state changes, errors aren’t written to log. select * from sys.database_query_store_options;

Query Store Settings Operation Mode (Requested) / operation_mode Default value: Off Valid values are Off, Read Write, Read Only T-SQL: off, read_write, read_only Actual operation mode can change to “read only” with full Consider monitoring – change not written to logs Data Flush Interval (Minutes) When data is written from memory to disk Default is 15 minutes T-SQL: DATA_FLUSH_INTERVAL_SECONDS 900 secs = 15 min

Query Store Settings – cont. Statistics Collection Interval Defaults to 1 hour Consider changing to 10 minutes to get granular aggregation Higher frequency will require more storage Lowest level – 1 minute T-SQL: INTERVAL_LENGTH_MINUTES Max Size (MB) Maximum size allocated to the Query Store Stored in primary file of database When full, changes to read-only T-SQL: MAX_STORGE_SIZE_MB ALTER DATABASE AdventureWorks SET QUERY_STORE CLEAR;

Query Store Settings – cont. Query Store Capture Mode ‘All’ collects all plans (default) ‘Auto’ captures queries based on resource consumption ‘None’ stops the capture process T-SQL: QUERY_CAPTURE_MODE Size Based Cleanup Mode Activates data cleanup when it reaches maximum size Defaults to ‘Off’ Consider changing to ‘Auto’ Flushes older queries and trivial plans as new data comes in Will not change to ‘read only’ T-SQL: SIZE_BASED_CLEANUP_MODE

Query Store Settings– cont. Stale Query Threshold (Days) Duration to retain query store statistics Default 367 days Consider changing to 30 days or less T-SQL: CLEANUP_POLICY (STALE_QUERY_THRESHOLD_DAYS) MAX_PLANS_PER_QUERY Default is 200 plans Have you ever seen 200 plans to 1 query? Consider changing to something less Must change via T-SQL not in SSMS

Query Store Considerations Consider enabling in the MODEL database In SSMS - no Query Store property in MODEL db Need to turn on via T-SQL ALTER DATABASE MODEL SET QUERY_STORE=ON; Consider proper sizing of Query Store Default is 100mb May be too small for busy database By default, it converts to read-only when full select max_storage_size_mb - current_storage_size_mb qs_free_space_mb from sys.database_query_store_options;

Gotchas Of Using Query Store Disabled by default Need to enable at each individual database level Can modify Model so it’s on by default Consider sizing / performance implications Can’t enable for master or tempdb Can’t enable for Read-Only databases E.g. AG replication for read-only No global way to control configuration Without manual scripting or setting up Model DB DBAs can easily change in SSMS so hard to track Changes to QS settings are logged in ERRORLOG

Gotchas – cont. Query Store (QS) requires care & feeding Default sizes may be too small for a busy database If QS storage is full, switches to READ-Only mode May miss important performance info Query Store data is stored in primary file May conflict or compete for resource with user data Can add 3-5% load to the database load May take longer database restores Plans may contain literal values (HIPAA) No additional context – Who ran it, etc…

Query Store Management Don’t blindly turn it on / plan, plan, plan Consider what you will use it for Which databases are enabled / what’s the retention period How will you manage it? Must monitor QS status, performance & storage Create standard setup & configuration scripts Consider backups / restores Restore into development / test or NOT Replication – sensitive data, forced plans, etc…

Query Store Performance Impact Microsoft states 3% to 5% impact on average Depends on # of databases monitored Transaction / Compile rates Query Store Configurations Can monitor Query Store via Perfmon Counters QDS Wait Types Extended Events

Query Store Performance Impact 58 QS events Not very well documented

Management – Stored Procedures sp_query_store_flush_db; Flushes in-memory portion of data to disk sp_query_store_reset_exec_stats plan_id; Clears runtime stats for a specific query plan sp_query_store_force_plan query_id, plan_id; Enables forcing a particular plan, if forcing fails a Xevent is fired to optimize normally sp_query_store_unforce_plan query_id, plan_id; Removes forcing a particular plan for a particular query sp_query_store_remove_plan plan_id; Removes a single plan from the Query Store sp_query_store_remove_query query_id; Removes a single query, all associated plans and statistics from Query Store

Query Store DMVs … query_store_query_text - text entered by user, includes whites space, hints, etc... msdn-dn818159.aspx query_context_settings - unique combinations of plan affecting settings under which queries run msdn-dn818148.aspx query_store_plan - execution plan information for queries msdn-dn818155.aspx query_store_query - query information and overall aggregated runtime statistics msdn-dn818156.aspx query_store_runtime_stats - runtime execution statistics for queries (avg,min,max,std deviation) msdn-dn818158.aspx query_store_runtime_stats_interval - start and end times intervals for statistics collected msdn-dn818147.aspx

Query Store Reporting & Use Cases SSMS Reports Force Plans Compare Plans See Resource Consumption T-SQL Useful Queries Find Multiple Plans Find Long Running Queries Find Queries Doubling in time Find AD-Hoc Queries Using literals Plan Guides versus Forced Plans

Demo Time Review Properties Look at T-SQL Code Review SSMS Reports Database defaults Turn on QS in Model Database with QS on Show full QS Look at T-SQL Code Multiplan Queries Long running – doubled in duration (microseconds) Parameter Canidates Review SSMS Reports Force a Plan Show Forced Plan Failures Extended Events Q & A

SSMS - Regressed Queries Report

Overall Resource Consumption

Top Resource Consuming Queries Canceled Failed

Tracked Queries

Compare Query Plans

Forcing Plans To remove force plan via TSQL: sp_query_store_unforce_plan 615,665; SELECT p.plan_id, p.query_id, q.object_id as containing_object_id, force_failure_count, last_force_failure_reason_desc FROM sys.query_store_plan p JOIN sys.query_store_query q on p.query_id = q.query_id WHERE is_forced_plan = 1;

Failed Forced Plans Reasons why plan forcing failed 0: no failure 8637: ONLINE_INDEX_BUILD 8683: INVALID_STARJOIN 8684: TIME_OUT 8689: NO_DB 8690: HINT_CONFLICT 8691: SETOPT_CONFLICT 8694: DQ_NO_FORCING_SUPPORTED 8698: NO_PLAN 8712: NO_INDEX 8713: VIEW_COMPILE_FAILED <other value>: GENERAL_FAILURE

Useful Query Store Queries https://msdn.microsoft.com/en-us/library/dn817826.aspx ;WITH Query_MultPlans AS ( SELECT COUNT(*) AS cnt, q.query_id FROM sys.query_store_query_text AS qt JOIN sys.query_store_query AS q ON qt.query_text_id = q.query_text_id JOIN sys.query_store_plan AS p ON p.query_id = q.query_id GROUP BY q.query_id HAVING COUNT(distinct plan_id) > 1 ) SELECT q.query_id, object_name(object_id) AS ContainingObject, query_sql_text, plan_id, convert(xml,p.query_plan) AS plan_xml, p.last_compile_start_time, p.last_execution_time FROM Query_MultPlans AS qm JOIN sys.query_store_query AS q ON qm.query_id = q.query_id JOIN sys.query_store_plan AS p ON q.query_id = p.query_id JOIN sys.query_store_query_text qt ON qt.query_text_id = q.query_text_id ORDER BY query_id, plan_id; GO

Multi-plan Queries

Long Running – Last Hour ;WITH AggregatedDurationLastHour AS ( SELECT q.query_id, SUM(count_executions * avg_duration) AS total_duration, COUNT (distinct p.plan_id) AS number_of_plans FROM sys.query_store_query_text AS qt JOIN sys.query_store_query AS q ON qt.query_text_id = q.query_text_id JOIN sys.query_store_plan AS p ON q.query_id = p.query_id JOIN sys.query_store_runtime_stats AS rs ON rs.plan_id = p.plan_id JOIN sys.query_store_runtime_stats_interval AS rsi ON rsi.runtime_stats_interval_id = rs.runtime_stats_interval_id WHERE rsi.start_time >= DATEADD(hour, -1, GETUTCDATE()) AND rs.execution_type_desc = 'Regular' GROUP BY q.query_id ) ,OrderedDuration SELECT query_id, total_duration, number_of_plans, ROW_NUMBER () OVER (ORDER BY total_duration DESC, query_id) AS RN FROM AggregatedDurationLastHour ) SELECT qt.query_sql_text, object_name(q.object_id) AS containing_object, q.query_id, p.plan_id,rsi.start_time as interval_start, rs.avg_duration, CONVERT(xml, p.query_plan) AS query_plan_xml FROM OrderedDuration od JOIN sys.query_store_query AS q ON q.query_id = od.query_id JOIN sys.query_store_query_text AS qt ON q.query_text_id = qt.query_text_id JOIN sys.query_store_plan AS p ON q.query_id = p.query_id JOIN sys.query_store_runtime_stats AS rs ON rs.plan_id = p.plan_id JOIN sys.query_store_runtime_stats_interval AS rsi ON rsi.runtime_stats_interval_id = rs.runtime_stats_interval_id WHERE rsi.start_time >= DATEADD(hour, -1, GETUTCDATE()) AND number_of_plans > 1 ORDER BY total_duration DESC, query_id, rsi.runtime_stats_interval_id, p.plan_id

Long Running – Last Hour

Execution Time - Doubled in Last 48hrs SELECT qt.query_sql_text, q.query_id, qt.query_text_id, rs1.runtime_stats_id AS runtime_stats_id_1, rsi1.start_time AS interval_1, p1.plan_id AS plan_1, rs1.avg_duration AS avg_duration_1, rs2.avg_duration AS avg_duration_2, p2.plan_id AS plan_2, rsi2.start_time AS interval_2, rs2.runtime_stats_id AS runtime_stats_id_2 FROM sys.query_store_query_text AS qt JOIN sys.query_store_query AS q ON qt.query_text_id = q.query_text_id JOIN sys.query_store_plan AS p1 ON q.query_id = p1.query_id JOIN sys.query_store_runtime_stats AS rs1 ON p1.plan_id = rs1.plan_id JOIN sys.query_store_runtime_stats_interval AS rsi1 ON rsi1.runtime_stats_interval_id = rs1.runtime_stats_interval_id JOIN sys.query_store_plan AS p2 ON q.query_id = p2.query_id JOIN sys.query_store_runtime_stats AS rs2 ON p2.plan_id = rs2.plan_id JOIN sys.query_store_runtime_stats_interval AS rsi2 ON rsi2.runtime_stats_interval_id = rs2.runtime_stats_interval_id WHERE rsi1.start_time > DATEADD(hour, -48, GETUTCDATE()) AND rsi2.start_time > rsi1.start_time AND p1.plan_id <> p2.plan_id AND rs2.avg_duration > 2*rs1.avg_duration ORDER BY q.query_id, rsi1.start_time, rsi2.start_time;

Execution Time - Doubled in Last 48hrs Plan_1 = 370ms Plan_2 = 3.7s

Summary Query Store helps you manage query performance See plan changes & performance regression Find bad parameter sniffing & stale statistics See resource consumption at query & database level Info is only kept in database where query runs If you run them out of master or tempdb, they won’t be kept Need to plan & maintain the Query Store Size appropriately for workload Consider how long to keep the data Consider backups & restores

Resolve performance issues quicklY Try Database Performance Analyzer FREE for 14 days Improve root cause of slow performance Quickly identify root cause of issues that impact end-user response time See historical trends over days, months, and years Understand impact of VMware® performance Agent-less architecture with no dependence on Oracle® Packs; installs in minutes