SQL Server Stored Procedures Architecture & Performance Victor Isakov MCT, CTT, MSCE, MCDBA

© Victor Victor Isakov Victor Isakov has been involved with SQL Server since Version 4.21a, enjoying a reputation as one of the best "SQL Gurus" in the country He has taught the Administration, Development, Data Warehousing and Performance Tuning courses since 1995 in the Australasia-Pacific region and provided consultancy services to companies like the University of NSW, Department of Education and Training, Westpac and the United Nations He specialises as a consultant in Optimising, Troubleshooting and Architecting Database Solutions based on Microsoft technology

© Victor Agenda Stored Procedure Architecture Stored Procedure Optimization Managing Query Plan Performance Undocumented Microsoft Stored Procedures

© Victor Agenda Stored Procedure Architecture Stored Procedure Optimization Managing the Query Plan for Performance. Undocumented Microsoft Stored Procedures

© Victor Stored Procedure Architecture SPs are SQL Server programs Traditionally written in Transact-SQL In Yukon will include VB.NET, C# Provide Query Management Data Modifications Data / Referential Integrity Maintenance Tasks Protect Data Provide Gateway to Database Objects

© Victor Stored Procedure Architecture Stored in Database Protected like other DB objects Logic moved to server Increased Developer Productivity Reusable Code More structured 3-Tier solutions Eliminate needless query plan construction Cached query plans can improve performance

© Victor Compiled Execution Plan Query Changes Stored Procedure Architecture SQL Server Security Unauthorized Stored Procedure SELECT logic Business Rules Constraints

© Victor Execution Plan in Detail Query Plan Re-Entrant, Read-Only Structure Shared Separate Parallel / Serial Plans Execution Context User Context Parameter Values, etc Not Shared Can be Re-Used

© Victor Agenda Stored Procedure Architecture Stored Procedure Optimization Managing the Query Plan for Performance. Undocumented Microsoft Stored Procedures

© Victor CREATE PROCEDURE… Stored Procedure Optimization Parse TSQL Syntax Resolve References Save in Database Resolve references Optimize Compile Cache (RAM) Execute

© Victor Compilation in Detail

© Victor RAM Cache Stored Procedure Optimization SQL Server 1st instance compiled and QP loaded 2nd instance shares loaded QP (new EC) 3rd instance shares first QP (& 1st EC) 1st instance finishes Data Pages

© Victor Compilation / Execution in Detail

© Victor Stored Procedures Optimization Ad-hoc Queries Compiled when first used EP potentially cached Subsequent references Query Optimizer compares query with Query-Templates Conservative Use cached plan if it’s recognized Auto-Parameterization Cache (RAM) Data Pages

© Victor Processing T-SQL in Detail

© Victor Stored Procedure Optimization SPs compiled on first use Query plan cached in RAM Subsequent calls generally use cached plan Recompiled if…. Cache (RAM) Data Pages

© Victor Recompilation WITH RECOMPILE in CREATE PROCEDURE or EXECUTE statement Schema changes to referenced objects Table referenced by stored procedure Adding / dropping constraints, defaults, or rules sysobjects.schema_ver Restoring the database or any objects procedure references

© Victor Recompilation Running sp_recompile on table SET Options May affect query result ANSI_DEFAULTSANSI_NULLSANSI_PADDINGANSI_WARNINGSCONCAT_NULL_YIELDS_NULL Certain operations on temporary tables

© Victor Recompilation Interleaving DDL and DML in procedure Server activity ages plan out of cache Table Type Empty Condition Threshold When Empty Threshold When Not Empty Permanent < 500 rows Changes >= 500 Changes >= (20% of Cardinality) Temporary < 6 rows Changes >= 6 Changes >= 500 Variable Change in Cardinality does not affect Automatic Recompilation

© Victor Profiler Trap SP:Recompile CodeReason 1 Schema, bindings, or permissions changed between compile or execute 2 Statistics changed 3 Object not found at compile time, deferred check to run time 4 Set option changed in batch 5 Temp table schema, binding, or permission changed 6 Remote rowset schema, binding, or permission changed

© Victor Avoiding Recompilation Use sp_executesql Preferred Method Calling Stored Procedure unaffected Use Sub-Procedures Only smaller sub-procedure recompiled Use KEEP PLAN Option SET Options

© Victor Agenda Stored Procedure Architecture Stored Procedures Optimization Managing Query Plan Performance Undocumented Microsoft Stored Procedures

© Victor Query Optimization All input parameters Whether used or not Suitable indexes Data statistics All logic in the procedure Whether or not the code is executed Query Complexity IF

© Victor Query Optimization in Detail

© Victor Managing Query Plan Performance Generated query plan based on parameters Provided by first query (arbitrary) Cached and reused for all subsequent use Regardless of suitability Some queries run normally, others do not

© Victor Managing the Cache Flushing the Cache DBCC FREEPROCCACHE DBCC DROPCLEANBUFFERS Tip: Should CHECKPOINT first to ensure all data is flushed, otherwise “dirty” data pages will remain in the cache. Restart SQL Server Power-cycle system

© Victor SQL Server Cache Least-Frequently-Used strategy Least used objects flushed when cache fills Unified cache stores data and procedure pages Lazywriter does know the difference Monitor cache with Perfmon

© Victor Inserting QP into Cache

© Victor Retrieving QP From Cache

© Victor Lazywriter Cache Processing Flow

© Victor Managing Query Plan Performance Recompilation might help performance When “optimized” query is not optimal Cost of recompile insignificant compared to poorly running query Test all parameter combinations Check for consistent plans, performance If query plan varies based on parameters Recompile for each execution? Best to redesign procedure

© Victor Managing Query Plan Performance Recompiling on Demand CREATE PROCEDURE … WITH RECOMPILE Compiles QP each time stored procedure is executed EXECUTE … WITH RECOMPILE When parameters are not “typical” sp_recompile Forces all plans to be recompiled (very cheap) Point to stored procedure, table… Statement-based recompile Dynamic string execution (dangerous, but powerful) Smaller, more-focused procedures

© Victor Managing Query Plan Performance Use Query Analyzer to view Query Plan Execute query with a range of input parameters Clear procedure and data cache View IO Statistics Enable “Show Execution Plan”

© Victor Managing Query Plan Performance

© Victor Managing Query Plan Performance

© Victor Managing Query Plan Performance Strategy Flush cache DBCC FREEPROCCACHE Force recompile WITH RECOMPILE Avoid “all-purpose” stored procedures SET Options

© Victor Managing Query Plan Performance Strategy Re-engineer stored procedures Break up larger, more complex procedures Each sub-procedure gets its own query plan Design procedures to work with “typical” parms Build special case procedures Use templates!

© Victor Performance Construct Command object Execute query Parse, resolve, build QP Execute query Return resultsets Client-side processing Don’t sweat the small stuff… Client-side preparation Client-side consumption Server-side execution

© Victor Agenda Stored Procedure Architecture. Stored Procedures Optimization Managing Query Plan Performance Undocumented Microsoft Stored Procedures

© Victor Undocumented Microsoft Stored Procedures Stored Procedure Architecture. Stored Procedures Optimization Managing Query Plan Performance Undocumented Microsoft Stored Procedures

© Victor Undocumented Microsoft Stored Procedures Seven for the DBA sp_EnumErrorLogssp_ReadErrorLogsp_TempDbSpacesp_EnumOleDbDatasourcessp_MSForEachDbsp_MSLoginMappingssp_MSTableSpace

© Victor sp_EnumErrorLogs

© Victor sp_ReadErrorLog

© Victor sp_TempDbSpace

© Victor sp_EnumOleDbDataSources

© Victor sp_MSForEachDB

© Victor sp_MSLoginMappings

© Victor sp_MSTableSpace

© Victor Undocumented Microsoft Stored Procedures Seven for the Developer sp_GetTypeStringsp_MSCheck_UId_Owns_Anythingsp_MSGet_Qualified_Namesp_MSForEachTablesp_MSIs_PK_Colsp_MSTable_Has_Unique_Indexsp_MSTableRefs

© Victor sp_GetTypeString

© Victor sp_MSCheck_UId_Owns_Anything

© Victor sp_MSGet_Qualified_Name

© Victor sp_MSForEachTable

© Victor sp_MSIs_PK_Col

© Victor sp_MSTable_Has_Unique_Index

© Victor sp_MSTableRefs

© Victor Summary Audience: “Who cares, let us go home! Please!” Victor: “But I need to put up something…” Understand how SQL Server works Helps you help SQL Server to execute “optimal” plans “KISS” Principle Slides are up on