SQL Server 2017 has more cool features than just running on Linux Marcin Szeliga Damian Widera SQL Server 2017 has more cool features than just running on Linux

Sponsors

Session will begin very soon :) Please complete the evaluation form from your pocket after the session. Your feedback will help us to improve future conferences and speakers will appreciate your feedback! Enjoy the conference!

Marcin Szeliga Data philosopher 20 years of experience with SQL Server Data Platform MVP & MCT Microsoft Certified Solutions Expert Data Management and Analytics Cloud Platform and Infrastructure Business Intelligence Microsoft Certified Solutions Developer Azure Solution Architect

Damian Widera 15+ years of experience with SQL Server Data Platform MVP & MCT Microsoft Certified Solutions Expert Data Management and Analytics Business Intelligence http://sqlplayer.net/

Agenda Automated Tuning and wait statistics in Query Store Adaptive Query Processing Not only for developers… Credits to Joe Sack, Principal Program Manager, Microsoft

Automated Tuning and wait statistics in Query Store

Query execution and wait statistics Query Execution Started Query Execution WAIT: RESOURCE SEMAPHORE – MEMORY WAIT: PAGEIOLATCH_SH – Buffer IO Query Execution Finished

Why is my query slow? Locks Latches Buffer Latches Buffer IO Compilation Transaction Log IO Network IO Parallelism Memory User Waits Tracing Other Disk IO

Demo: Wait statistics in Query Store

Automatic tuning Server can now detect and correct these scenarios without manual intervention Recommended actions surfaced via sys.dm_db_tuning_recommendations We can now automatically switch to the last known good plan whenever the regression is detected

Demo: Automated Tuning

Adaptive Query Processing

Function Types Scalar Function Multi-statement table-valued function Series of T-SQL statements that evaluate to a scalar value Multi-statement table-valued function Series of T-SQL statements that populate a TABLE return variable Inline table-valued function TABLE return value is defined via a single SELECT statement

MSTVF execution plan example

Example “problem” query

MSTVF definition

Query Processing and Cardinality Estimation When estimates are accurate (enough), we make informed decisions around order of operations and physical algorithm selection CE uses a combination of statistical techniques and assumptions During optimization, the cardinality estimation (CE) process is responsible for estimating the number of rows processed at each step in an execution plan

Common reasons for incorrect estimates Missing statistics Stale statistics Inadequate statistics sample rate Bad parameter sniffing scenarios Out-of-model query constructs E.g. MSTVFs, table variables, XQuery Assumptions not aligned with data being queried E.g. independence vs. correlation

Cost of incorrect estimates Slow query response time due to inefficient plans Excessive resource utilization (CPU, Memory, IO) Spills to disk Reduced throughput and concurrency T-SQL refactoring to work around off-model statements

Introducing the new Adaptive Query Processing feature family Interleaved execution Problem: Prior to this feature, MSTVFs are treated as a black box by QP Interleaved Execution will materialize estimates for multi-statement table valued functions (MSTVFs) Downstream operations will benefit from the corrected MSTVF cardinality estimate

Demo: Interleaved execution for MSTVFs

About Interleaved Execution 140 How to enable? Benefits workloads with skews and downstream operations Expected performance improvements?

About Interleaved Execution Minimal, since we’re already materializing MSTVFs Expected overhead? First execution cached will be used by consecutive executions Cached plan considerations Contains Interleaved Execution Candidates Is Interleaved Executed Plan attributes Execution status, CE update, disabled reason Xevents

Introducing the new Adaptive Query Processing feature family Batch-mode memory grant feedback Problem: Queries may spill to disk or take too much memory based on poor cardinality estimates MGF will adjust memory grants based on execution feedback MGF will remove spills and improve concurrency for repeating queries

Demo: Batch-mode memory grant feedback

About Batch Mode Memory Grant Feedback 140 How to enable? Benefits workloads with spills or overages Expected performance improvements? Before After

About Batch Mode Memory Grant Feedback If there is oscillation, we will disable the loop Expected overhead? First execution cached will be used by consecutive executions Plan attributes Spill report, and updates by feedback Xevents For spills – spill size plus a buffer. For overages – waste less the buffer. Expected decrease and increase size? Memory grant size will go back to original. RECOMPILE or eviction scenarios

Introducing the new Adaptive Query Processing feature family Batch-mode adaptive joins Problem: If cardinality estimates are skewed, we may choose an inappropriate join algorithm AJ will defer the choice of hash join or nested loop until after the first join input has been scanned AJ uses nested loop for small inputs, hash joins for large inputs

Demo: Batch-mode adaptive joins

About Batch Mode Adaptive Join How to enable? 140 Compatibility Level Eligible statements The join is eligible to be executed both by an indexed nested loop join or a hash join physical algorithm The hash join uses batch mode – either through the presence of a Columnstore index in the query overall or a Columnstore indexed table being referenced directly by the join The generated alternative solutions of the nested loop join and hash join should have the same first child (outer reference)

Adaptive Join Threshold

About Batch Mode Adaptive Join Expected performance benefit? Performance gains occur for workloads where, prior to adaptive joins being available, the optimizer chooses the wrong join type due to cardinality misestimates

About Batch Mode Adaptive Join We grant memory even for NL scenario, so if NL is * always * optimal, you have more overhead Expected overhead? Adaptive Threshold Rows, Estimated and Actual Join Type Plan attributes Adaptive join skipped Xevents Single compiled plan can accommodate low and high row scenarios Cached plan considerations

Not only for developers….

Not only for developers… CLR security Resumable online index rebuild New DMV’s TSQL Improvements

CLR

Clr security – breaking change A CLR assembly created with PERMISSION_SET = SAFE may be able to access external system resources, call unmanaged code, and acquire sysadmin privileges. Beginning with SQL Server 2017, an sp_configure option called clr strict security is introduced to enhance the security of CLR assemblies. clr strict security is enabled by default, and treats SAFE and EXTERNAL_ACCESS assemblies as if they were marked UNSAFE. After enabling strict security, any assemblies that are not signed will fail to load. You must either alter or drop and recreate each assembly so that it is signed with a certificate or asymmetric key that has a corresponding login with the UNSAFE ASSEMBLY permission on the server. 38 | 11/13/201811/13/2018 | Footer Goes Here

Clr security – breaking change In SQL Server 2017, Microsoft now by default requires that all type of assemblies (SAFE, EXTERNAL_ACCESS, UNSAFE) are authorized for UNSAFE access, by: The database is set to be TRUSTWORTHY, OR The assembly is signed with a certificate that has a corresponding login with UNSAFE ASSEMBLY permission, OR The assembly is signed with an asymmetric key that has a corresponding login with UNSAFE ASSEMBLY permission.

Clr security – breaking change the assembly is not signed, and the database where you want to deploy it to is not TRUSTWORTHY. sys.sp_add_trusted_assembly sys.trusted_assemblies sys.sp_drop_trusted_assembly 40 | 11/13/201811/13/2018 | Footer Goes Here

Resumable online index rebuild Need to cancel the index maintenance operation running out of time in the maintenance period, blocking issues, low disk space due to the transaction log filling up. Resumable Online Index Rebuilds provide the option to pause an index rebuild and provides an option of executing, suspending, resuming or aborting an online index operation. 41 | 11/13/201811/13/2018 | Footer Goes Here

Demo: Resumable index rebuild

Demo: New DMV’s

Demo: T-SQL Improvements