1. Deep dive on four SQL Server 2017 Features
Joe Sack, Principal Program Manager, Microsoft

2. THANKS to all Sponsors! EVENT SPONSORS EXPO SPONSORS
EXPO LIGHT SPONSORS

3. Agenda Wait statistics in Query Store Automated Tuning
Adaptive Query Processing
SQL Graph
Q&A

4. Wait statistics in Query Store
In SQL Server 2017 and Azure SQL Database

5. Query execution and wait statistics
Query Execution Started
Query Execution
WAIT: RESOURCE SEMAPHORE – MEMORY
WAIT: PAGEIOLATCH_SH – Buffer IO
Query Execution Finished

6. Why is my query slow? Locks Latches Buffer Latches Buffer IO
Compilation
Transaction Log IO
Network IO
Parallelism
Memory
User Waits
Tracing
Other Disk IO
Why is my query slow?

7. Demo: Wait statistics in Query Store

8. Automatic Tuning
In SQL Server 2017 and Azure SQL Database

9. SQL Server 2017: Automatic tuning
We can now detect and correct these scenarios without manual intervention
Recommended actions surfaced via sys.dm_db_tuning_recommendations
We can now automatically switch to the last known good plan whenever the regression is detected

10. Demo: Automated Tuning

11. Adaptive Query Processing
In SQL Server 2017 and Azure SQL Database

12. Example “problem” query

13. MSTVF definition

14. Function Types Scalar Function Multi-statement table-valued function
Series of T-SQL statements that evaluate to a scalar value
Multi-statement table-valued function
Series of T-SQL statements that populate a TABLE return variable
Inline table-valued function
TABLE return value is defined via a single SELECT statement
Function Types

15. MSTVF execution plan example

16. Query Processing and Cardinality Estimation
When estimates are accurate (enough), we make informed decisions around order of operations and physical algorithm selection
CE uses a combination of statistical techniques and assumptions
During optimization, the cardinality estimation (CE) process is responsible for estimating the number of rows processed at each step in an execution plan

17. Common reasons for incorrect estimates
Missing statistics
Stale statistics
Inadequate statistics sample rate
Bad parameter sniffing scenarios
Out-of-model query constructs
E.g. MSTVFs, table variables, XQuery
Assumptions not aligned with data being queried
E.g. independence vs. correlation

18. Cost of incorrect estimates
Slow query response time due to inefficient plans
Excessive resource utilization (CPU, Memory, IO)
Spills to disk
Reduced throughput and concurrency
T-SQL refactoring to work around off-model statements

19. Introducing the new Adaptive Query Processing feature family
Interleaved execution
Problem: Prior to this feature, MSTVFs are treated as a black box by QP
Interleaved Execution will materialize estimates for multi-statement table valued functions (MSTVFs)
Downstream operations will benefit from the corrected MSTVF cardinality estimate

20. Introducing the new Adaptive Query Processing feature family
Batch-mode memory grant feedback
Problem: Queries may spill to disk or take too much memory based on poor cardinality estimates
MGF will adjust memory grants based on execution feedback
MGF will remove spills and improve concurrency for repeating queries

21. Introducing the new Adaptive Query Processing feature family
Batch-mode adaptive joins
Problem: If cardinality estimates are skewed, we may choose an inappropriate join algorithm
AJ will defer the choice of hash join or nested loop until after the first join input has been scanned
AJ uses nested loop for small inputs, hash joins for large inputs

22. Demo: Adaptive query processing feature family

23. About Interleaved Execution
140
How to enable?
Benefits workloads with skews and downstream operations
Expected performance improvements?

24. About Interleaved Execution
Minimal, since we’re already materializing MSTVFs
Expected overhead?
First execution cached will be used by consecutive executions
Cached plan considerations
Contains Interleaved Execution Candidates
Is Interleaved Executed
Plan attributes
Execution status, CE update, disabled reason
Xevents

25. About Batch Mode Memory Grant Feedback
140
How to enable?
Benefits workloads with spills or overages
Expected performance improvements?
Before
After

26. About Batch Mode Memory Grant Feedback
If there is oscillation, we will disable the loop
Expected overhead?
First execution cached will be used by consecutive executions
Plan attributes
Spill report, and updates by feedback
Xevents
For spills – spill size plus a buffer.
For overages – waste less the buffer.
Expected decrease and increase size?
Memory grant size will go back to original.
RECOMPILE or eviction scenarios

27. About Batch Mode Adaptive Join
How to enable?
140
Eligible statements
The join is eligible to be executed both by an indexed nested loop join or a hash join physical algorithm
The hash join uses batch mode – either through the presence of a Columnstore index in the query overall or a Columnstore indexed table being referenced directly by the join
The generated alternative solutions of the nested loop join and hash join should have the same first child (outer reference)

28. Adaptive Join Threshold

29. About Batch Mode Adaptive Join
Expected performance benefit?
Performance gains occur for workloads where, prior to adaptive joins being available, the optimizer chooses the wrong join type due to cardinality misestimates

30. About Batch Mode Adaptive Join
We grant memory even for NL scenario, so if NL is * always * optimal, you have more overhead
Expected overhead?
Adaptive Threshold Rows, Estimated and Actual Join Type
Plan attributes
Adaptive join skipped
Xevents
Single compiled plan can accommodate low and high row scenarios
Cached plan considerations

31. SQL Graph
In SQL Server 2017 and Azure SQL Database

32. Database engine enhancements Support for graph data
Professional networking app
Full create, read, update and delete support for nodes and edges
Query language extension provides multi-hop navigation using join-free pattern matching
SQL engine integration enables querying across SQL tables and graph data
Existing tools work out of the box with graph data
Bachelor of Science, Finance
Statistics
Degree earned
Skill
Andy Smith
Coworker
Position
Skill
Mary Jones
Business analyst
Former employer
Contoso
Employer
Employer
AdventureWorks
Position
Role in company
Role in company
Program manager
Role in company

33. Recommendation Engine
Scenario examples
Product sales data
If a person P bought product A, find friends P who bought A and also bought other products. Make recommendation based on that.
Social Network
Yelp like application, find me friends who like the same restaurant that I like and also recommend other restaurants.
Friends
location
hasBought
isLooking
sellsAt
Matured and trusted product
Columnstore, Advanced Analytics, ML, etc.
Both Relational and Graph, no ETL required
Will be available on Azure and on-prem
Security and Compliance
Product

34. Recommending Songs: Approach
User1
Song1
User2
Song2
User3
Song3
User4
Song4
User5
Song5

35. The Million Song Dataset
Data courtesy of the Million Song Dataset ( and the Echo Nest Taste Profile Subset (
Thierry Bertin-Mahieux, Daniel P.W. Ellis, Brian Whitman, and Paul Lamere. The Million Song Dataset. In Proceedings of the 12th International Society for Music Information Retrieval Conference (ISMIR 2011), 2011.
The Echo Nest Taste profile subset, the official user data collection for the Million Song Dataset, available at:

36. Implementation using SQL Graph
UniqueUser
(node table)
UniqueSong
(node table)
Likes
(edge table)
CREATE TABLE UniqueUser (UserId VARCHAR(80))
AS NODE
CREATE TABLE Likes (ListenCount BIGINT))
AS EDGE
CREATE TABLE UniqueSong (SongId VARCHAR(50)
,SongTitle VARCHAR(500)
,ArtistName VARCHAR(500))
AS NODE

37. Demo: SQL Graph
Special thanks to Arvind Shyamsundar from the SQL CAT team for putting together this demo scenario

38. SQL Graph and the Database Engine
Queries can lookup against existing SQL Server database tables and graph nodes/edges
Columnstore, Advanced Analytics, HA, etc. – all supported
Existing tools will all work out of the box, for example backup and restore, import and export, etc.

39. Thank you!

40. Please evaluate all sessions! QR / LINK on posters and in program