1. Deep Dive into Adaptive Query Processing
PASS SQLSATURDAY
MANCHESTER 2018
Hugo Kornelis
Deep Dive into Adaptive Query Processing
@SQLSatMcr
@ SQLSatMcr

2. Saturday Agenda
Morning Break 09: :45 Break 10: :00 Lunch Sponsor sessions 12: :20 Lunch 12: :30
Afternoon
Break : :45
Break : :00
Prize giving 17:00
Times can be found in the event booklet / Guidebook
@SQLSatMcr

3. Updates & Amends Lighting Talks Current amends to the day
Paul Andrew - Cognitive Services Embedded in Azure Data Lake & U-SQL
Hugo Kornelis - Managing Execution Plans
Andrew Pruski - A real world implementation of SQL Server containers
Alex Yates - "Reflections on a career of writing code... bring low expectations and a good sense of humour. Earplugs optional."
Current amends to the day
4pm – 5pm : Theatre 4 : Steve Morgan (Microsoft) with "Dealing with SQL 2008 end of life & migration options to Azure"
@SQLSatMcr
@ SQLSatMcr

4. Lunch Sponsor Sessions
Overview of the IT Market & Emerging Recruitment Trends
Ahsan Iqbal & Alex Taylor - Robert Walters
in Theatre 1
Session 2
Simplifying SQL Server Data Protection - Protecting Microsoft SQL Server with Rubrik
Tim Hynes - Rubrik
in Theatre 1
Bricking It – A Lightning Tour of Azure Data Bricks
Paul Andrew – Adatis
in Theatre 2
@SQLSatMcr
@ SQLSatMcr

5. Gold Sponsors
@SQLSatMcr

6. Silver Sponsors
@SQLSatMcr

7. Bronze Sponsors
@SQLSatMcr

8. Thank you, Learn and Enjoy
If you have any queries, find a Helper (Green shirts) or Organiser (Yellow).
Please be aware of your own and other peoples opinions around you. We support an open policy, and would love everyone to get on, but realise that doesn't always happen. 
Please thank the Sponsors and visit them, we can not run it without their support. And the helpers, as they make your day happen.
Thanks SQL Saturday Manchester Team (Shaun, Paul, Martin & Ian)
@SQLSatMcr

9. Hugo Kornelis
I make SQL Server Fast
July 14, 2018
(c) Hugo Kornelis

10. Hugo Kornelis I make SQLServerFast.com
Blog:
Execution Plan Reference:
Resources (planned)
Deck and demo for this session will go there
Check twitter for exact link
Articles (planned)
July 14, 2018
(c) Hugo Kornelis

11. Hugo Kornelis I make SQLServerFast.com I do other community things
Technical editor for various books
SQL Server Execution Plans, 3rd edition (“Real Soon Now”™)
11 years MVP (2006 – 2016, SQL Server/Data Platform)
Lots of other things
July 14, 2018
(c) Hugo Kornelis

12. Hugo Kornelis I make SQLServerFast.com I do other community things
I work
Independent database consultant
Will do (almost) anything for money
July 14, 2018
(c) Hugo Kornelis

13. Agenda
Why?
What?
How?
Future
July 14, 2018
(c) Hugo Kornelis

14. Why? Steps in query processing Preprocessing Parse Syntax check
July 14, 2018
(c) Hugo Kornelis

15. Why? Steps in query processing Preprocessing Parse Bind
Verify object names
Find actual data types
July 14, 2018
(c) Hugo Kornelis

16. Why? Steps in query processing Preprocessing Parse Bind Optimize
Find “good enough” plan in “reasonable” time
Very cpu intensive
Result stored in plan cache, for future reuse
July 14, 2018
(c) Hugo Kornelis

17. Why? Steps in query processing Preprocessing Execution
Plan executed exactly as prescribed
No feedback loop if plan is unexpectedly slow
This means that a “bad” plan stays in cache, for reuse
July 14, 2018
(c) Hugo Kornelis

18. Why? Adaptive Query Processing
Introduced in Azure DB / SQL Server 2017
First (limited) attempt to add a feedback loop
Database compatibility level: 140
More planned for future releases
New functionality will probably be in Azure DB first
July 14, 2018
(c) Hugo Kornelis

19. What? Adaptive Query Processing in SQL 2017
Batch Mode Memory Grant Feedback
Interleaved Execution
Batch Mode Adaptive Joins
July 14, 2018
(c) Hugo Kornelis

20. Memory Grant Feedback What is a memory grant?
Memory allocated for query execution
E.g. for Sort or Hash Match operators
July 14, 2018
(c) Hugo Kornelis

21. Memory Grant Feedback What is a memory grant?
Memory allocated for query execution
Computed during optimization
Allocated when query execution starts
Fixed throughout entire execution
With some (very specific) exceptions
July 14, 2018
(c) Hugo Kornelis

22. Memory Grant Feedback What is a memory grant?
Affect of too low memory grant
Operators “spill” to tempdb
Slower algorithms
July 14, 2018
(c) Hugo Kornelis

23. Memory Grant Feedback What is a memory grant?
Affect of too low memory grant
Operators “spill” to tempdb
Warning signals:
Extended Events: hash_warning / sort_warning
Warnings in actual execution plan
July 14, 2018
(c) Hugo Kornelis

24. Memory Grant Feedback What is a memory grant?
Affect of too low memory grant
Affect of too high memory grant
Queries have to wait for memory
Warning signals
High RESOURCE_SEMAPHORE waits
Error 8645
sys.dm_exec_query_memory_grants
July 14, 2018
(c) Hugo Kornelis

25. Memory Grant Feedback How does it work? Batch mode only
First time query runs
Normal compilation + execution
Spills and actual memory usage recorded
July 14, 2018
(c) Hugo Kornelis

26. Memory Grant Feedback How does it work? Batch mode only
First time query runs
Later executions
Last execution used < 50%?
Adjust memory grant down (1 MB minimum)
Last execution spilled?
Adjust memory grant up
July 14, 2018
(c) Hugo Kornelis

27. Memory Grant Feedback How does it work? Batch mode only
First time query runs
Later executions
Queries with “random” memory usage pattern
After few consecutive adjustments: increase threshold
After several threshold increases: turn off completely
July 14, 2018
(c) Hugo Kornelis

28. Memory Grant Feedback How does it work? Batch mode only
First time query runs
Later executions
Queries with “random” memory usage pattern
Stored in plan cache
All learnings forgotten when plan is recompiled
July 14, 2018
(c) Hugo Kornelis

29. Memory Grant Feedback Scenarios
Memory grant based on estimates is wrong
Grant will be correct after 1 or 2 executions
Effect lost on recompile
July 14, 2018
(c) Hugo Kornelis

30. Memory Grant Feedback Scenarios
Memory grant based on estimates is wrong
Required memory evolves over time
Grant correction even before statistics are updated
Effect lost on recompile
July 14, 2018
(c) Hugo Kornelis

31. Memory Grant Feedback Scenarios
Memory grant based on estimates is wrong
Required memory evolves over time
Required memory different every time
Change based on last execution (usually) not helpful
Feature will increase thresholds and eventually stop
Effect lost on recompile
July 14, 2018
(c) Hugo Kornelis

32. Memory Grant Feedback
DEMO
July 14, 2018
(c) Hugo Kornelis

33. Interleaved Execution
What is the problem with multi-statement table valued user-defined functions (MSTVFs)?
Compilation needs an estimated rowcount
Full compilation before execution
Actual rowcount from MSTVF not known at compile time
July 14, 2018
(c) Hugo Kornelis

34. Interleaved Execution
What is the problem with multi-statement table valued user-defined functions (MSTVFs)?
Compilation needs an estimated rowcount
Full compilation before execution
Code units compiled separately
Estimated rowcount from MSTVF not used for outer query
July 14, 2018
(c) Hugo Kornelis

35. Interleaved Execution
What is the problem with multi-statement table valued user-defined functions (MSTVFs)?
Compilation needs an estimated rowcount
Full compilation before execution
Code units compiled separately
But we still need an estimated rowcount to compile!
SQL Server 2005 – 2012: Estimate of 1 row
SQL Server 2014 – 2016: Estimate of 100 rows
SQL Server 2017+: Interleaved execution
July 14, 2018
(c) Hugo Kornelis

36. Interleaved Execution
What is the problem with multi-statement table valued user-defined functions (MSTVFs)?
Compilation needs an estimated rowcount
Incorrect estimation results in bad (slow) plans
Biggest risk: Incorrect join type
Nested Loops: Ideal for one small and one cheap input
MSTVF with “100” rows // index seek repeated for each row
Merge Join or Hash Match: Ideal for bigger inputs
MSTVF with “mucho” rows // single index scan
July 14, 2018
(c) Hugo Kornelis

37. Interleaved Execution
What is the problem with multi-statement table valued user-defined functions (MSTVFs)?
Compilation needs an estimated rowcount
Incorrect estimation results in bad (slow) plans
Biggest risk: Incorrect join type
Other bad things can also happen
July 14, 2018
(c) Hugo Kornelis

38. Interleaved Execution
What is the problem with multi-statement table valued user-defined functions (MSTVFs)?
Compilation needs an estimated rowcount
Incorrect estimation results in bad (slow) plans
No easy fixes
Fixed parameters (usually after FROM or JOIN)
Execute MSTVF first, save result in temporary table
Have main query join to temporary table instead of MSTVF
July 14, 2018
(c) Hugo Kornelis

39. Interleaved Execution
What is the problem with multi-statement table valued user-defined functions (MSTVFs)?
Compilation needs an estimated rowcount
Incorrect estimation results in bad (slow) plans
No easy fixes
Fixed parameters (usually after FROM or JOIN)
Parameters from other tables (requires APPLY or subquery)
Rewrite as inline table-valued user-defined function
Requires rewriting as a single statement
Will also work when used after FROM / JOIN, with fixed parameters
July 14, 2018
(c) Hugo Kornelis

40. Interleaved Execution
How does it work?
Compiler recognizes interleaved execution candidates
Requires MSTVF with fixed parameters
Plan marked with ContainsInterleavedExecutionCandidates
July 14, 2018
(c) Hugo Kornelis

41. Interleaved Execution
How does it work?
Compiler recognizes interleaved execution candidates
Compilation halted
MSTVF is executed
Corresponding operator marked: IsInterleavedExecuted
July 14, 2018
(c) Hugo Kornelis

42. Interleaved Execution
How does it work?
Compiler recognizes interleaved execution candidates
Compilation halted
Compilation continues
Actual number of rows from MSTVF is now known
Rest of plan optimized for this rowcount
(When compiling for estimated plan, fall back to “100”)
July 14, 2018
(c) Hugo Kornelis

43. Interleaved Execution
How does it work?
Compiler recognizes interleaved execution candidates
Compilation halted
Compilation continues
Rest of plan executes
Plan gets stored in plan cache
Future executions reuse, without interleaved execution
Can still get bad plan if rowcount varies between executions
July 14, 2018
(c) Hugo Kornelis

44. Interleaved Execution
Scenarios
MSTVF with “fixed” parameters (after FROM or JOIN)
Automatically fixed, without need for temporary table
Can backfire when rowcount varies between executions
Similar to parameter sniffing
First execution: 27 rows: plan optimized for low rowcount
Nested loops join
Low memory grant
July 14, 2018
(c) Hugo Kornelis

45. Interleaved Execution
Scenarios
MSTVF with “fixed” parameters (after FROM or JOIN)
Automatically fixed, without need for temporary table
Can backfire when rowcount varies between executions
Similar to parameter sniffing
First execution: 27 rows: plan optimized for low rowcount
Next execution: 7,934,402 rows: query very slow
Millions of iterations in Nested Loops join
Hash Match / Sort spill to tempdb
July 14, 2018
(c) Hugo Kornelis

46. Interleaved Execution
Scenarios
MSTVF with “fixed” parameters (after FROM or JOIN)
Automatically fixed, without need for temporary table
Can backfire when rowcount varies between executions
Similar to parameter sniffing
First execution: 27 rows: plan optimized for low rowcount
Next execution: 7,934,402 rows: query very slow
Can be hard to troubleshoot
Just as every other parameter sniffing issue
July 14, 2018
(c) Hugo Kornelis

47. Interleaved Execution
Scenarios
MSTVF with “fixed” parameters (after FROM or JOIN)
Automatically fixed, without need for temporary table
Can backfire when rowcount varies between executions
MSTVF with variable parameters (APPLY, subquery)
Not a candidate for interleaved execution
Unfortunately, this is the more common use case of TVFs
July 14, 2018
(c) Hugo Kornelis

48. Interleaved Execution
DEMO
July 14, 2018
(c) Hugo Kornelis

49. Adaptive Joins What algorithm is optimal for a join? Merge Join
Combines two sets that are sorted on the join column(s)
Most cost effective join operator (if the relationship is one to many)
May require additional sort operators
If data not already in the required order
Cost of sorting often exceeds saving of “cheap” join algorithm
July 14, 2018
(c) Hugo Kornelis

50. Adaptive Joins What algorithm is optimal for a join? Merge Join
Nested Loops
For each row in top input, reprocesses bottom input
Efficient when
Top input is small
Bottom input is “cheap” (seeks and/or very small tables)
Performance scales linearly with top input
July 14, 2018
(c) Hugo Kornelis

51. Adaptive Joins What algorithm is optimal for a join? Merge Join
Nested Loops
Hash Match
Stores top input in memory
“Hash table” – optimized for rest of algorithm
Bottom input finds matches in memory
Prefers small top input
July 14, 2018
(c) Hugo Kornelis

52. Adaptive Joins What algorithm is optimal for a join? Merge Join
Nested Loops
Hash Match
Stores top input in memory
Relatively expensive overall
Large memory requirement
High CPU usage
July 14, 2018
(c) Hugo Kornelis

53. Adaptive Joins What algorithm is optimal for a join? Merge Join
Nested Loops
Hash Match
Stores top input in memory
Relatively expensive overall
Performance scales logarithmical with top input
July 14, 2018
(c) Hugo Kornelis

54. Adaptive Joins Nested Loops vs Hash Match July 14, 2018
Source:
July 14, 2018
(c) Hugo Kornelis

55. Adaptive Joins How do adaptive joins work? New execution plan operator
Only join operator with three inputs
However, only two of them are ever used!
New property: Adaptive Threshold Rows
July 14, 2018
(c) Hugo Kornelis

56. Adaptive Joins How do adaptive joins work? New execution plan operator
First phase: Store top input in memory
Exactly the same as a Hash Match
Actual row count is known when done
July 14, 2018
(c) Hugo Kornelis

57. Adaptive Joins How do adaptive joins work? New execution plan operator
First phase: Store top input in memory
Second phase depends on row count
When > threshold: continue as Hash Match
Using second input
Third input not used in this case
July 14, 2018
(c) Hugo Kornelis

58. Adaptive Joins How do adaptive joins work? New execution plan operator
First phase: Store top input in memory
Second phase depends on row count
When > threshold: continue as Hash Match
When < threshold: switch to Nested Loops
Second input not used in this case
Third input executed once per row in hash table
July 14, 2018
(c) Hugo Kornelis

59. Adaptive Joins How do adaptive joins work? New execution plan operator
First phase: Store top input in memory
Second phase depends on row count
Batch mode only
July 14, 2018
(c) Hugo Kornelis

60. Adaptive Joins Scenario Two plan options
With Nested Loops into an Index Seek
Very efficient if actual row count is low
Can spiral out of control if row count higher than estimate
July 14, 2018
(c) Hugo Kornelis

61. Adaptive Joins Scenario Two plan options
With Nested Loops into an Index Seek
With Hash Match into an Index Scan
Scales much better
Index Scan makes base cost quite high
July 14, 2018
(c) Hugo Kornelis

62. Adaptive Joins Scenario Two Three plan options
With Nested Loops into an Index Seek
With Hash Match into an Index Scan
With Adaptive Join
Slightly slower than Nested Loops for low rowcount
Same performance as Hash Match for high rowcount
July 14, 2018
(c) Hugo Kornelis

63. Adaptive Joins Limitations Only in batch mode
Same limitations as Nested Loops join
Limited logical join types
Inner Join
Left Outer Join
Left Semi Join
Left Anti Semi Join
July 14, 2018
(c) Hugo Kornelis

64. Adaptive Joins Limitations Only in batch mode
Same limitations as Nested Loops join
Same limitations as Hash Match join
Requires equality predicate
Semi-blocking
High memory requirement
July 14, 2018
(c) Hugo Kornelis

65. Adaptive Joins
DEMO
July 14, 2018
(c) Hugo Kornelis

66. Future July 14, 2018 (c) Hugo Kornelis
Source:
July 14, 2018
(c) Hugo Kornelis

67. Conclusion Memory Grant Feedback Interleaved Execution Adaptive Join
Future work
July 14, 2018
(c) Hugo Kornelis

68. Twitter: @Hugo_Kornelis
T H E E N D
Questions?
July 14, 2018
(c) Hugo Kornelis