1. Database Engine Enhancements in SQL Server 2017
Joe Sack, Principal Program Manager, Microsoft

2. Microsoft Data Platform Landscape
DATA MANAGEMENT
DATA INSIGHTS
Operational data
ON-PREM CLOUD
Business intelligence
Power BI
Azure SQL Database
Azure Document DB
SQL Server
SQL Server Reporting Services
Data warehousing
Advanced Analytics & AI
Azure SQL Data Warehouse
SQL Server
SQL Server Analysis Services, R Services
Azure Machine Learning
SQL Server is our mission critical OLTP database with industry-leading TPC-E performance benchmarks. We’ve been recognized by Gartner as a leader in Operational Database Management four years in a row.
Azure SQL Database is the cloud answer to managing your operational data - built on SQL Server, so your existing applications and skills transfer. And as a managed database service, so it can save you time and money in both set-up and administration.
DocumentDB is our database-as-a-service NoSQL database service designed for modern mobile and web applications.
SQL Server continues to be viable for data warehousing. Today we have companies doing petabyte scale data warehousing in a scale up SMP architecture just with SQL Server.
Azure SQL Data Warehouse is the industry’s first enterprise-class cloud data warehouse that can grow, shrink, and pause in seconds.
Azure Data Lake - Offers data management and analytics at scale to customers, as Azure Data Lake. It is composed of three parts: Azure Data Lake Store, Azure Data Lake Analytics, and HD Insight. Together with these products, you can store big data of any size and type, and analyze it with familiar open-source tools – all while getting a leading, enterprise-class SLA.
Big data processing
Azure
Stream Analytics
Apache Hadoop
Azure HDInsight
Azure Data Lake
Azure Cognitive Services

3. Gartner Magic Quadrant for Operational Database Management Systems
SQL Server 2016
Gartner Magic Quadrant for Operational Database Management Systems
Everything Built-in
An industry leader four years in a row
Most secure database over the last 7 years
Highest performing data warehouse
In-database Advanced Analytics
The above graphics were published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document. The Gartner document is available upon request from Microsoft. Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner's research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose
This graphic was published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document. The Gartner document is available upon request. Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner's research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

4. Security vulnerabilities reported by NIST
SQL Server 2016
Security vulnerabilities reported by NIST
Everything Built-in
An industry leader four years in a row
Most secure database over the last 7 years
Highest performing data warehouse
In-database Advanced Analytics
National Institute of Standards and Technology Comprehensive Vulnerability Database update 2/2016.

5. #1 in 30TB, 10TB, 1TB TPC-H non-clustered results
SQL Server 2016
#1 in 30TB, 10TB, 1TB TPC-H non-clustered results
Everything Built-in
An industry leader four years in a row
Most secure database over the last 7 years
Highest performing data warehouse
In-database Advanced Analytics #1
We now have the #1, #2 and #3 TPC-H benchmarks in the world.
as of 4/17/2017

6. R + in-memory at massive scale
SQL Server 2016
R + in-memory at massive scale
Everything Built-in
An industry leader four years in a row
Most secure database over the last 7 years
Highest performing data warehouse
In-database Advanced Analytics
significantly enhanced our BI and reporting service capabilities.
publish modern reports to an iPhone, Android phone all at a fraction of the cost of competitive solutions
1 Million
PREDICTIONS/SEC

7. SQL Server 2017
Run anywhere and build apps using the language of your choice
Analyze complex relationships with support for graph objects & queries
Advanced Machine Learning with R & Python
Unparalleled performance with adaptive query processing
SQL Server 2017
Run anywhere and build apps using the language of your choice
Analyze complex relationships with support for graph data
Advanced Machine Learning with R & Python
Unparalleled performance with adaptive query processing
Industry-leading, most secure data platform with built-in intelligence for all your data

8. SQL Server 2017 Themes Adaptability Choice Mindshare
Adapt based on customer workload characteristics
Adaptability
Provide customers with a choice
Choice
Leverage the strength of strong technical communities
Mindshare
SQL Server 2017 Themes

9. Adaptability
In SQL server 2017

10. The middle-of-the-night call
You’re on call for supporting the data tier of a mission-critical SQL Server instance
There has been a jump in CPU utilization on a key server, and one of the critical stored procedure calls is now running (much) more slowly then it used to?
You’ve been asked to mitigate the issue and then determine the root cause

11. What does Query Store show?

12. What does Query Store show?
Good
Regression

13. SQL Server 2017: Automatic tuning
We can now detect and correct these scenarios without manual intervention
Recommended actions surfaced via sys.dm_db_tuning_recommendations
We can now automatically switch to the last known good plan whenever the regression is detected

14. Resumable index operations
Adapt to outages without losing maintenance work
Resume an index rebuild operation after an unexpected failure
Pause and resume an index build operation at any time, for example, to temporarily free up systems resources in order to execute a high priority task
Supports rebuilding large indexes online without requiring significant log space, allowing log truncation while rebuild operation is running

15. Adaptive Query Processing Feature Family

16. Interleaved Execution
Pre 2017
Optimize
Execute
Problem: Multi-statement table valued functions (MSTVFs) are treated as a black box by QP and we use a fixed optimization guess
Interleaved Execution will materialize row counts for multi-statement table valued functions (MSTVFs)
Downstream operations will benefit from the corrected MSTVF cardinality estimate
100 rows guessed for
MSTVFs
Performance issues if skewed!
2017+
MSTVF
identified
Execute
MSTVF
500k rows
assumed
Good
Performance!

17. Batch Mode Memory Grant Feedback
Problem: Queries may spill to disk or take too much memory based on poor cardinality estimates
MGF will adjust memory grants based on execution feedback
MGF will remove spills and improve concurrency for repeating queries

18. Batch Mode Memory Grant Feedback
Before
Spill
Spill
Spill detected
and feedback generated
After

19. Batch Mode Adaptive Joins
Problem: If cardinality estimates are skewed, we may choose an inappropriate join algorithm
AJ will defer the choice of hash join or nested loop until after the first join input has been scanned
AJ uses nested loop for small inputs, hash joins for large inputs

20. Choice
In SQL server 2017

21. Businesses are embracing choice
Heterogenous environments
Multiple data types
Different development languages
On-premises, cloud, and hybrid environments
{ }
T-SQL
Java
C/C++
C#/VB.NET
PHP
Node.js
Python
Ruby
The business landscape is becoming increasingly diverse
Development and deployment environments include Windows, Linux, macOS, and Docker
Data is no longer relational, with companies accessing diverse data, including video, streaming, documents, relational, both external data and data internal to their org
Languages and frameworks are also expanding with the poularity of Node.js, Python, Ruby and others
Companies deploy on-premises, in the cloud, or both, with a hybrid solution

22. SQL Server on the platform of your choice

23. Licensing Same license, new choice
Buying a SQL Server license—per- server or per-core—grants the option to use it on Windows Server or Linux
Previews are free to download and use in a non-production capacity
Same set of editions on Linux: Developer, Express, Standard, Web, Enterprise
LICENSE

24. What operational features are available on Linux?
Support for RHEL, Ubuntu, Docker
Package based installs, Docker image
Support for Open Shift, Docker Swarm
Backup/restore
SSMS on Windows connected to Linux
Command line tools: sqlcmd, bcp, sqlpackage
SQL Server Agent

25. What operational features are available on Linux?
Failover clustering through Pacemaker
Availability groups through Pacemaker
Replication
Log shipping
Transparent data encryption
SCOM management pack
DMVs
Full-Text Search

26. What programmability features are available on Linux?
All major language driver compatibility
In memory OLTP and Columnstore
Compression
Always Encrypted, Row Level Security, and data masking
AD user authentication (planned)
Service Broker
Change data capture
Partitioning
Auditing
CLR
JSON, XML
Third party tools
…and more

27. Enterprise building a mission critical app
Scenario
All Linux infrastructure
Application-level protection
Automatic and within seconds failover during unplanned outages
No downtime during planned maintenance
Performance sensitive app
DR required for compliance regulations
Solution
HADR with Always On Availability Groups on Linux or Windows
Backups
Reports HA DR
Async Log Synchronization
Sync Log Synchronization

28. Read scale-out Scenario Solution SaaS app (website)
Catalog database with high volume of concurrent read-only transactions
Bottlenecks on primary due to read workloads
Increased response time
Solution
Read scale with availability groups
No cluster required
Both Linux and Windows

29. Distributed Availability Group
Migration/testing
Scenarios
ISV solution built on SQL Server on Windows
Linux certification
Enterprise moving to an all-Linux infrastructure
Rigorous business requirements
Seamless migration
Solution
Minimum downtime and HA for cross-platform migrations with Distributed Availability Groups
Distributed Availability Group
AG2
AG1
Migration/Testing

30. Mindshare
In SQL server 2017

31. T-SQL Improvements STRING_AGG string concatenation aggregate
CONCAT_WS, TRANSLATE, TRIM string functions
New Japanese Collation Families
New BULK Insert options and CSV Support options (including Azure Storage and RFC CSV file support)

32. In-Memory OLTP
New T-SQL enhancements for In-Memory OLTP Tables and Natively Compiled Stored Procedures
JSON Support
CROSS Apply
Computed Columns Support
Removed index limit
CASE support
Sp_rename/sp_spaceused support
TOP (N) WITH TIES
Improved Performance on ALTER TABLE, failover operations
Azure VM Support for In-Memory Checkpoint Files

33. Columnstore Clustered Columnstore LOB Support
LOBs are compressed, reducing DB size
Queries run over the compressed format, improving performance
Reduced backup times
Online Non-Clustered Columnstore Index Build/Rebuild
Create and Rebuild Indexes without long blocking

34. Why is my query slow? Locks Latches Buffer Latches Buffer IO
Compilation
Transaction Log IO
Network IO
Parallelism
Memory
User Waits
Tracing
Other Disk IO
Why is my query slow?

35. Query Store and Wait Statistics

36. SQL Server Python SQL Server 2016 introduced SQL Server R Services
SQL Server 2017 now introduces SQL Server Python Services
Two options to execute inside SQL Server:
Start from Python and execute remote jobs
Start from T-SQL with an embedded Python script
Secure and fully governed feature
Supports SQL and integrated authentication
Manage resources (CPU & memory etc.)
Source:

37. Graph Database Scenarios
Represent hierarchical or interconnected data with complex relationships
Accommodate many-to-many relationships
Allow for evolving requirements - organically growing connections as the business evolves
Easily analyze interconnected data, materialize new information from existing facts and identify non-obvious connections

38. Graph Database Terminology
Nodes: Entities – for example, stores, people, products, businesses
Edges: Relationships between nodes, lines that connect nodes to other nodes
Properties or Attributes: information associated with specific nodes and edges

39. SQL Server 2017 Support for graph data
Native nodes and edge table support
Query language extension provides multi-hop navigation using join-free pattern matching
Query across regular SQL tables and graph data
Interoperability – for example, support for Columnstore indexes

40. Nodes and Edges

41. MATCH clause

42. What’s next after SQL Server 2017?
You can expect that we aren’t returning to a slower release cycle
Faster Cadence
Our choices will be very influenced by customer drivers both in SQL Server and Azure SQL Database
Customer driven
Azure SQL DB service best practices will continue to flow into SQL Server – what we fix in the service, we’ll want to port to SQL Server as well
Cloud Service Influenced
While not every complexity can be addressed, we will look to minimize and mitigate through adaptive mechanisms (feedback loops, sampling, adaptive switches based on runtime conditions)
Adaptation
What’s next after SQL Server 2017?
Microsoft: Engineering the Future by Matthew Johnson up next