Making Developers lives easier with SQL Server 2016 SAMIR BEHARA

About Me I am Samir Behara. Currently working as a Senior Developer in EBSCO Industries, Birmingham. I have more than ten years of experience in IT industry, working on large-scale enterprise applications involving complex business functions, web integration, and data management in various domains like Insurance, Manufacturing and Publishing. Twitter –  @samirbehara LinkedIn - https://www.linkedin.com/in/samir-behara-78953870 Website – www.dotnetvibes.com Steel City SQL User Group, Birmingham, AL - https://www.meetup.com/SteelCitySQL/

What we will try to cover in this session? Performance Productivity Security Query Store Live Query Statistics Comparing Execution Plans JSON Support Temporal Tables Stretch Database TSQL Enhancements Always Encrypted Dynamic Data Masking

Current State of a T-SQL Developer

SQL Server Management Studio 2016

SSMS 2016 is FREE. SSMS is an independent web installer. SSMS is decoupled from the underlying SQL Server Engine. SSMS now uses Visual Studio 2015 isolated shell. More frequent releases for new features, enhancement and bug fixes.

Live Query Statistics

Have you come across this scenario? You execute a TSQL Query and it just runs and runs, without displaying any results and you finally end up cancelling the executing query. How do you troubleshoot this problem?

Collect actual metrics of the query while it is running…

Benefits of Live Query Statistics Troubleshoot Long running queries Real Time Insights into SQL Query Execution Process Visually track overall query progress

How to enable Live Query Statistics? Using SQL Server Management Studio Using Activity Monitor

Live Query Statistics in action...

Query Store

Have you come across this scenario? There is a temporary performance issue with your application and you realize that your queries are running slow all of a sudden. TSQL queries slow down after a server/application upgrade. You are trying to troubleshoot a performance issue but don’t have any historical information on how the query ran over a period of time. Do you have baseline performance data for your application?

What is Query Store? A new feature in SQL Server 2016 which collects and presents detailed historical information of queries, execution plans and run-time statistics. Gives the SQL developers/DBAs lot more control and insight into the query performance issues. Directly integrated in SQL Server, so gives more power to developers/DBAs to have an access to run-time query statistics. No need of any 3rd party performance monitoring tool. Simple to use – Graphical user interface built into SSMS.

What does Query Store do? Store history of execution plans and statistics of each query Baseline performance metrics and track deviations Displays customizable UI reports Diagnosing Performance Issues after upgrades

How does Query Store work? Compile Message Execute Message Compile Execute Plan Store Runtime Stats Query Store

DMVs vs Query Store Information stored in DMVs clear out whenever SQL Server gets restarted or the plan cache is cleared. Query Store stores all information on disk, and as a result the data persists across server restarts, upgrades and recompiles.

What are the various built in Query Store reports? Types of Reports Description Regressed Queries Shows all the queries whose Execution Plan worsened over a period of time. Overall Resource Consumption  Shows summary of the query runtime statistics executed during a time interval. Top Resource Consuming Queries Shows most expensive queries executed during a time interval, based on Duration, CPU Time, Logical/Physical Reads, Memory consumption. Tracked Queries Shows the historic runtime statistics of a specific query.

How to enable and configure Query Store? Query Store is a Database level configuration and by default it is turned OFF.  Within your SQL Server 2016 database, you will now see a new folder named ‘Query Store’. 

Query Store Configurations Operation Mode  Data Flush Interval  Statistics Collection Interval Max Size Query Store Capture Mode  Size Based Cleanup Mode Stale Query Threshold

Quick look at the new Query Store related DMVs DMV Names Description sys.database_query_store_options Contains information on the Query Store options for the database. sys.query_store_plan Contains information about the execution plans for a query. sys.query_store_query Contains information about the query and its aggregated runtime statistics. sys.query_store_query_text  Contains the Transact-SQL text. sys.query_store_runtime_stat Contains information about the query runtime statistics. sys.query_store_runtime_stats_interval  Contains information about the start and end time of each interval over which statistical information for a query was collected.

Query Store in action…

Any Performance Impact by enabling Query Store? Ideally no performance degradation. However need to be aware of these configurations -- Data Flush Interval –  By default it is set as 15 min. Reducing this interval further would indicate writing to the disk faster, and might have an impact on performance.  Max Size – You can allocate a max size to the Query Store within the application database. Once you go over that limit, you can use Query Store as ‘Read Only’ and not ‘Read Write’ mode.

Comparing Execution Plans

When do you need to compare Execution Plans? A query runs fine in DEV/QA environment but takes more time to execute in PROD. Query Performance has slowed down after doing a server/database upgrade. You have fine tuned a TSQL query ( be it by rewriting the query with apt table joins or by making index changes) and want to explain the performance gain.

How to compare Execution Plans in SSMS? SQL Server 2016 introduces a ‘Compare Showplan‘ feature which allows side-by-side comparison of two execution plans & will be helpful when working in troubleshooting performance issues. Part of the plan that are same in both the plans, are highlighted in the same color (in this case purple)

Dual Properties Window Choose the specific Operator on the Dual Properties Window to view -- Actual Number of rows Estimated Operator Cost Number of rows read

How to compare Global Properties? Choose the SELECT Operator on the Dual Properties Window to view -- Compile Time Estimated Subtree Cost Missing Indexes

Comparing Execution Plans in action…

Built-in JSON Support

Why JSON? Javascript Object Notation is a lightweight data interchange format which helps to communicate between client and server side technologies. JSON is considered as the best tool for sharing data, because the data is stored in an array format. This makes data transfer easier since the array structure is pretty much familiar to object oriented languages.

What are the advantages of JSON over XML? JSON is lightweight in comparison to XML and has a smaller message size. In JSON, data is stored in arrays whereas in XML data is stored in trees, hence XML needs to be first transformed before it can be imported. JSON parsing is generally faster than XML parsing. Because of the similarity in syntax, JSON is easier to be handled with Javascript. Formatted JSON is generally easier to read than formatted XML. Due to its simplicity, JSON runs faster and consumes lesser memory.

How to format query results as JSON? SQL Server 2016 provides built-in support for storing, managing and parsing JSON data. There is no separate JSON data type created, like XML – rather JSON is represented by NVARCHAR datatype. FOR JSON clause allows us to format query results as JSON text. Appending this syntax to a standard TSQL query returns the result set in JSON format. Types of FOR JSON clause Description FOR JSON AUTO JSON is formatted by the database engine based on the order of the columns in the SELECT statement. FOR JSON PATH JSON is formatted based on the user’s discretion. It gives us full control over the format of the JSON output.

How to transform JSON text to relational table? OPENJSON function can be used to convert JSON text into table rows and columns or to import JSON into SQL tables. By default, when we use the OPENJSON function, it returns 3 values – key, value and type. Type Value JSON Data Type NULL 1 STRING 2 INT 3 BOOL 4 ARRAY 5 OBJECT

JSON Functions There are a number of newly added JSON functions to provide support for handling JSON data – Function Name Description ISJSON() Verifies that the text has valid JSON data. JSON_QUERY() Extract JSON fragment from JSON text. JSON_VALUE() Extract value from JSON text. JSON_MODIFY() Add, Delete or Update properties in JSON text

ISJSON ISJSON (expression) where expression is a variable or column that contains a JSON string, which we want to test.   If it is valid JSON, the function returns 1. If it is not valid, then the function returns a 0. It returns NULL if the expression is NULL.  OPENJSON function can be used to convert JSON text into table rows and columns. Before using this function, it is better to validate the correctness of the JSON text. When we receive a JSON object from the front end application, we will need to do a quick verification in SQL Server side to ensure that the JSON is valid, before storing it into the database. It is always a safe idea to validate the JSON text, before parsing JSON data.

JSON_VALUE Used to extract one scalar value from the JSON data   JSON_VALUE ( expression , path ) where expression is a variable or column that contains a JSON string. and path specifies the property whose value we want to extract. You can control the return value of the JSON_VALUE function by specifying 2 types of mode - Lax mode - Returns NULL if the path does not exist. Strict mode - Returns an error if path does not exist. Earlier we looked at OPENJSON function which converts the JSON text into table rows. Sometimes we might need to just extract one scalar value from the JSON data, instead of parsing and returning the entire data. In such cases, we can use a new function called JSON_VALUE.

JSON_QUERY It is used to extract an object or array from a JSON string. It is very similar to JSON_VALUE function and has a similar syntax -   JSON_VALUE ( expression , path ) where expression is a variable or column that contains a JSON string. and path specifies the object/array whose value we want to extract. JSON_VALUE function returns a scalar value whereas JSON_QUERY returns an object or an array from the JSON data.

JSON_MODIFY This function updates the value of a property in a JSON string and returns the updated JSON string.   JSON_MODIFY ( expression , path , newValue ) where expression is a variable or column that contains a JSON string. and path specifies the property whose value we want to extract. and newValue is the modified value of the property specified by path.

JSON Support in action…

Temporal Tables

What is Temporal Table? A new type of system-versioned user table that holds the entire history of data changes. Temporal Table in SQL Server 2016 provides inbuilt support for tracking old versions of data over a period of time, without any need for additional programming.

Why is Temporal required ? Gives us the ability to reconstruct the state of the data, as it was a point of time. It tracks all changes that has happened to our data – who/what/when data changed.  It assists in historical data analysis.  We have the provision of going back to a point of time and fetching the correct data, which got wiped out/corrupted. Time Travel Simplify Data Audit Trend Analysis Repair Corruptions

How Temporal Data works ? Temporal Table History Table Old Versions Update/ Delete Insert/ Bulk Insert

Temporal Tables in action…

Stretch Database

What is a Stretch Database? Stretch Databases will allow you to dynamically extend your on-premise database to Azure, enabling your frequently accessed data to stay on-premise and your infrequently accessed data to be moved to the cloud.

What are the Benefits of Stretch Database? Provides cost-effective availability for cold data Changes to queries or applications are not required Reduces on-premises maintenance Keeps data secure during migration

Azure Stretch Database On-premise SQL Server instance

T-SQL Enhancements

DROP IF EXISTS Statement Syntax – DROP OBJECT_TYPE [ IF EXISTS ] OBJECT_NAME Example - DROP TABLE IF EXISTS dbo.Employee DROP IF EXISTS conditionally drops the column or constraint only if it already exists. If the object does not exist, it will not throw any error and the TSQL execution will continue on. Applies to – Database, Table, Function, Trigger, Stored Procedure, Column, User, View, Schema, Index , Role

DATEDIFF_BIG Function Syntax – DATEDIFF_BIG ( DatePart , StartDate , EndDate ) This new function gives the difference between the two dates (StartDate and EndDate) in the units specified by the DatePart parameter and the returned unit is of type bigint.   It is very much similar to the DATEDIFF function, difference being that DATEDIFF function return type is INT, whereas the DATEDIFF_BIG functions return type is BIGINT.

STRING_SPLIT() Function Syntax - STRING_SPLIT ( string , separator ) STRING_SPLIT is a T-SQL function that splits an input string by a separator and outputs the results as a table.

COMPRESS and DECOMPRESS Function COMPRESS – Compress data using GZip algorithm and returns binary data. DECOMPRESS – Decompress binary data using GZip algorithm and returns binary data. You will need to cast binary data to text to get the original compressed text. You can compress data in Client Side and send compressed data to SQL Server. You can compress data in SELECT query and decompress it in the Client side.

T-SQL Enhancements in action…

Always Encrypted

What is Always Encrypted? It is a new data encryption technology designed to protect sensitive data, stored in SQL Server databases at rest, during movement between client and server as well as when data is in use. Client side encryption and decryption. Data is never in plain text while being stored or accessed in SQL Server. Once you encrypt data, only client applications or app servers, that have access to the keys, can access plaintext data.

What are the Always Encrypted Key types? Column Encryption Key Column Master Key Stored in SQL Server Stored in a trusted key store Used to encrypt/decrypt the sensitive data in the columns. Used to protect the Column Encryption Key. Using the Column Master Key, it can decrypt the Column Encryption Key to decrypt/encrypt sensitive data. SQL Server does not have access to the Master Key directly, hence wont be able to decrypt the data.

How does Always Encrypted work ? Client Side "SELECT Name FROM Employees WHERE SSN = @SSN", “123-45-6789" "SELECT Name FROM Employees WHERE SSN = @SSN", “0x6ff345ea5a" ADO.NET Driver Name Samir Behara Name 0x12ea958bdf8a Name SSN Country 0x12ea958bdf8a 0x6ff345ea5a USA Column Master Key Column Encryption Key

Why is Always Encrypted required? Additional security for sensitive data in flight Prevent database users from having access to sensitive data Running database in the cloud Regulatory Compliance And Data Audits

Dynamic Data Masking

What is Dynamic Data Masking? A new built-in security feature which will limit access to sensitive information by masking stored data. We can configure users who can have access to the unmasked data and for other unauthorized users, just show the masked data. Data is masked on the fly and the underlying data in the database does not change. Very simple to enable this feature in existing databases. No need of any application changes to take advantage of this functionality. Not a replacement for encryption.

What are the different Masking functions available? Description DEFAULT() This method can be described as Full masking. It replaces the content of the column with ‘XXXX’. EMAIL() This method exposes the first letter of the email address and the constant suffix ‘.com’. It converts the email-id to the format aXXX@XXX.com PARTIAL() This method exposes the first and the last letters and adds a custom padding string in the middle. RANDOM() This method is used to mask numeric types with a random value within a specified range.

How does Dynamic Data Masking work? Security Officer defines the Data Masking Policy for sensitive data. Admin Other Users

Dynamic Data Masking in action…

More information… SQL Scripts used for the purpose of this presentation are upload in – https://gist.github.com/samirbehara More in-depth details on various SQL Server 2016 features on my blog – https://dotnetvibes.com/

Questions?

THANK YOU