Mission critical features in SQL 2016 David Lyth Pat Martin Premier Field Engineers, Microsoft New Zealand

SQL Server 2016 highlights

PerformanceSecurityAvailabilityScalability Operational analytics Insights on operational data; Works with in-memory OLTP and disk- based OLTP In-memory OLTP enhancements Greater T-SQL surface area, terabytes of memory supported, and greater number of parallel CPUs Query Store Monitor and optimise query plans Native JSON Expanded support for JSON data Temporal Database Query data as at points in time Always encrypted Sensitive data remains encrypted at all times with ability to query Row Level Security Apply fine-grained access control to table rows Dynamic Data Masking Real-time obfuscation of data to prevent unauthorised access Other enhancements Audit success/failure of database operations TDE support for storage of in-memory OLTP tables Enhanced auditing for OLTP with ability to track history of record changes Enhanced AlwaysOn Three synchronous replicas for auto failover across domains Round robin load balancing of replicas Automatic failover based on database health DTC for transactional integrity across database instances with AlwaysOn Support for SSIS with AlwaysOn Enhanced caching Cache data with automatic, multiple TempDB files per instance in multi-core environments Support for Windows Server TB of memory supported Mission critical features

Have You Ever…? …had your system slow down and everyone wait for you to magically fix the problem … …upgraded an application to the latest SQL Server version and had an issue with a plan change slowing your application down?

With Query Store…

Durability latency controlled by DB option DATA_FLUSH_INTERNAL_SECONDS Compile Execute Plan Store Runtime Stats Query Store Schema Query Data Store Collects query texts (+ all relevant properties) Stores all plan choices and performance metrics Works across restarts / upgrades / recompiles Simplifies performance troubleshooting New Views Intuitive and easy plan forcing

Keeping stability while upgrading to SQL Sever 2016 Upgrade to SQL vNext Keep 110/120 Compatibility Level Freeze plans (optional) Run Query Store (establish performance baseline) Move to 130 Compatibility Level and unfreeze plans Monitor performance and fix regressions with plan forcing

Monitoring performance using Query Store The Query Store feature provides DBAs with insight on query plan choice and performance

/* (1) Turn ON Query Store */ ALTER DATABASE MyDB SET QUERY_STORE = ON; /* (2) Review current Query Store parameters */ SELECT * FROM sys.database_query_store_options /* (3) Set new parameter values */ ALTER DATABASE MyDB SET QUERY_STORE ( OPERATION_MODE = READ_WRITE, CLEANUP_POLICY = ( STALE_QUERY_THRESHOLD_DAYS = 30 ), DATA_FLUSH_INTERVAL_SECONDS = 3000, MAX_SIZE_MB = 500, INTERVAL_LENGTH_MINUTES = 15 ); /* (4) Clear all Query Store data */ ALTER DATABASE MyDB SET QUERY_STORE CLEAR; /* (5) Turn OFF Query Store */ ALTER DATABASE MyDB SET QUERY_STORE = OFF; /* (6) Performance analysis using Query Store views*/ SELECT q.query_id, qt.query_text_id, qt.query_sql_text, SUM(rs.count_executions) AS total_execution_count FROM sys.query_store_query_text qt JOIN sys.query_store_query q ON qt.query_text_id = q.query_text_id JOIN sys.query_store_plan p ON q.query_id = p.query_id JOIN sys.query_store_runtime_stats rs ON p.plan_id = rs.plan_id GROUP BY q.query_id, qt.query_text_id, qt.query_sql_text ORDER BY total_execution_count DESC /* (7) Force plan for a given query */ exec sp_query_store_force_plan DB-level feature exposed through T-SQL extensions ALTER DATABASE Catalog views (settings, compile and runtime stats) Stored Procs (plan forcing, query/plan/stats cleanup)

Live Query Statistics Collect actual metrics about query while running View CPU/memory usage, execution time, query progress, etc. Enables rapid identification of potential bottlenecks for troubleshooting query performance issues. Allows drill down to live operator level statistics: Number of generated rows Elapsed time Operator progress Live warnings, etc.

Real data sources are dynamic Historical data may be critical to business success Traditional databases fail to provide required insights Workarounds are… Complex, expensive, limited, inflexible, inefficient SQL Server 2016 makes life easy No change in programming model Provides new insights Why Temporal Time TravelData Audit Slowly Changing Dimensions Repair record- level corruptions

N o change in programming model New insights INSERT / BULK INSERT UPDATE DELETE MERGE DML SELECT * FROM temporal Querying How to start with Temporal CREATE temporal TABLE PERIOD FOR SYSTEM_TIME… ALTER regular_table TABLE ADD PERIOD… DDL FOR SYSTEM_TIME AS OF FROM..TO BETWEEN..AND CONTAINED IN Temporal Querying

Temporal table (actual data) Insert / Bulk Insert * Old versions Update */ Delete * How system-time works? History Table

Temporal table (actual data) Temporal Queries * (Time travel,etc.) How system-time works? History Table Regular queries (current data) * Include Historical Version

DepNumDepNameMngrIDFromTo A001Marketing A002Sales A003Consulting A003Consulting DepNumDepNameMngrID A001Marketing5 A001Marketing6 A002Sales2 A002Sales5 A003Consulting6 A003Consulting10 DepNumDepNameMngrIDFromTo A001Marketing62008 ∞ A002Sales52007 ∞ A001 A002 A003 period of validity current time ∞ ∞ Department (current) Department (history) Department (current + history) SELECT * FROM Department SELECT * FROM Department FOR SYSTEM_TIME BETWEEN ' ' AND ' ' SELECT * FROM Department FOR SYSTEM_TIME CONTAINED IN (' ', ' ') SELECT * FROM Department FOR SYSTEM_TIME AS OF ' ' Getting insights from temporal A001 A002 A003 “Get actual row versions” AS OFBETWEEN..ANDCONTAINED IN

SELECT * FROM Department FOR SYSTEM_TIME AS OF ' ' Facts: 1.History is much bigger than actual data 2.Retained between 3 and 10 years 3.“Warm”: up to a few weeks/months 4.“Cold”: rarely queried Solution: History as a stretch table: PeriodEnd < “Now - 6 months” Azure SQL Database

Provides correct information about stored facts at any point in time, or between two points in time. There are two orthogonal sets of scenarios with regards to temporal data: System(transaction)-time Application-time SELECT * FROM Person.BusinessEntityContact FOR SYSTEM_TIME WHERE ContactTypeID = 17 Performance Temporal database support - BETWEEN

Fine-grained access control over specific rows in a database table Help prevent unauthorised access when multiple users share the same tables, or to implement connection filtering in multitenant applications Administer via SQL Server Management Studio or SQL Server Data Tools Enforcement logic inside the database and schema bound to the table. Protect data privacy by ensuring the right access across rows SQL Database Customer 1 Customer 2 Customer 3 Row Level Security

Fine-grained access control Keeping multi-tenant databases secure by limiting access by other users who share the same tables. Application transparency RLS works transparently at query time, no client application changes needed. Compatible with RLS in other leading products. Centralised security logic Enforcement logic resides inside the database and is schema-bound to the table it protects, providing greater security, reduced application maintenance and complexity. Store data intended for many consumers in a single database/table while at the same time restricting row level read and write access based on users’ execution context. Benefits of Row Level Security

Sample RLS use cases A hospital can create a security policy that allows nurses to view data rows for their own patients only. A bank can create a policy to restrict access to rows of financial data based on the employee's business division, or based on the employee's role within the company. A multi-tenant application can create a policy to enforce a logical separation of each tenant's data rows from every other tenant's rows. Efficiencies are achieved by the storage of data for many tenants in a single table. Of course, each tenant can view only their own data rows.

CREATE SECURITY POLICY mySecurityPolicy ADD FILTER PREDICATE dbo.fn_securitypredicate(wing, startTime, endTime) ON dbo.patients Predicate function User-defined inline table-valued function (iTVF) implementing security logic Can be arbitrarily complicated, containing joins with other tables Security predicate Applies a predicate function to a particular table Two types: filter predicates and blocking predicates – blocking predicate not in SQL16 CTP2 Security policy Collection of security predicates for managing security across multiple tables RLS concepts

Security CREATE FUNCTION int) RETURNS TABLE WITH SCHEMABINDING AS return SELECT 1 as [fn_securitypredicate_result] FROM StaffDuties d INNER JOIN Employees e ON (d.EmpId = e.EmpId) WHERE e.UserSID = SUSER_SID() = d.Wing; CREATE SECURITY POLICY dbo.SecPol ADD FILTER PREDICATE dbo.fn_securitypredicate(Wing) ON Patients WITH (STATE = ON) Fine-grained access control over rows in a table based on one or more predefined filtering criteria, e.g., user’s role or clearance level in an organisation Concepts: Predicate function Filter Predicate Security policy RLS implementation example

Two App user (e.g., nurse) selects from Patients table Three Security Policy transparently rewrites query to apply filter predicate Database Policy Manager CREATE FUNCTION int) RETURNS TABLE WITH SCHEMABINDING AS return SELECT 1 as [fn_securitypredicate_result] FROM StaffDuties d INNER JOIN Employees e ON (d.EmpId = e.EmpId) WHERE e.UserSID = SUSER_SID() = d.Wing; CREATE SECURITY POLICY dbo.SecPol ADD FILTER PREDICATE dbo.fn_securitypredicate(Wing) ON Patients WITH (STATE = ON) Filter Predicate: INNER JOIN… Security Policy Applicatio n Patients One Policy manager creates filter predicate and security policy in T-SQL, binding the predicate to the Patients table Nurse SELECT * FROM Patients SEMIJOIN APPLY dbo.fn_securitypredicate(patients.Wing); SELECT Patients.* FROM Patients, StaffDuties d INNER JOIN Employees e ON (d.EmpId = e.EmpId) WHERE e.UserSID = SUSER_SID() AND Patients.wing = d.Wing; RLS in three steps

ALTER TABLE [Employee] ALTER COLUMN [SocialSecurityNumber] ADD MASKED WITH (FUNCTION = ‘SSN()’ ALTER TABLE [Employee] ALTER COLUMN [ ] ADD MASKED WITH (FUNCTION = ‘ ()’) GRANT UNMASK to admin1 1)Security officer defines dynamic data masking policy in T-SQL over sensitive data in Employee table 2) App user selects from Employee table 3) Dynamic data masking policy obfuscates the sensitive data in the query results SELECT [Name], [SocialSecurityNumber], [ ], [Salary] FROM [Employee] SELECT Patients.* FROM Patients, StaffDuties d INNER JOIN Employees e ON (d.EmpId = e.EmpId) WHERE e.UserSID = SUSER_SID() AND Patients.wing = d.Wing;

ALTER TABLE Membership ALTER COLUMN FirstName ADD MASKED WITH (FUNCTION = 'partial(1,"XXXXXXX",0)'); ALTER TABLE Membership ALTER COLUMN Phone# ADD MASKED WITH (FUNCTION = 'default()'); ALTER TABLE Membership ALTER COLUMN ADD MASKED WITH (FUNCTION = ' ()'); EXECUTE AS USER = 'TestUser'; SELECT * FROM Membership; REVERT; --Grant unmask to see all data GRANT UNMASK TO TestUser; EXECUTE AS USER = 'TestUser'; SELECT * FROM Membership; REVERT; -- Removing the UNMASK permission REVOKE UNMASK TO TestUser; --Remove masking on a column ALTER TABLE Membership ALTER COLUMN FirstName DROP MASKED; ALTER TABLE [Membership] ALTER COLUMN [FirstName] ADD MASKED WITH (FUNCTION = 'partial(1,"XXXXXXX",0)' ALTER TABLE [Membership] ALTER COLUMN [Phone#] ADD MASKED WITH (FUNCTION = 'default()'); ALTER TABLE [Membership] ALTER COLUMN [ ] ADD MASKED WITH (FUNCTION = ‘ ()’) GRANT UNMASK to admin1

PerformanceSecurityAvailabilityScalability Operational analytics Insights on operational data; Works with in-memory OLTP and disk- based OLTP In-memory OLTP enhancements Greater T-SQL surface area, terabytes of memory supported, and greater number of parallel CPUs Query Store Monitor and optimise query plans Native JSON Expanded support for JSON data Temporal Database Query data as at points in time Always encrypted Sensitive data remains encrypted at all times with ability to query Row Level Security Apply fine-grained access control to table rows Dynamic Data Masking Real-time obfuscation of data to prevent unauthorised access Other enhancements Audit success/failure of database operations TDE support for storage of in-memory OLTP tables Enhanced auditing for OLTP with ability to track history of record changes Enhanced AlwaysOn Three synchronous replicas for auto failover across domains Round robin load balancing of replicas Automatic failover based on database health DTC for transactional integrity across database instances with AlwaysOn Support for SSIS with AlwaysOn Enhanced caching Cache data with automatic, multiple TempDB files per instance in multi-core environments Support for Windows Server TB of memory supported Mission critical features

Subscribe to our fortnightly newsletter Free Online Learning Sessions on Demand