SQL Server 2016 Mohit K. Gupta | Microsoft SQL Server PFE

About Me Microsoft SQL Server PFE 5+ Years Specialize in SQL Server Engine

PerformanceSecurityAvailabilityScalability Operational Analytics Insights on operational data; works with in-memory OLTP and disk-based OLTP In-Memory OLTP Enhancements Greater T-SQL surface area, terabytes of memory supported, and greater number of parallel CPUs Query Data Store Monitor and optimize query plans Native JSON Expanded support for JSON data Temporal Database Support Query data as points in time Always Encrypted Sensitive data remains encrypted at all times with ability to query Row-Level Security Apply fine-grained access control to table rows Dynamic Data Masking Real-time obfuscation of data to prevent unauthorized access Enhanced AlwaysOn Three synchronous replicas for automatic failover across domains Round-robin load balancing of replicas Automatic failover based on database health DTC for transactional integrity across database instances with AlwaysOn Support for SSIS with AlwaysOn Enhanced Database caching Cache data with automatic, multiple TempDB files per instance in multicore environments Mission-Critical Performance

Performance / Operational Analytics

Traditional Operational Analytics Architecture  Key Issues  Complex Implementation  Requires two servers (Total Cost of Ownership)  Data Latency in Analytics  High Demand; Requires Real-Time Analytics IIS Server BI Analysts

Operational Analytics/ Goal  Goal/Challenges  Analytics Queries are resource intensive and can cause blocking.  Minimal impact on operational workload.  Performance analytics for Operational Schema.  How?  Column Store Indexes  Available in disk based and memory optimized tables  Updateable indexes  No application changes  Column store indexes maintained just like any other index  Query optimizer will choose appropriate indexes.

Operational Analytics/ Summary  Capability  Ability to run analytics queries concurrently with operational workloads using the same schema.  Data Warehouse queries can be run on In- Memory OLTP workload with no application changes.  Benefits  Minimal impact on OLTP workloads.  Best performance and scalability available.  Offloading analytics workload to readable secondary. IIS Server BI Analysts

Performance/ In-Memory OLTP Enhancements

In-Memory OLTP Enhancements  Alter Support  Full Schema Change Support  Add/Drop Index Support  Surface Area Improvement  Improved T-SQL Coverage  Improved Scaling  Indexes size for durable tables.  Other Improvements  MARS  Lightweight Migration Reports

In-Memory OLTP Enhancements / Improved Scaling / Enhancements  Multi-threads to persist memory optimized tables.  Multi-thread recovery.  Merge operations.  TDE Support.  MARS Support.

In-Memory OLTP Enhancements / SSMS Improvements  Lightweight Performance Analysis  Transaction Performance Analysis Overview report  Generating migration checklists

In-Memory OLTP Enhancements / SSMS Improvements Demo

Performance/ Query Data Store

Problem with Query Performance  Fixing query plan choice regressions is difficult  Query plan cache is not well- suited for performance troubleshooting.  Long time to detect the issue (TTD)  Which query is slow? Why is it slow?  What was the previous plan?  Long time to mitigate (TTM)  Can I modify the query?  How to use plan guide? Database is not working Website is down Impossible to predict / root cause analysis Temporary Perf Issues Regression caused by new bits DB Upgrade

Solution: Query Data Store  Dedicated store for query workload performance data  Captures the history of plans for each query  Captures the performance of each plan over time  Persists the data to disk (works across restarts, upgrades, and recompiles)  Significantly reduces TDD/TTM  Find regressions and other issues in seconds.  Allows you to force previous plans from history.  DBA is now in control

Query Store Architecture  Write-Behavior  Query Store captures data in- memory to minimize I/O overhead.  Data is persisted to disk asynchronously in the background. Query Store Query Execution Internal tables Query and Plan Store Runtime stats store Query exec. stats Compile Execute async Query text and plan Query Store views  Read-Behavior  View merged in-memory and on- disk content.  Users always see ‘latest’ data.

Performance/ Query Data Store Demo

Live Query Statistics  View CPU/Memory Usage, Execution Time, Query Progress, and More  Enable rapid identification of potential bottlenecks for troubleshooting query performance issues.  Allows drill down to live operator level statistics.

Performance/ Live Query Statistics Demo

Performance/ Native JavaScript Object Notation (JSON)

JSON and Microsoft  Top feature requested on Microsoft Connect (1050 Votes) OneDriveOffice DynamicsBing Yammer TFS

Use Cases for JSON Generate web service content Generate JSON that will be returned to clients Flexible database schema Make reasonable trade-off to database schema design Analyze JSON documents Parse, query, and analyze JSON documents Service Integration Exchange information with various services

JSON in SQL Server  Easily transform table to JSON by using “FOR JSON PATH” in select.  Built in functions to interact with JSON (ISJSON, JSON_VALUE, JSON_MODIFY)  Stores in NVARCHAR.  Works with in-memory, row-level security, stretch, etc.  Different from DocumentDB.  Transform JSON with OPENJSON.

JSON Summary  Capabilities  Greatly enhances developer productivity.  Benefits  Added native JSON support in the core database engine supports schema-free data. Tackle more diverse data types right in SQL Server.  Support in DoucmentDB.

Performance/ Temporal Database Support

Why Temporal?  Data changes over time  Tracking and analyzing changes is often important  Temporal in DB  Automatically track history of data changes  Enables easy querying of historical data states  Advantages over workarounds  Simplifies app development and maintenance  Efficiently handles complex logic in DB engine Time travelData audit Slowly changing dimensions Repair record- level corruptions

Temporal Table Details  Uses system-versioning using two tables current + history table.  No change in programming model.  New syntax introduced to query temporal data.  Time is stored in UTC.  Supports stretch database for history table.  Supports in-memory for current + on-disk for history table.

Security / Always Encrypted

Why Always Encrypted?  Allows customers to securely store sensitive data outside of their trust boundary Data remains protected from high-privileged, yet unauthorized users. Client-side encryption of sensitive data using keys that are never given to the database system Prevent Data Disclosure Support for equality comparison, including join, group by, and distinct operators Queries on Encrypted Data Minimal application changes via server and client library enhancements Application Transparency

Always Encrypted / How it Works?  Protects Data  At Rest, In Motion, On-Prem and in the Cloud  Two methods supported  Randomized encryption  More secure  Deterministic encryption  Allows Grouping, Equality Filtering, and Joins.  Uses column collation with BIN2 sort order.

Key Provisioning Security Officer 1. Generate CEKs and master key 2. Encrypt CEK 3. Store master key securely 4. Upload encrypted CEK to DB CMK store: Certificate store HSM Azure Key Vault … Encrypted CEK Column encryption key (CEK) Column master key (CMK) CMK Database Encrypted CEK Security

Security / Row-Level Security

Row Level Security / Why do we need it?  Protect data privacy by ensuring the right access across rows.  Fine-grained access control over specific rows in a database table  Help prevent unauthorized access when multiple users share the same tables, or to implement connection filtering in multitenant applications  Administer via SQL Server Management Studio or SQL Server Data Tools  Enforcement logic inside the database and schema is bound to the table

Benefits of Row-Level Security (RLS)  Store data intended for many consumers in a single database/table while at the same time restricting row- level read and write access based on users’ execution context. Keeping multitenant databases secure by limiting access by other users who share the same tables Fine-Grained Access Control RLS works transparently at query time, no app changes needed Compatible with RLS in other leading products Application Transparency Enforcement logic resides inside database and is schema-bound to the table it protects providing greater security. Reduced application maintenance and complexity Centralized Security Logic

Row Level Security / Concepts  Create Security Policy Composed of …  Predicate function (iTVF) used to implement security logic  And Security Predicate  SQL Server will dynamic expand the query to include the security function and predicate to minimize the rows being exposed.  No application change or code change required.

Row Level Security / Common Use Cases  Traditional RLS Workloads  Custom business logic to determine which rows each user can SELECT, INSERT, UPDATE, and DELETE based on their role, department, and security level  Target sectors: Finance, insurance, healthcare, energy, and government  Multitenant Databases  Ensuring tenants can only access their own rows of data in a shared database, with enforcement logic in the database rather than in the app tier  For example: multitenant shards with elastic database tools in SQL Database  Reporting, Analytics, and Data Warehousing  Different users access same database through various reporting tools, and work with different subsets of data based on their identity/role

Security / Dynamic Data Masking

Dynamic Data Masking  Prevent the Abuse of Sensitive data by hiding it from users  Configuration made easy in the new Azure portal  Policy-driven at the table and column level, for a defined set of users  Data masking applied in real-time to query results based on policy  Multiple masking functions available (e.g. full, partial) for various sensitive data categories (credit card numbers, SSN, etc.) SQL Database SQL Server 2016 Table.CreditCardNo Real-time data masking; partial masking

Title of Slide Here  Limit access to sensitive data by defining policies to obfuscate specific database fields, without affecting the integrity of the database. A strong demand for applications to meet privacy standards recommended by regulating authorities Regulatory Compliance Protects against unauthorized access to sensitive data in the application, and against exposure to developers or DBAs who need access to the production database Sensitive Data Protection Data is masked on the fly, with underlying data in the database remaining intact. Transparent to the application and applied according to user privilege Agility and Transparency

Data Masking / How It Works  Limit Sensitive Data Exposure by Obfuscating it to Non- Privileged Users  On-the-fly obfuscation of data in query results  Policy-driven on the table and column  Multiple masking functions available for various sensitive data categories  Flexibility to define a set of privileged logins for un- masked data access  By default, database owner is unmasked

Security / Dynamic Data Masking Demo

Availability / AlwaysOn Enhancements

Enhanced AlwaysOn Availability Groups  Great Scalability  Load balancing readable Secondaries.  Increased number of automatic failover targets  Log transport performance  Improved Manageability  Partial DTC Support  Database-Level health monitoring  Group managed service accounts  Domain-independent Availability Groups. AG Listener New York (Primary) Asynchronous data Movement Synchronous data Movement Unified HA solution AG Hong Kong (Secondary ) AG New Jersey (Secondary) AG

Load Balancing in Readable Secondaries Computer5 DR site Computer2 Computer3 Computer4 Primary site Computer1 (Primary) READ_ONLY_ROUTING_LIST= (('COMPUTER2', 'COMPUTER3', 'COMPUTER4'), 'COMPUTER5')

Access any dataScale and managePowerful insights Advanced analytics PolyBase Insights from data across SQL Server and Hadoop with the simplicity of T-SQL Enhanced SSIS Designer support for previous SSIS versions Enterprise-grade Analysis Services Enhanced performance and scalability for Analysis Services Single SSDT in Visual Studio 2015 (CTP3) Build richer analytics solutions as part of your development projects in Visual Studio Enhanced MDS Excel add-in 15x faster; more granular security roles; archival options for transaction logs; and reuse entities across models Mobile BI Business insights for your on- premises data through rich visualization on mobile devices with native apps for Windows, iOS, and Android Enhanced Reporting Services New modern reports with rich visualizations R integration (CTP3) Bringing predictive analytic capabilities to your relational database Expand your “R” script library with Microsoft Azure Marketplace Deeper Insights Across Data

SQL SATURDAY | 507 | EDMONTON 2016 Hybrid solutionsSimplicityConsistency Stretch Database Stretch operational tables in a secure manner into Azure for cost-effective historic data availability. Works with Always Encrypted and row-level security Power BI with on-premises data New interactive query with Analysis Services. Customer data stays behind your firewall Enhanced backup to Azure Faster restore times and 50% reduction in storage. Supports larger DBs with block blobs and custom backup schedule with local staging Easy migration of on-premises SQL Server Simple point-and-click migration to Azure Suite of advisors for upgrading to SQL Server 2016 SQL Server 2016 Upgrade Advisor in the adoption of new SQL Server features Simplified Add Azure Replica Wizard Automatic listener configuration for AlwaysOn in Azure VMs Common development, management, and identity tools Including Active Directory, Visual Studio, Hyper- V, and System Center Consistent experience from SQL Server on-premises to Microsoft Azure IaaS and PaaS Hyperscale Cloud

Q & A