Developing modern applications with Temporal Tables and JSON Jos de Bruijn, Borko Novakovic Senior Program Manager

1 2 3 Takeaways Make your life easier, let the database do it. Tech Ready 15 Takeaways 4/22/2018 Make your life easier, let the database do it. Temporal tables enable easy time travel over data. JSON connects SQL to modern apps and services. 1 2 3 © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Less code is better Works but error-prone Simple and intuitive 4/22/2018 Works but error-prone IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[mytable]') AND TYPE IN (N'U') ) DROP TABLE [dbo].[DumpMe] Simple and intuitive DROP IF EXISTS [dbo].[mytable] © 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Complex tasks made simple 4/22/2018 Analyze data history yourself Let the temporal tables do it 1 SCHEMA MAINTENANCE Double effort to maintain current and history. 1 SCHEMA MAINTENANCE Automatic and online. 2 HISTORY TRACKING User code is required (triggers, SPs, app). Hard to maintain and achieve good performance. 2 HISTORY TRACKING Automatic and optimal. 3 DATA ANALYSIS Complex queries are required. 3 DATA ANALYSIS Simple and efficient with FOR SYSTEM_TIME clause. 4 DATA PROTECTION Immutability of history data cannot be guaranteed. 4 DATA PROTECTION Out-of-box immutability of history data. © 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

4/22/2018 Temporal tables © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

When to use temporal tables 4/22/2018 Audit and data forensics Historical data analysis Slowly changing dimensions Pinpoint and correct erroneous data © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

How to start with temporal tables Microsoft Ignite 2015 How to start with temporal tables 4/22/2018 3:42 PM ANSI 2011 compliant no change in programming model new insights SELECT * FROM temporal Querying FOR SYSTEM_TIME AS OF FROM..TO BETWEEN..AND CONTAINED IN Temporal Querying INSERT / BULK INSERT UPDATE DELETE MERGE DML CREATE temporal TABLE PERIOD FOR SYSTEM_TIME… ALTER regular_table TABLE ADD PERIOD… DDL © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Getting started (DDL) PERIOD definition VERSIONING options CREATE TABLE Department ( DepNum char(10) NOT NULL PRIMARY KEY CLUSTERED, DepName varchar(50) NOT NULL, MngrID INT NULL,   SysStart datetime2 GENERATED ALWAYS AS ROW START, SysEnd datetime2 GENERATED ALWAYS AS ROW END, PERIOD FOR SYSTEM_TIME (SysStart,SysEnd) )WITH(SYSTEM_VERSIONING = ON (HISTORY_TABLE = DepartmentHistory) ) ALTER TABLE Department ADD ValidFrom datetime2 (0) GENERATED ALWAYS AS ROW START HIDDEN, ADD ValidTo datetime2 (0) GENERATED ALWAYS AS ROW END HIDDEN, PERIOD FOR SYSTEM_TIME (ValidFrom, ValidTo); ALTER TABLE Product SET (SYSTEM_VERSIONING = ON (HISTORY_TABLE = dbo.Department_History)); PERIOD definition VERSIONING options

Temporal System-Versioning (DML) Microsoft Ignite 2015 Temporal System-Versioning (DML) 4/22/2018 3:42 PM temporal table (current data) history table * Old versions Insert / Bulk Insert / Select Update */ Delete * © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

* Include historical versions Microsoft Ignite 2015 Temporal queries 4/22/2018 3:42 PM temporal table (current data) history table * Include historical versions Regular queries: SELECT … Temporal queries*: SELECT … AS OF time © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Temporal queries (details) AS OF “get actual row versions” BETWEEN..AND CONTAINED IN Department (current + history) Department (current) DepNum DepName MngrID From To A001 Marketing 6 2008 ∞ A002 Sales 5 2007 DepNum DepName MngrID A001 Marketing 5 6 A002 Sales 2 A003 Consulting 10 A001 ∞ A001 A002 Department (history) ∞ A002 DepNum DepName MngrID From To A001 Marketing 5 2005 2008 A002 Sales 2 2007 A003 Consulting 6 2006 10 2009 2012 A003 A003 2005 2016 period of validity current time SELECT * FROM Department FOR SYSTEM_TIME CONTAINED IN ('2007.01.01', '2009.01.01') SELECT * FROM Department FOR SYSTEM_TIME BETWEEN '2006.01.01' AND '2007.01.01' SELECT * FROM Department SELECT * FROM Department FOR SYSTEM_TIME AS OF '2006.01.01'

Managing Retention of Historical Data Stretched history Partitioned history (sliding window) Custom delete script ALTER TABLE dbo.DepartmentHistory SWITCH PARTITION 1 TO DepartmentHistoryStaging; ALTER PARTITION FUNCTION [fn_Partition_DepartmentHistory_By_SysEndTime]() MERGE RANGE(N'2015-09-30T23:59:59.999'); /*...*/ Check out MSDN page for more details ALTER TABLE dbo.DepartmentHistory SET (REMOTE_DATA_ARCHIVE = ON (MIGRATION_STATE = OUTBOUND));

4/22/2018 JSON © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Scenarios for JSON in SQL 4/22/2018 Interchange data with apps and services. Exploit NoSQL agility to extend your app. Analyze relevant parts of semi-structured data. How to choose the service? Mostly relational data model + JSON use cases -> SQL Database. Only JSON docs -> Document DB. © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

What is (not) JSON in SQL Additional functions for data interchange and flexibility Not a native data type Store it as NVARCHAR. What works with NVARCHAR, works with JSON. Index with computed columns and B-trees There is no specific JSON index type.

Features Number Date Customer Price Quantity SO43659 4/22/2018 Built-in functions ISJSON JSON_VALUE JSON_MODIFY JSON_QUERY OPENJSON Transforms JSON text to table [ { "Number":"SO43659", "Date":"2011-05-31T00:00:00" "AccountNumber":"AW29825", "Price":59.99, "Quantity":1 }, { "Number":"SO43661", "Date":"2011-06-01T00:00:00“ "AccountNumber":"AW73565“, "Price":24.99, "Quantity":3 } ] Number Date Customer Price Quantity SO43659 2011-05-31T00:00:00 MSFT 59.99 1 SO43661 2011-06-01T00:00:00 Nokia 24.99 3 FOR JSON Formats result set as JSON text. © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

JSON to table @json: Query with OPENJSON function: Input table data: 4/22/2018 @json: [ { "Order": { "Number":"SO43659", "Date":"2011-05-31T00:00:00" }, "Account": "Microsoft", "Item": { "Price":59.99, "Quantity":1 } }, { "Order":{ "Number":"SO43661", "Date":"2011-06-01T00:00:00“ }, "Account": “Nokia“, "Item":{ "Price":24.99, "Quantity":3 } } ] Query with OPENJSON function: SELECT * FROM OPENJSON (@json) WITH ( Number varchar(200) N'$.Order.Number', Date datetime N'$.Order.Date', Customer varchar(200) N'$.Account', Quantity int N'$.Item.Quantity' ) Input table data: Number Date Customer Quantity SO43659 2011-05-31T00:00:00 Microsoft 1 SO43661 2011-06-01T00:00:00 Nokia 3 © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Table to JSON JSON output: Input table data: 4/22/2018 Input table data: JSON output: [ { "Order": { "Number":"SO43659", "Date":"2011-05-31T00:00:00" }, "Account": "Microsoft", "Item": { "Price":59.99, "Quantity":1 } }, { "Order":{ "Number":"SO43661", "Date":"2011-06-01T00:00:00“ }, "Account": “Nokia“, "Item":{ "Price":24.99, "Quantity":3 } } ] Number Date Customer Price Quantity SO43659 2011-05-31T00:00:00 MSFT 59.99 1 SO43661 2011-06-01T00:00:00 Nokia 24.99 3 Query with FOR JSON clause: SELECT Number AS [Order.Number], Date AS [Order.Date], Customer AS Account, Price AS 'Item.UnitPrice', Quantity AS 'Item.Qty' FROM SalesOrder FOR JSON PATH © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Built-ins and indexing 4/22/2018 CREATE TABLE SalesOrderRecord ( Id int PRIMARY KEY IDENTITY, OrderNumber NVARCHAR(25) NOT NULL, OrderDate DATETIME NOT NULL, SalesOrderItems NVARCHAR(MAX) CONSTRAINT SalesOrderDetails_IS_JSON CHECK ( ISJSON(SalesOrderItems)>0 ), Price AS JSON_VALUE(SalesOrderItems, '$.Order.Price') ) CREATE INDEX idx_JsonPrice ON SalesOrderRecord(Price) INCLUDE (Id, OrderNumber, OrderDate) © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Demo Product catalogs 4/22/2018 © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

More T-SQL enhancements 4/22/2018 More T-SQL enhancements © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

STRING_SPLIT FORMATMESSAGE COMPRESS/DECOMPRESS DROP IF EXISTS 4/22/2018 STRING_SPLIT Splits a string on a provided separator. FORMATMESSAGE Constructs a message from sys.messages or provided string. COMPRESS/DECOMPRESS Compresses the input expression using GZIP algorithm. DROP IF EXISTS Conditional drop for SQL objects. AT TIME ZONE Converts input date into a target time zone. DATEDIFF_BIG Returns signed big integer. BULK IMPORT with UTF8 © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Timelines and roadmap Temporal tables JSON T-SQL enhancements 4/22/2018 Temporal tables SQL Server 2016 RTM SQL Database - in public preview GA Automated data retention JSON SQL Database – in public preview Native compilation Column store integration T-SQL enhancements New string functions STRING_AGG IMPORT FROM Blob storage © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

1 2 3 Takeaways Make your life easier, let the database do it. Tech Ready 15 Takeaways 4/22/2018 Make your life easier, let the database do it. Temporal tables enable easy time travel over data. JSON connects SQL to modern apps and services. 1 2 3 © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

jodebrui@microsoft.com Tech Ready 15 4/22/2018 © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.