Reinhard Flügel Possiblities and Limitations of System-Versioned Temporal Tables beyond the Basics

Sponsors Gold Sponsors Trusted Partner Innovation Sponsor PASS Global Sponsor PASS Swag Sponsor

About Me Reinhard Flügel (r.fluegel@fluegel-se.de) Freelancer (Programming C++,C#, Access, SQL-Sever since the early 90’s) MCSE Data Platform, Business Intelligence, Data Management and Analytics (Charter) My way to System-Versioned Tables (New Project in October 2016; Project manager: We use Temporal Tables in a DWH-Project, but when we load the data we plan to set the valid times for all tables to a specific value.)

Agenda Short Introduction System-Versioned Temporal Tables Manipulate Data with versioning switched off Query multiple System-Versioned tables Views on System-Versioned tables

Short Introduction System-Versioned Temporal Tables

From the Microsoft article: https://docs. microsoft System-versioning for a table is implemented as a pair of tables, a current table and a history table. Within each of these tables, the following two additional datetime2 columns are used to define the period of validity for each row: Period start column: The system records the start time for the row in this column, typically denoted as the SysStartTime column. Period end column: The system records the end time for the row in this column, typically denoted at the SysEndTime column. The current table contains the current value for each row. The history table contains each previous value for each row, if any, and the start time and end time for the period for which it was valid.

Sample: CREATE TABLE dbo.Employee ( [EmployeeID] int NOT NULL PRIMARY KEY CLUSTERED , [Name] nvarchar(100) NOT NULL , [Position] varchar(100) NOT NULL , [Department] varchar(100) NOT NULL , [Address] nvarchar(1024) NOT NULL , [AnnualSalary] decimal (10,2) NOT NULL , [ValidFrom] datetime2 (2) GENERATED ALWAYS AS ROW START , [ValidTo] datetime2 (2) GENERATED ALWAYS AS ROW END , PERIOD FOR SYSTEM_TIME (ValidFrom, ValidTo) ) WITH (SYSTEM_VERSIONING = ON (HISTORY_TABLE = dbo.EmployeeHistory));

Remark: The history table is page compressed. The above statement creates although an index on the history table. CREATE CLUSTERED INDEX [ix_EmployeeHistory] ON [dbo].[EmployeeHistory] ( [ValidTo] ASC, [ValidFrom] ASC ) Different index strategy: https://docs.microsoft.com/en-us/sql/relational-databases/tables/creating-a-system-versioned-temporal-table https://www.infoq.com/news/2015/06/SQL-Server-Temporal Illustrate the use of a Clustered Columnstore index on the history table with an additional Nonclustered Index.

Remark: Information on system-versioned temporal tables: Select * from sys.tables (temporal_type, temporal_type_desc, history_table_id) Select * from sys.columns (generated_always_type, generated_always_type_desc, ishidden) Select * from sys.periods Demo: Create system-versioned table based on Microsoft article) 01-01 – TableFromMSArticle Demo: (Show infos on system-versioned table) 01-02 – InfoONTable

Manipulate Data with versioning switched off

Modifying Data in a System-Versioned Temporal Table Articles: Modifying Data in a System-Versioned Temporal Table https://docs.microsoft.com/en-us/sql/relational-databases/tables/modifying-data-in-a-system-versioned-temporal-table Temporal Table System Consistency Checks https://docs.microsoft.com/en-us/sql/relational-databases/tables/temporal-table-system-consistency-checks Data Consistency Check Before SYSTEM_VERSIONING is set to ON and as part of any DML operation, the system performs the following check: SysEndTime ≥SysStartTime Marko Zivkovic: Temporal tables in SQL Server https://www.sqlshack.com/temporal-tables-sql-server

Switch Versioning OFF Switch Versioning ON ALTER TABLE <table, sysname, tablename> SET (SYSTEM_VERSIONING = OFF); Switch Versioning ON ALTER TABLE <table, sysname, tablename> SET (SYSTEM_VERSIONING = ON (HISTORY_TABLE = <table, sysname, tablename>History, DATA_CONSISTENCY_CHECK = ON ));

Demo: Prepare Table 02-00 – PrepareTable History Table: Demo: (Update not auto-generated columns in the history table) 02-01-01 – UpdateHistoryTableColumnNotGenerated 02-01-02 – UpdateHistoryTableColumnNotGenerated Demo: (Update auto-generated columns in the history table) 02-02 – UpdateHistoryTableColumnGenerated Actual Table: Demo: Update not auto-generated columns in the actual table 02-03 - UpdateActualTableColumnNotGenerated Demo: Update auto-generated columns in the actual table 02-04-01 – UpdateActualTableColumnGenerated 02-04-02 – UpdateActualTableColumnGenerated Demo: Use DATA_CONSISTENCY_CHECK = OFF 02-05-00 – PrepareTable 02-05-01 – SetVersioningONWithCHECK_ON 02-05-02 – SetVersioningONWithCHECK_OFF 02-05-03 - Queries

Summary To Update Data in System-Versioned temporal tables the Versioning has to be switched of. For Updating data in the actual table Period for SYSTEM_Time has to be dropped, when preserving the Valid Times, or when updating the times. DATA_CONSISTENCY_CHECK kann be turned off, but on your own risk (Depending on the data, it can be substantially faster) Demo 02-05 shows what can be done, but I have not tested it in practice.

Query multiple System-Versioned tables

Querying Data in a System-Versioned Temporal Table Article: Querying Data in a System-Versioned Temporal Table https://docs.microsoft.com/en-us/sql/relational-databases/tables/querying-data-in-a-system-versioned-temporal-table Without the use of FOR SYSTEM_TIME only actual tables are used. Demo: Basic queries without FOR SYSTEM_TIME, with FOR SYSTEM_TIME ALL and FOR SYSTEM_TIME AS OF in 02-05-03 - Queries

Demo: Query non temporal table with FOR SYSTEM_TIME 03-01-00 CreateNonTemporalTable 03-01-01 QueryNonTemporalTableForSYSTEM_TIME Demo: Prepare tables 03-02-00 CreateDemoTables Demo: Query two temporal tables (for the same date) 03-02-01 QueriesOnTwoTemporalTablesSameDate Demo: Query two temporal tables (for not the same date) 03-02-02 QueriesOnTwoTemporalTablesNotSameDate Demo: Use a CTE 03-03-01 UseCTE_Version1 03-03-02 UseCTE_Version2 Demo: Use a Scalar function without use of SYSTEM_TIME 03-04-00 - Create funcScalar_RemarkForIDNoSYSTEM_TIME 03-04-01 - Use funcScalar_RemarkForIDNoSYSTEM_TIME Demo: Use a Scalar function witht use of SYSTEM_TIME 03-05-00 - Create funcScalar_RemarkForIDWITH_SYSTEM_TIME 03-05-01 - Use funcScalar_RemarkForIDWITH_SYSTEM_TIME

Demo: Use an inline table valued function with and without use of SYSTEM_TIME 03-06-00 - Create inline table valued functions 03-06-01 - Use inline table valued functions Demo: Use a multistatement table valued function witht and without use of SYSTEM_TIME 03-07-00 - Create multistatement table valued functions 03-07-01 - Use multistatement table valued functions

Summary There is no magic The Clause FOR SYSTEM_TIME has effect only on the table it follows directly It only acts as a Filter for the referenced Time It does not prevent logically invalid results Use in a CTE or a function is similar Use of the Clause FOR SYSTEM_Time not directly following a System-Versioned temporal table RESULTS IN AN ERROR

Views on System-Versioned tables

Querying Data in a System-Versioned Temporal Table Article: Querying Data in a System-Versioned Temporal Table https://docs.microsoft.com/en-us/sql/relational-databases/tables/querying-data-in-a-system-versioned-temporal-table Using views with AS-OF sub-clause in temporal queries Using views is very useful in scenarios when complex point-in time analysis is required. A common example is generating a business report today with the values for previous month. Usually, customers have a normalized database model which involves many tables with foreign key relationships. Answering the question how data from that normalized model looked like at a point in the past can very challenging, since all tables change independently, on their own cadence. In this case, the best option is to create a view and apply the AS OF sub-clause to the entire view. Using this approach allows you to decouple modeling of the data access layer from point-in time analysis as SQL Server will apply AS OF clause transparently to all temporal tables that participate in view definition. Furthermore, you can combine temporal with non-temporal tables in the same view and AS OF will be applied only to temporal ones. If view does not reference at least one temporal table, applying temporal querying clauses to it will fail with an error.

Demo: View on non temporal table 04-01-01 - CreateViewNonTemporalTableAndQueryWithSYSTEM_TIME Demo: Query views on two system-versioned tables 04-02-00 - CreateViewONTemporalTables 04-02-01 - QueryViewWithSYSTEM_TIME 04-02-02 - QueryViewWithSYSTEM_TIME_Interval 04-02-03 - CreateVIEWWithJoinOnWithSYSTEM_TIME 04-02-04 - QueryViewWITHJoinOnWithSYSTEM_TIME Demo: Query views including inline table valued functions 04-03-00 - CreateViewWithInlineTBV_WITHOUT_SYSTEM_TIME 04-03-01 - UseViewWithInlineTBV_WITHOUT_SYSTEM_TIME 04-04-00 - CreateViewWithInlineTBV_WITH_SYSTEM_TIME 04-04-01 - UseViewWithInlineTBV_WITH_SYSTEM_TIME

Demo: Query views including multistatement table valued functions 04-05-00 - CreateViewWithIMultiTBV_WITHOUT_SYSTEM_TIME 04-05-01 -UseViewWithMultiTBV_WITHOUT_SYSTEM_TIME 04-06-00 - CreateViewWithIMultiTBV_WITH_SYSTEM_TIME 04-06-01 - UseViewWithIMultiTBV_WITH_SYSTEM_TIME Demo: Query view including a CTE and scalar functions with use and without use FOR SYSTEM_TIME 04-07-00 – CreateViewWithCTEAndScalarFunctions 04-07-01 - UseViewWithCTEAndScalarFunctions

Important to remember: DECLARE @ReferenceDateCurrent datetime2(0) = '20180411 03:00:00' DECLARE @ReferenceDateYesterday datetime2(0) = '20180410 03:00:00' SELECT * from [dbo].[DemoViewTemporalMultiTables] FOR SYSTEM_TIME AS OF @ReferenceDateCurrent WHERe IDPK = 1 UNION ALL SELECT * from [dbo].[DemoViewTemporalMultiTables] FOR SYSTEM_TIME AS OF @ReferenceDateYesterday -- IS NOT SELECT * from [dbo].[DemoViewTemporalMultiTables] FOr SYSTEM_TIME BETWEEN @ReferenceDateYesterday and @ReferenceDateCurrent where IDPK=1 My Opinion: For Views only FOR SYSTEM_TIME AS OF should be allowed

Summary Applying SYSTEM_TIME to a non System-Versioned View leads to an error. Applying SYSTEM_TIME to a SYSTEM-Versioned Table based on a single table works like expected A View can be build with a FOR SYSTEM_TIME clause, but than it can not be called with FOR SYSTEM_TIME Applying FOR SYSTEM_TIME to a view referencing multiple temporal tables works as expected.

If a View is based on multiple temporal tables and one uses FOR SYSTEM_TIME AS OF the View cannot be called with FOR SYSTEM_TIME AS Of Use of table valued functions in a view leads for Inline TBVF‘s to unexpected behaviour. For multistatement TBVF‘s the FOR SYSTEM_TIME clause does not propagate to the function. This allows constructs to use a function to get results for a different date than the base view. CTE‘s and scalar funtions work as expected.

Sponsors Gold Sponsors Trusted Partner Innovation Sponsor PASS Global Sponsor PASS Swag Sponsor