SQL Server Analysis Services Tabular Model A brief end to end By Kirsten burnett - ENTELECT

Our Journey through Tabular Models Tabular in a nut shell The Business Intelligence Semantic Model A brief history of Tabular Tabular benefits over Power Pivot Tabular vs Multidimensional and choosing between the two xVelocity In Memory Analytics Engine DirectQuery Managing a Tabular Model (Processing and Partitioning) Automating with ASSL/TMSL and AMO Creating and Deploying a Tabular model (Demo) Tabular and xVelocity in the market and the future for SQL Server Analysis Services Image from: https://sqldusty.com/2013/08/06/performance-tuning-of-tabular-models-in-ssas-2012-whitepaper-now-available/

Tabular in a nutshell Tabular SSAS Database Measures, KPI’s, Hierarchies Perspectives & Translations DAX Partitioning & Processing DirectQuery ASSL, AMO, TMSL, DMV’s xVelocity Role and Row Security Tabular Relationships, Tables, Columns, Rows Various Data Sources SSAS Database SSDT & SSMS http://www.nutritionbynature.com.au/blog/nuts-seeds-are-for-birds-and-squirrels-not-humans

Business Intelligence Semantic Model BI APPLICATIONS SSRS EXCEL 3RD PARTY APPS MULTIDIMENSIONAL TABULAR DATA MODEL MDX DAX BUSINESS LOGIC ROLAP MOLAP DIRECT QUERY DATA ACCESS xVelocity LOB CLOUD DATA SOURCES RDBMS FILE

A Brief History of Tabular 2 Internal papers in 2006 by Amir Netz about a BI Sandbox and In-memory technology Power Pivot for Excel was first released as an add-in in May 2010 as part of the SQL Server 2008 R2 release SQL Server 2012 saw the introduction of a completely new data model, SQL Server Analysis Services Tabular model   Microsoft’s entry into the Self-Service BI market. https://news.microsoft.com/stories/people/amir-netz.html

TABULAR BENEFITS OVER POWER PIVOT Power Pivot vs Tabular SCALABILITY SECURABILITY MANAGEABILITY TABULAR BENEFITS OVER POWER PIVOT PowerPivot has essentially become the desktop version of Analysis Services, but remains an excellent prototyping tool for Tabular.

Tabular vs Multidimensional EASIER, QUICKER, NEWER TECHNOLOGY EXCELLENT QUERY PERFORMANCE VARIED DATA SOURCES CALCULATED COLUMNS/TABLES MULTI-DIMENTIONAL PROVEN, STABLE, OLDER TECHNOLOGY VERY LARGE DATABASES (TERABYTES) MORE GRANULAR SECURITY WRITEBACK, AGGREGATIONS, ACTIONS, NAMED SETS

xVelocity In Memory Analytics Engine Compression algorithm Compression techniques Less RAM usage and decreased costs Increased performance as less data to scan Dictionary/Hash Encoding Value Encoding Run Length Encoding Originally called the VertiPaq engine, xVelocity is an in-memory columnar database where all of the data in a model lives in RAM. Columnstore indexes use this same compression technology.

xVelocity Dictionary Encoding A dictionary of unique values in a column is built and then data values are replaced with the values of the dictionary index. Regardless of datatype the same performance gains are achieved. SIZE Small Extra Large Medium Extra Small Large SIZE ID 1 2 3 4 ID SIZE Small 1 Extra Large 2 Medium 3 Extra Small 4 Large

xVelocity Value Encoding xVelocity looks for mathematical relationships between the values in a column, then uses the value itself, with or without a mathematical operation applied, to decrease the memory usage. PRODUCT_KEY 100023 100024 100025 100026 100027 100028 100029 Provides better performance as computations can operate directly on the data without a separate lookup step as with dictionary encoding. Product_key - 100000 23 24 25 26 27 28 29 Less storage used for smaller value by subtracting 100000. Adds the value back in when required.

xVelocity Run Length Encoding Complementary compression technique that tries to reduce the size of a dataset by avoiding repeated values. quarter Q1 ….. Q2 … Values are replaced with a slightly more complex structure that contains the value only once, with the number of contiguous rows having the same value. Repeated 210 times QUARTER START COUNT Q1 1 210 Q2 211 260 Repeated 260 times Example from Ferrari, Alberto; Russo, Marco. The Definitive Guide to DAX: Business intelligence with Microsoft Excel, SQL Server Analysis Services, and Power BI (Business Skills) (p. 411). Pearson Education. Kindle Edition.

xVelocity Re-encoding SAMPLE Read a sample of the data CHOOSE Choose a compression technique based on sample COMPRESS Compress the data using chosen technique RE-ENCODE Determine a new compression technique, if required Provide a good sample of data when doing an initial partition process to keep re-encoding to a minimum.

xVelocity Relationships To improve performance, relationships between tables are stored in memory as pairs of ID’s and row numbers. Sale amount Product id 10 2 12 3 34 7 Product id product 2 Nutty Kitty 3 Chocolate Bunny 7 Nutty Chocolate Bunny ROW 1 ROW 2 Given the ID in one table, it can immediately find the corresponding rows in the joined table. ROW 3   Sales[product id] Product[row number] 2 1 3 7

xVelocity Design Considerations A column can have both Run Length Encoding AND Dictionary OR Value encoding. 1. CARDINALITY Number of unique values in the column, lower is better 2. DISTRIBUTION Repeated values have higher compression rates than those columns with frequently changing values DESIGN CONSIDERATIONS 3. TABLE SIZE Number of rows in the table 4. DATA TYPE Affects the dictionary size

DirectQuery REAL TIME REPORTING VERY LARGE DATABASES DAX CONVERTED TO SQL DIRECTQUERY SEMANTIC LAYER SINGLE RELATIONAL SOURCE DIFFERENCES IN DAX FORMULAS AND SUPPORT

Partitioning Partitions divide a table into logical parts and provides several benefits. DECREASED PROCESSING TIME EASILY REMOVE OR UPDATE DATA CONSOLIDATE DATA FROM VARIOUS SOURCES amount date 220 2015-12-31 …. ….. 468 2015-01-01 1290 2014-12-31 … 34 2014-01-01 3 2013-12-31 98 2013-01-01 Partition for 2015 Partition for 2014 Partition for 2013

Processing – Full, Add, Recalc, Default REFRESHES ALL STRUCTURES AVAILABLE AT DATABASE, TABLE AND PARTITION LEVEL MODEL IS QUERYABLE THROUGHOUT ENTIRE COPY OF DATABASE STORED IN MEMORY ADD UPDATES PARTITION WITH NEW DATA AVAILABLE AT PARTITION LEVEL MODEL IS QUERYABLE THROUGHOUT NO AUTOMATIC DETECTION OF DUPLICATES RECALC UPDATES AND RECALCULATES AVAILABLE AT DATABASE LEVEL NEEDS TO BE MANUALLY ISSUED AFTER PROCESS DATA AFFECTS HIERARCHIES, CALCS AND RELATIONSHIPS DEFAULT MAKES OBJECT QUERYABLE AVAILABLE AT DATABASE, TABLE AND PARITION LEVEL ONLY OPERATIONS NECCESARY TO MAKE OBJECT QUERYABLE CAN RUN PROCESS DATA AND OR PROCESS RECALC

Processing - Data, Clear, Defrag PROCESS DATA LOADS DATA ONLY AVAILABLE AT TABLE AND PARTITION LEVEL MODEL IS NOT QUERYABLE AFTERWARDS RUN PROCESS DEFAULT OR PROCESS RECALC AFTER PROCESS CLEAR DROPS ALL DATA AVAILABLE AT DATABASE, TABLE AND PARTITION LEVEL MODEL IS NOT QUERYABLE AFTERWARDS NEED TO LOAD DATA AND RECALC PROCESS DEFRAG REBUILDS TABLE DICTIONARY AVAILABLE AT TABLE LEVEL USED WHEN PARTITIONS FREQUENTLY CHANGE USELESS AFTER PROCESS FULL OR PROCESS DATA SQL Server 2016 Analysis Services (SSAS) includes parallel processing for tables with two or more partitions.

Automating with ASSL, TMSL & AMO Analysis Services Scripting Language (ASSL) is an extension of the XMLA Open Standard and can be used to automate admin tasks. Tabular Model Scripting Language (TMSL) – JSON. New in SQL Server 2016 with relevant Tabular language elements. Analysis Management Objects (AMO) – Managed Interface used to run XMLA, MDX etc. ASSL TMSL AMO

Basic Tabular Demo Image from: http://quoteaddicts.com/topic/good-luck-funny/

? The Future MERGED TECHNOLOGIES TABULAR REPLACES MULTI-DIMENSIONAL HYBRID SOLUTIONS From version 2008 R2 onwards there have been no major updates added to the multidimensional model – this could be because it is a stable technology. But they are adding more and more features to the tabular model. For the 2016 release version Microsoft added loads of features to Tabular. Tabular model is also closely related to Excel and the xVelocity technology is being used by PowerPivot, Analysis Services Tabular, SQL Server Columnstore and PowerBI, Microsoft is clearly investing more in tabular which makes them more competitive against other in-memory technologies like SAP HANA, Tableau, QlikView etc. Even newer BI players like Pyramid with their offering of BI Office are using Tabular as a back-end. The only data source for Power View is a Tabular database or PowerPivot workbook. I’ve read a lot of articles about where analysis services is going, and opinions range from, ssas eventually merging multidimensional and tabular into a single product to multidimensional being dropped altogether. Personally I think there is space for both technologies, and we could expect hybrid solutions. There is too much investment in Multidimensional, and it is a stable technology, so I don’t see it going anywhere for some time. One thing seems fairly likely, that SQL Server Analysis Service will be offered as a service and will be Azure based at some point, and Tabular will be the choice. Also, as soon as Tabular is up to date with current expectation and it can offer full functionality, I can only see multidimensional being used in very specific circumstances, and not widely.   ANALYSIS SERVICES AS A SERVICE (AZURE)

References http://sqlblog.com/ Ferrari, Alberto; Russo, Marco; Webb, Chris. Microsoft SQL Server 2012 Analysis Services: The BISM Tabular Model (Developer Reference) Pearson Education. Kindle Edition. https://msdn.microsoft.com/en-us/library/hh212945.aspx https://www.mssqltips.com/sqlservertip/2818/understanding-the-sql-server-2012-bi-semantic- model-bism/ Credit: Business Intelligence in Microsoft SharePoint 2010 By Norm Warren, Mariano Neto, John Campbell, Stacia Misner Ferrari, Alberto; Russo, Marco. The Definitive Guide to DAX: Business intelligence with Microsoft Excel, SQL Server Analysis Services, and Power BI (Business Skills) (p. 411). Pearson Education. Kindle Edition.

Thank you to our sponsors