6/12/2018 2:19 PM BRK3245 DirectQuery in Analysis Services: best practices, performance, and use cases Marco Russo SQLBI © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Agenda DirectQuery in SSAS 2016/2017 (and Azure AS) What’s changed since 2012/2014 How to create a data model in DirectQuery mode Querying data models in DirectQuery mode What happens when you use DirectQuery Differences between clients: Excel and Power BI Security models Optimizing DirectQuery Best practices for DAX, data modeling, and data sources Leveraging columnstore indexes Final considerations When to use DirectQuery, Q & A

Whitepaper DirectQuery in Analysis Services 2016 https://www.sqlbi.com/articles/directquery-in-analysis-services-2016/ Include demos and further details about the content of this presentation All the considerations are still valid in Analysis Services 2017 and Azure Analysis Services as of September 2017

Storage engines in SSAS Tabular In-Memory (VertiPaq) Create a copy of data in memory Single access to data source Queries run on a “snapshot” of data DirectQuery Queries sent to data source, SSAS Tabular does not “cache” data No processing time Real-time queries Higher workload on data source

DirectQuery vs In-Memory (VertiPaq)

VertiPaq pros/cons Pros Cons Data compressed in memory, faster queries Different data sources for one model Single scan of source tables Cons Processing takes time (latency of updates) Processing requires CPU time Limited querying capacity during process All data have to fit in memory Copy data from data source to SSAS Network could be one bottleneck

DirectQuery pros/cons No processing time No limits of data size related to memory No data duplication No latency in updates Cons Data slower to query Specific optimizations on data source Only one data source for one model Larger workload on data source for queries

DirectQuery evolution SSAS 2012/2014 Support only for SQL Server, no other RDBMS Support only for DAX queries, no MDX One DAX query equal to one SQL query No cache in SSAS No formula engine in SSAS Limited support to DAX language No security roles SSAS 2016/2017 – Azure AS Support for several RDBMS (also non-MS) Support for DAX and MDX queries Multiple SQL queries for one MDX/DAX query Partial cache in SSAS Formula engine support in SSAS Full DAX language in query/measures Security roles supported

Supported data sources Microsoft Microsoft SQL Server 2008 and later Microsoft Azure SQL Database Microsoft Azure SQL Data Warehouse Microsoft Azure SQL Analytics Platform System (APS) Oracle Oracle 9i and later Teradata Teradata V2R6 and later List of specific providers supported for DirectQuery: https://msdn.microsoft.com/en-us/library/hh230898.aspx

Demo Create a Tabular model in DirectQuery mode

Set DirectQuery Mode property

Set DirectQuery Mode property

Use DirectQuery connection mode

Developing for DirectQuery Using default partitions Switch between On/Off for DirectQuery Mode Features unlimited when DirectQuery = Off You cannot switch to On if the model uses unsupported features Using sample data Create one or more partitions with sample data Default partition is DirectQuery only – no preview in SSDT Switch to DirectQuery after deployment (SSMS) You can change the DirectQuery mode after deployment Use Database Properties in SSMS, or XMLA, or PowerShell

Demo Sample data in SSDT Switch DirectQuery in SSMS

Sample partition in SSDT

Switch DirectQuery in SSMS

Limitations in Tabular models for DirectQuery Data modeling No calculated tables DAX All functions supported in measures and queries Restrictions in calculated columns and row-level security (RLS) filters Detailed list of functions available at https://msdn.microsoft.com/en-us/library/mt723603.aspx MDX No relative object names No session-scope MDX statements No tuple with member from different levels in subselects No used-defined hierarchies Detailed list of MDX limitations available at https://msdn.microsoft.com/en-us/library/hh230898.aspx

What happens in DirectQuery mode Optimizing DirectQuery  optimizing RDBMS Understand how SQL queries are generated by SSAS Tabular Create indexes to improve performance Design data model and DAX expressions to improve efficiency of SQL code Tools SQL Profiler DAX Studio

Demo Analyze queries using SQL Profiler and DAX Studio

Differences between clients Excel PivotTable Generates MDX queries No cache at the SSAS layer Repeating the same drilldown repeats the SQL queries Multiple subqueries repeat same queries at different granularities PivotTable in Excel Power BI Multiple DAX queries in the same request Strong caching at Power BI application level Repeating the same drilldown reuse previous results Explicit Refresh required to empty client cache enforcing SQL queries to read data More efficient DAX queries Reduced number of SQL queries sent for the same report

Tuning query limits in DirectQuery Limit results in SQL queries Default: 1 million rows Avoid long running queries and memory issues Only affects queries returning large intermediate results to SSAS (it depends on DAX expressions) Can be configured through MaxIntermediateRowsetSize setting in msmdsrv.ini <ConfigurationSettings> . . . <DAX> <DQ> <MaxIntermediateRowsetSize>1000000 </MaxIntermediateRowsetSize> </DQ> </DAX> . . .

Security Role based security Impersonation with DirectQuery Same rules defined by DAX filter expressions, one for each table Dynamic security using windows username DAX filters translated in corresponding SQL filters (WHERE conditions) Performance depends on data source optimization Every SQL query repeats the security filter Impersonation with DirectQuery SQL queries sent to RDBMS using the credentials of user connected to SSAS SQL Server 2016 has its own row-level security RDBMS must support Windows Integrated authentication Can replace role based security in SSAS, but you can also use both at the same time

Demo Role based security Impersonation with DirectQuery

Optimizing DirectQuery Optimize DAX Use optimized DAX functions in DAX measures and queries Reduce complex calculations in aggregations (materialize calculated columns in RDBMS) Optimize data model Enable referential integrity in relationships It will generate INNER JOIN instead of LEFT JOIN Careful consider bidirectional filters Optimize RDBMS Columnstore index Indexed views

Demo Optimize data model and RDBMS

Scenarios for DirectQuery Small database updated often DirectQuery removes latency caused by data refresh Data is always up-to-date Large database that cannot fit in memory Removes processing time Does not require large hardware for Analysis Services (reduced memory requirements) VertiPaq is usually faster than DirectQuery, but you need SSAS memory to store the entire database

Recap and call to action DirectQuery in SSAS 2016/2017 and Azure AS Huge improvements over 2012/2014 Larger number of use cases where it can be used (now it also supports Excel as a client) Design models for DirectQuery mode Consolidate calculated columns on RDBMS for performance Use referential integrity assumption in relationships Choose between different security models Optimizing DirectQuery You have to optimize the RDBMS Columnstore indexes are a primary tool for SQL Server Use cases for DirectQuery Small database updated often Large database that cannot fit in memory

Whitepaper DirectQuery in Analysis Services 2016 https://www.sqlbi.com/articles/directquery-in-analysis-services-2016/

Please evaluate this session Tech Ready 15 6/12/2018 Please evaluate this session From your Please expand notes window at bottom of slide and read. Then Delete this text box. PC or tablet: visit MyIgnite https://myignite.microsoft.com/evaluations Phone: download and use the Microsoft Ignite mobile app https://aka.ms/ignite.mobileapp Your input is important! © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

6/12/2018 2:19 PM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.