Data modelling for Power BI using brand new Analysis Services Features Phil Seamark

Phil Seamark - MVP Radacad Wellington Power BI User Group Microsoft MVP Author Active in Power BI Community @PhilSeamark phil@radacad.com

Session Agenda Incremental Refresh In memory Aggregations The problem The solution The benefits In memory Aggregations Questions and Giveaways

Incremental Updates

Incremental Refresh The Problem Large data sets refresh IN FULL each time Refresh duplication Compounded when refreshing multiple times per day

Incremental Refresh The Solution Unlock and use SSAS Partitions Auto scripted in the Webservice Elegant solution using Merge partitions (Merge , Drop & Recovery) The table to be incrementally refreshed needs to have a Date column Some manual configuration required (see demo)

Demo : Incremental Update

Incremental Refresh The Benefits (and notes) Available now!! Speed of refreshing from data source. The larger the table, the bigger the benefit Less load on data source Premium is required for now – but IS coming to shared XMLA endpoints later this year

In-memory Aggregations

In-memory Aggregations The Problem I have a very large data set and want to user Power BI to analyse and report Queries against very large tables are inefficient Slow reports even with smaller data sets

In-memory Aggregations The Solution Aggregation tables (Summary tables) Composite models Mapping Columns Table Precedence Profiler – new events Storage modes (Import, Direct Query and DUAL) ……

Import vs. DirectQuery Big Models Query Performance Data Freshness 4/24/2019 5:23 AM Import vs. DirectQuery Big Models Query Performance Data Freshness DirectQuery Import © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Import vs. DirectQuery Big Models Query Performance Data Freshness 4/24/2019 5:23 AM Import vs. DirectQuery Big Models Query Performance Data Freshness DirectQuery Import AT TABLE LEVEL!!! © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Import DirectQuery Geography Geography Customer Customer Sales Sales 4/24/2019 5:23 AM DirectQuery Import Geography Geography Customer Customer Sales Sales Date Date Product Product Reseller Sales Reseller Sales Employee Employee © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

& Import DirectQuery IMPORT Geography Geography Direct Query Customer 4/24/2019 5:23 AM DirectQuery & Import IMPORT Geography Geography Direct Query Customer Customer Sales Sales Date Date Product Product Reseller Sales Reseller Sales Employee Employee © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

& Import DirectQuery IMPORT Geography Geography Direct Query Customer 4/24/2019 5:23 AM DirectQuery & Import IMPORT Geography Geography Direct Query Customer Customer Sales Sales Agg Date Date Product Product Reseller Sales Reseller Sales Employee Employee © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Aggregations IMPORT Geography Geography Direct Query Sales Agg 4/24/2019 5:23 AM Aggregations IMPORT Geography Geography Direct Query Sales Agg Customer Customer Sales Date Date Product Product Reseller Sales Reseller Sales Employee Employee © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Aggregations Hits in-memory cache SummarizeColumns( Date[Year], 4/24/2019 5:23 AM Aggregations IMPORT Geography Geography Direct Query Sales Agg Customer Customer SummarizeColumns( Date[Year], Geography[City], "Sales", Sum(Sales[Amount]) ) Sales Date Date Product Product Reseller Sales Reseller Sales Employee Employee Hits in-memory cache © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Aggregations DirectQuery SummarizeColumns( Date[Year], Customer[Name], IMPORT Geography Geography Direct Query Sales Agg Customer Customer SummarizeColumns( Date[Year], Customer[Name], "Sales", Sum(Sales[Amount]) ) Sales Date Date Product Product Reseller Sales Reseller Sales Employee Employee DirectQuery © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Aggregations “Many side” “One side” Dual Import Import or Dual DQ 4/24/2019 5:23 AM Aggregations IMPORT Geography Geography DIRECT QUERY Sales Agg Storage Mode = Dual DUAL Customer Customer Sales 1-to-many relationship rules Date Date Product Product “Many side” “One side” Dual Import Import or Dual DQ DQ or Dual Reseller Sales Reseller Sales Employee Employee (both sides are from a single source) © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Aggregations “Many side” “One side” Dual Import Import or Dual DQ 4/24/2019 5:23 AM Aggregations IMPORT Invalid Geography DIRECT QUERY Sales Agg DUAL Customer Customer Sales 1-to-many relationship rules Date Date Product Product “Many side” “One side” Dual Import Import or Dual DQ DQ or Dual Reseller Sales Reseller Sales Employee Employee © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Aggregations “Many side” “One side” Dual Import Import or Dual DQ 4/24/2019 5:23 AM Aggregations IMPORT Invalid Geography Geography DIRECT QUERY Sales Agg DUAL Customer Customer Sales 1-to-many relationship rules Date Date Product Product “Many side” “One side” Dual Import Import or Dual DQ DQ or Dual Reseller Sales Reseller Sales Employee Employee © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

In-memory Aggregations DB Table Storage Mode Dimension Rows Size Agg_A Import Year + Measures 14 100Kb Agg_B Date + Measures 5,110 32MB Agg_C Date, hour, Payment + Measures 585,774 39MB Agg_D Import or Direct Query Date, hour, Payment, Geography 257,423,110 8.8GB Trip Direct Query ALL 2.3 Billion ½ TB

Demo : In-memory Aggregations

In-memory Aggregations The Benefits (and notes) Speed, speed and more speed Can use incremental refresh with AGG tables DQ tables need to be from the same source to take full advantage for DUAL Not a one click solution – still requires thought and care Not for every Power BI scenario If you have a slow report…..

Questions and Prize Draw

Don’t forget to join your local PUG to enjoy year-round networking and learning.