Welcome! Power BI User Group (PUG) Sydney

Creating Enterprise Grade BI Models with Azure Analysis Services Darren Gosbell Data Platform MVP As you have seen in the previous two sessions, Analysis Services enables consistent data across reports and users of Power BI. This session will focus on optimizing configurations, management automation, governance, sizing considerations and best practices. Various modeling enhancements will be included such as Detail Rows allowing users to easily see transactional records, deployment and application-lifecycle management (ALM) features to bridge the gap between self-service and corporate BI. Security drill down such as logging where and how your data is stored, and look at how to leverage additional Azure services to make your information even more secure. @darrengosbell

Meet Your Presenter DARREN GOSBELL Leader – SQL Server User Group – Melbourne Data Platform MVP since 2006 SSAS Maestro Contributor to open source BI projects DAX Studio BIDS Helper AS Stored Procedures

8/1/2018 11:03 AM 1200 Compatibility Level Azure Analysis Services / SQL Server Analysis Services 2016 Performance and scalability Parallel partition processing NUMA awareness and memory allocator (SP1) Super DAX Developer Tools Tabular Model Explorer Integrated workspace server DAX formula editing Modeling and analytics Bi-directional cross filters Calculated tables Display folders Translations Over 50 new DAX functions DirectQuery enhancements Manageability Tabular Object Model (TOM) Tabular Model Scripting Language (TMSL) Modeling and analytics Bi-directional cross filters Many-to-many dimension scenarios without need to write complex DAX formulas For whole table, not just a measure Various use cases such as account balance, currency conversion, distinct count of attributes Calculated tables Dynamically generate a table based on a DAX formula. Various use cases including role-playing dimensions Some performance improvements available Display folders Present model elements and measures by business function in Pivot Tables and in Power BI Translations DirectQuery enhancements Data is up to date with no management of loading the in-memory cache Big data sets that don’t fit into memory New data sources: APS / SQL DW, Oracle, Teradata Support for MDX queries (Pivot Tables) Improved query generation resulting in faster performance Row-level security defined with DAX filters Over 50 new DAX functions DATEDIFF, SELECTCOLUMNS, SUMMARIZECOLUMNS, CROSSFILTER Manageability Tabular Object Model (TOM) Tables, columns, relationships – not cubes, measure groups, dimensions Tabular Model Scripting Language (TMSL) JSON representation of TOM – with commands for manageability such as create/alter database, backup, restore, attach, detach Performance and scalability NUMA awareness and memory allocator The in-memory engine in SP1 maintains a separate job queue on each NUMA node The Intel TBB-based scalable allocator that provides separate memory pools for every core. As the number of cores increases, the system can scale almost linearly Parallel partition processing Super DAX Strict evaluation of IF/SWITCH, Variables Some cases just restoring a 2014 model to 2016 provides significant DAX performance benefits © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

8/1/2018 11:03 AM 1400 Compatibility Level Azure Analysis Services / SQL Server Analysis Services 2017 Data connectivity Rich set of data sources Data transformations and mashups with Power Query Formula Language Modeling and analytics Detail Rows Object level security Enhanced support for ragged hierarchies Developer tools SSDT for VS 2017 DAX Editor for SSDT and SSMS © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Demo: Modelling & Analytics Showing Object Level Security Ragged Hierarchies DetailRow Expressions

Ragged Hierarchies Default View Desired View

Ragged Hierarchies pre-1400 Nodes[HierarchyPath] =PATH ( Nodes[NodeKey], Nodes[ParentKey] ) Nodes[Level1] = LOOKUPVALUE ( Nodes[Name], Nodes[NodeKey], PATHITEM ( Nodes[HierarchyPath], 1, INTEGER ) ) Nodes[Level2] = IF ( Nodes[HierarchyDepth] >= 2, LOOKUPVALUE ( Nodes[Name], Nodes[NodeKey], PATHITEM ( Nodes[HierarchyPath], 2, INTEGER ) ), Nodes[Level1] ) Nodes[Level3] = IF ( Nodes[HierarchyDepth] >= 3, LOOKUPVALUE ( Nodes[Name], Nodes[NodeKey], PATHITEM ( Nodes[HierarchyPath], 3, INTEGER ) ), Nodes[Level2] ) Nodes[IsLeaf]=CALCULATE ( COUNTROWS (Nodes), ALL (Nodes), Nodes[ParentKey] = EARLIER ( Nodes[NodeKey] ) ) = 0 Nodes[HierarchyDepth] =PATHLENGTH( Nodes[HierarchyPath] ) BrowseDepth:=ISFILTERED ( Nodes[Level1] ) + ISFILTERED ( Nodes[Level2] ) + ISFILTERED ( Nodes[Level3] ) MaxNodeDepth:=MAX ( Nodes[HierarchyDepth] ) Sales Amount:=IF ( [BrowseDepth] > [MaxNodeDepth] + 1, BLANK (), IF ( [BrowseDepth] = [MaxNodeDepth] + 1, AND ( VALUES ( Nodes[IsLeaf] ) = FALSE, SUM ( Transactions[Amount] ) <> 0 ), SUM ( Transactions[Amount] ), BLANK () SUM ( Transactions[Amount] ) ) Source: http://www.daxpatterns.com/parent-child-hierarchies/

Self-Service & Corporate BI … 8/1/2018 11:03 AM Self-Service & Corporate BI … > “Bimodal” BI Self-service BI is characterized by having a large number of small models. Corporate BI is characterized by having a small number of large models. Kurt Schlegel et al. (2016) observes the following. Create a Centralized and Decentralized Organizational Model for Business Intelligence, Page 6. Retrieved from Gartner database; “Gartner Foundational”. Local teams are being empowered to create and innovate. The centralized team identifies the most successful work being done at a local level, and provides a platform to share and promote this work globally. Self-service & corporate BI IT-owned, enterprise BI has the following characteristics compared to self-service BI. Enterprise BI is characterized by having a small number of large models. This promotes reusability, consistent decisions based on corporate metrics, and efficiencies around the management of data. Self-service BI is characterized by having a large number of small models. This is a result of enabling agility and freedom for analysts to uncover new insights and data sets. The reality is most organizations need to strike a balance between the two camps. The challenge lies in how the two camps can work together in harmony. © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Azure Analysis Services 8/1/2018 11:03 AM Demo Architecture Cloud data sources Azure Analysis Services Visualization & Insights SQL Data Warehouse Power BI Blob Storage Cloud processing Azure Automation TOM/AMO Partition Creation Authoring & Development Self-Service Authoring Power BI Desktop Power BI Desktop Visual Studio Visual Studio © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Demo: bridge the gap between self-service & corporate BI Showing Import from PBIX Merging Self-Serve with Corporate Incremental Deployment

Demo Summary Fine-grain partition management through API BI schema-compare for application lifecycle management (ALM) Power Query data integration Deployment across environments

Corporate BI Features Automation Partition management to refresh big datasets Application lifecycle management (ALM) Deployment across environments Enhanced security Enhanced modeling & analytics Enhanced monitoring

Use Case: Azure Intelligence Team

Azure Intelligence Team Mission Statement: Azure Intelligence Platform (AIP) builds a comprehensive data platform for C+E focused on managing metrics for demand (our customers), monetization (usage and billing), and market intelligence. Main Customers: Sales, Marketing, Finance, Compensation © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

By the numbers 5 TB of data per day 1 PB of data 900 pipelines 70 nodes 1200 cores

8/1/2018 11:03 AM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

8/1/2018 11:03 AM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

8/1/2018 11:03 AM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

8/1/2018 11:03 AM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

More information AS processing whitepaper: http://aka.ms/asprocessing Azure Analysis Services: https://aka.ms/AzureAS Azure BI Blog: https://aka.ms/azurebiblog

Thank you for Attending! Don’t forget to join your local PUG to enjoy year- round networking and learning. www.pbiusergroup.com/sydney www.pbiusergroup.com/melbourne www.pbiusergroup.com/adelaide www.pbiusergroup.com/brisbane