What query folding means to self-service BI projects 1/18/2019 2:26 AM What query folding means to self-service BI projects Ken Puls, FCPA, FCMA, Microsoft MVP President & CTO, Excelguru Consulting © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

The Impact of Query Folding Microsoft 2016 1/18/2019 2:26 AM The Impact of Query Folding Client’s architecture ~70 million rows stored in SQL (adding ~1 million rows per day) Self service BI created in Excel and Power BI The issue Retrieval and reshaping took hours to execute BI solution risked getting abandoned By leveraging Query Folding Retrieval and reshaping now takes minutes The solution can actually be used © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Would you rather? Connect to the data source and Retrieve 1 million rows to the Client Then, in the client application Remove all rows except Dept 500 Remove all but 3 columns Group rows by Month End Land the 2000 rows to a table Connect to the data source and Select only rows for Dept 500 Select only 3 columns Group the rows by Month End Retrieve 2000 rows to the client Then, in the client application Land the 2000 rows to a table Summary: Pulls ALL data from the server Processes with client CPU and RAM Summary: Pushes processing to the server Pulls as little to client as possible

What is Query Folding? User interface driven Example: Query steps are “folded” into consolidated SQL statements Example: Connect to table Remove all columns except MonthEnd, Dept & Amount Filter to only Dept 500 Group by MonthEnd SELECT MonthEnd, Dept, Amount FROM tblTransactions WHERE Dept=‘500’ Group By MonthEnd SELECT MonthEnd, Dept, Amount FROM tblTransactions WHERE Dept=‘500’ SELECT * FROM tblTransactions SELECT MonthEnd, Dept, Amount FROM tblTransactions

What Sources Support Query Folding Supported Unavailable Databases Any database with a connector ODBC* Files Excel, CSV, Text, etc.. Access databases Non-supported databases OLE DB ODBC* DON’T GET BURNED! Only use ODBC if a specific connector is not available!

Key Goals for Self Service Users Minimize the data being retrieved Removing columns Filtering rows Removing duplicates Push “heavy” operations to the server Grouping Merging tables (Join operations) Pivoting or Unpivoting operations

Breaking Query Folding Using data sources that don’t support query folding Starting from a custom SQL statement Using complex ‘M’ functions Buffering tables or lists Leveraging custom (user defined) functions

How to Tell if Query Folding is Active? Right click a step in the query… Query Folding is Active View Native Query is available Query Folding Broken View Native Query greyed out

Query Folding Demo Ken Puls, FCPA, FCMA, Microsoft MVP © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Please evaluate this session Your feedback is important to us! 1/18/2019 2:26 AM Please evaluate this session Your feedback is important to us! The slide will be replaced onsite through Silver Fox Productions with an updated QR code. This slide is required. Do NOT delete or alter the slide. From your PC or Tablet visit MyIgnite at http://myignite.microsoft.com From your phone download and use the Ignite Mobile App by scanning the QR code above or visiting https://aka.ms/ignite.mobileapp © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

1/18/2019 2:26 AM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.