1. Data Management Gateway Deep Dive

2. Thanks to our Main Sponsors:

3. Say Thank you to Volunteers:  They spend their FREE time to give you this event.  Because they are crazy.  Because they want YOU to learn from the BEST IN THE WORLD.

4. Jean-Pierre Riehl Practice Manager Data & BI @djeepy1 - http://blog.djeepy1.net Who am I ? Pure-Player Microsoft  Self-Service BI experts  « Agile BI » projects  Data Strategy  Hybrid-BI MVP SQL Server French PASS Chapter Leader

5. La communauté Data Microsoft Webcasts, Conférences, Afterworks.Pro Save the Date September, 5th Campus SUPINFO Tour Montparnasse, Paris Self-promotion

6. An hybrid story…

7. Microsoft Cloud-First strategy…

8. But your data remains On-Prem…

9. Here it comes DMG

10. DMG-aware tools Power BI for O365 PowerBI.com Azure Data Factory

11. Power BI Excel Power Query Power Pivot Power View Power Map Power BI Tenant SQL CloudOn-Prem Oracle … Power BI Admin Center Data Catalog Q&A Power BI Sites Data Refresh Indexing / Search OData Access

12. PowerBI.com Live Power BI reports & dashboards Live Query Scheduled Refresh Cloud On-premises

13. Azure Data Factory

14. Hybrid features  Data Refresh  Data Movement  Live Query  Indexing Sources  OData Access (Data Virtualization)

15. One DMG to rule them all DMZ SQL CloudOn-Prem Oracle … Power BI for O365 Data Management Gateway OnPrem Data Sources PowerBI.com Azure Data Factory Excel

16. DMG DETAILS

17. Versions of DMG  There are multiple versions of DMG Data Management Gateway Power BI ADF 1.4.5549.124/3/2015 Analysis Services ConnectorPower BI v21.4.5576.217/4/2015 Breaking News: 1.4.5612.1 (released 26/5)

18. What’s that  A Windows Service  DIAHostService  A configuration application  Managed by the cloud  Power BI Admin Center  Azure Portal (new version)

20. demo FIRST LOOK AT DMG

21. Data Sources Supported SourceAuthentification SQL Server / Oracle Sybase, Teradata, DB2, MySQL Windows, Basic Files (CSV, XML, Text, Excel, Access) Folders Windows SharePoint List (Online) SharePoint List (On-prem) Anonymous Anonymous, Windows OData Feed Anonymous, Windows, Basic Key (Data Market) Azure MarketplaceKey (Data Market) Azure HDInsight Azure Blob Storage Azure Table Storage Key (Azure Account) SQL Azure PostgreSQL Basic Teradata, Sybase, DB2, MySQLWindows, Basic SSAS Tabular (Live Query) Windows +SSIS Data Feed

22. Data Sources supported  Details on data sources supported  https://support.office.com/en-us/article/Supported-Data-Sources- and-Data-Types https://support.office.com/en-us/article/Supported-Data-Sources- and-Data-Types  Prequisites  Beware of versions  Beware of types  Drivers installed  https://support.office.com/en-us/article/Data-source-prerequisites- Power-Query https://support.office.com/en-us/article/Data-source-prerequisites- Power-Query

23. HOW DOES IT WORK

24. CloudOn-Prem Data Refresh explained Power BI OnPrem Sources DMG Azure Service Bus 0 Register DMG Register Data Sources 1 Ask for Data Refresh Check permissions Find Gateway 4 Execute Query 5 Get Data 6 Compress, Chunk Queue request 2 3 Read request Gateway Cloud Service 7 Update Model

25. CloudOn-Prem OData access explained Power BI OnPrem Sources DMG 0 Index Sources 2 Resolve Data Source Redirect to local Gateway 4 Execute Query 3 Request Data 1 Search Source Excel Power Query Hybrid Proxy

26. demo DMG IN ACTION

27. Data Source Discovery  Data Sources are declared on « Admin Center »  They are associated to a Gateway  Gateway Cloud Service resolves Data Source and find right Gateway  Resolution = Provider + Server + Catalog  Warning: Case sensitive

28. demo DATA SOURCE DISCOVERY

29. SECURITY

30. Excel Power Query Power BI DMG Data sources SQL (1433) FileShare Etc. +DNS resolution Azure Service Bus (outbound) 9350-9354 / 5671,5672 Power BI Admin Center (outbound) 80/443 Direct HTTP (inbound) HTTPS (8050) HTTP (8051) HTTPS (443) Cloud On-Prem Network Security Credential Manager

31. Identity Management 2 flavors :  Impersonation by DMG  Impersonation by Data Source How is query authentified on Data Source ?

32. Impersonation by DMG  Connection identity is impersonated by DMG  Peer-to-peer connection  Credentials are stored by DMG for each source  Credentials can be stored encrypted locally  Credentials can be stored encrypted on Cloud  Credentials are set up locally  Credential Manager is a Click Once application  Credential Manager discuss with DMG

33. demo CREDENTIAL MANAGER

34. Impersonation by Data Source  Used by Live Query  Keep identity of original user  Use the EffectiveUsername method  DMG authenticates to SSAS  DMG passes EffectiveUsername property  SSAS executes the query as if it was original user  DMG must be SSAS Admin

35. demo EffectiveUsername

36. HIGH AVAILABILITY & DISASTER RECOVERY

37. As easy as scale-out DMGs Cloud On-Prem Power BI … Instance 1 Instance 2 Server A Server B

38. HA & DR  Gateways and Data sources are managed by Cloud  Credentials are encrypted by MDG  Use of Certificate, self-generated or yours  Need Certificate when registering a new instance  Warning when stored localy  Load Balancing

39. demo HA & DR

40. Going further  Express Route  Mashup Container (aka Power Query)  Monitoring & Logging

41. Resources  Getting Started : https://support.office.com/en-us/article/Power-BI-for-Office-365-Admin-Center-Help- 5e391ecb-500c-47a3-bd0f-a6173b541044https://support.office.com/en-us/article/Power-BI-for-Office-365-Admin-Center-Help- 5e391ecb-500c-47a3-bd0f-a6173b541044  Data Factory Reference : https://msdn.microsoft.com/en-us/library/azure/dn834987.aspxhttps://msdn.microsoft.com/en-us/library/azure/dn834987.aspx  Supported Data sources : https://support.office.com/en-us/article/Supported-Data-Sources-and-Data- Typeshttps://support.office.com/en-us/article/Supported-Data-Sources-and-Data- Types  DMG Prerequisites : https://support.office.com/en-us/article/Data-source-prerequisites-Power-Queryhttps://support.office.com/en-us/article/Data-source-prerequisites-Power-Query  SSIS Data Feed : http://www.microsoft.com/en-us/download/details.aspx?id=39931http://www.microsoft.com/en-us/download/details.aspx?id=39931  How does DMG work - sequence diagram :  https://support.office.com/en-us/article/An-Inside-look-at-Microsoft-Data-Management-Gateway- 078D306D-2D28-409C-962B-D80B69A1D0B2 https://support.office.com/en-us/article/An-Inside-look-at-Microsoft-Data-Management-Gateway- 078D306D-2D28-409C-962B-D80B69A1D0B2  Azure IP adresses : http://www.microsoft.com/en-us/download/details.aspx?id=41653http://www.microsoft.com/en-us/download/details.aspx?id=41653  Store encrypted credentials in the cloud for business continuity : https://support.office.com/en- us/article/Business-Continuity-for-a-Data-Management-Gateway-00106c58-a0a1-4cea-8886- da51b8cadc5bhttps://support.office.com/en- us/article/Business-Continuity-for-a-Data-Management-Gateway-00106c58-a0a1-4cea-8886- da51b8cadc5b  Matt Masson Session at SPC : http://channel9.msdn.com/Events/SharePoint- Conference/2014/SPC397http://channel9.msdn.com/Events/SharePoint- Conference/2014/SPC397  Mini Nay & Luming Han Session at PASS Summit : http://www.sqlpass.org/summit/2014/Sessions.aspx#sid7231 http://www.sqlpass.org/summit/2014/Sessions.aspx#sid7231

42. @djeepy1 - http://blog.djeepy1.nethttp://blog.djeepy1.net Any questions ? Thank You!