1. Building a data connector for Power BI
Cédric L. Charlier
@Seddryck
Seddryck
Building a data connector for Power BI

2. Sponsors
Gold sponsors:
Bronze sponsors:
In partnership with:

3. Open-sources contributions
Skills & Interests
Cédric L. Charlier
@Seddryck
In a nutshell
Database architecture
Data modeling
Business intelligence
Data preparation
Data warehousing
ETL
Olap/Tabular
No SQL
Big data
Data analysis
Machine learning
seddryck.wordpress.com
Data & BI architect
Agile data warehouse
Quality-oriented
Automation fanboy
Open-sources contributions
Testing framework for BI
Other projects on github
Deployment SSRS – RsPackage
TmBundle for a few languages
Starting SSIS from .Net - Cassis
Toolset for special cases of modeling: ERMine et Tibre
And much more
Recognitions
Microsoft MVP Data Platform award
Agile leader of the month
Experience
15+ years with SQL Server and other data platforms
Former consultant, working internally for Elia

4. Power BI Data Connector
Your data
source
Power BI
(Desktop)

5. Adapt your solutions to your users

6. When to create data connectors
Creating a user-friendly experience over a REST API
Implementing an OAuth2 authentication flow
Limiting view over a data source to facilitate discovery operations
Branding a supported source (OData or ODBC)
Enabling DirectQuery for ODBC for a given source

7. Where can I use them At the moment … soon … Power BI Desktop
Downloadable from Power BI Desktop (Get data  “Certified Connectors”)
On-Premises Gateway (Personal mode)
Need a TestConnection section
New connectors or updates require restart of the gateway
… soon …
On-Premises Gateway (Enterprise mode)
Downloadable from Office Store

8. This is still in preview!

9. Raw materials
Building concepts and blocks
Power Query/ M language

10. Power Query M Language When you go further than click, click, done
The code that you’ve on the “Advanced Editor”

11. Benefits of functional languages
Evaluation of mathematical functions
Avoids changing-state and mutable data
No local or global state
No side effect
Output only depends on the arguments passed to this function

12. Expressiveness of a functional language
C# (imperative way)
Elixir (functional way)
static int Fibonacci(int n) { int a = 0; int b = 1; int c = 0; if (n != 1) { for (int i = 1; i <= n; i++){ c = a + b; a = b; b = c; } return a; } else { return 1; }
defmodule Fibonacci do def fib(0), do: 0 def fib(1), do: 1 def fib(n), do: fib(n-1) + fib(n-2) end

13. The standard library More than 500 predefined functions:
Manipulate values
Number.E,
TextPositionOf(“Hello”, “ll”)
Modify lists, records and tables
List.Transform(…)
Record.FieldCount()
Table.AddColumn(…)
Access data sources
Sql.Database(…)
Web.Contents(…)

14. Extensibility Creating new functions is easy but …
Difficult to share them (copy/paste)
Difficult to edit in Power BI Desktop
Difficult for entry-level users to use them

15. Power Query SDK Extension of Visual Studio Editor Create extensions
Poweful
Intellisense
Auto-completion
Frustrating
No-debug experience
Can I get the full error message?
Create extensions
Data connectors

16. Create your (very) first data connector
List numbers from 1 to 50
Run it into Visual Studio
Deploy for Power BI usage

17. Automating the deployment
.cmd or .ps file
Not possible to use post-build events
VsAutoDeploy
Visual Studio Extension (free)
Copy files from output directory to specified directory after each build

18. Path the way
to a REST api

19. Wind generation REST API
Quarter-hour by quarter-hour the wind power in Belgium
No authentication
Parameters : period (from a date to another date)
XML based
Local time
Forecasts and observations

20. Brand your new data connector
Labels in the list of connectors
Label when requesting credentials

21. Start a real implementation
Main and sub-functions
Passing parameters between functions

22. Declaration and implementation
Add information in the UI to guide the users
Tooltips
Friendly names
Implementation
Real code
Map them with
Value.ReplaceType(
implementation
, declaration )

23. Having multiple entry-point functions
Sharing the same security context
Multiples shared functions
Navigation tables (nested or not)

24. OAuth 2.0

25. Sharing your credentials with Power BI?

26. OAuth 2.0 flow User Power BI Resource Server Authorization Server
Opens redirect url
Present Authorization UI
Submit credentials and authorize
Return Auth. code
Submit Auth. code
Return Auth. token
Request protected resources with Auth. token
Return protected resources

27. Supports within Power BI
Authentication
OAuth
StartLogin
FinishLogin
Refresh tokens

28. Partial responses in REST api

29. Fast and not furious REST API must deliver fast responses
Send partial answers
Pagination
Specify some traversing tips
Headers is the best place
Link: < rel="next", < rel="last"

30. Don’t reinvent the wheel

31. Handle paginated responses with Power BI
Links to other pages are in the header
Value.Metadata(response)[Headers][#Link]
One of them is rel=“next”
Split, List.Select
Add this link as a metadata of your table
table meta [Next=link]
Iterate until this link is null
Recursive calls
Stop condition

32. Code available

33. Sponsors
Gold sponsors:
Bronze sponsors:
In partnership with: