1. Using Object Dependencies to develop smart solutions
Frans van der Geer

2. Thanks to our sponsors!
Please add this slide in your presentation

3. Please add this slide in your presentation

4. To auto document your BI solutions To prove your data lineage
Goal of this session
Inspire you to use object dependencies:
To auto document your BI solutions
To prove your data lineage
To do impact analysis
To do incremental deployments
To learn from how I did it

5. Solution Context Scope of this session Power BI Cloud Service
SQLMonitor
Scope of this session
Dependency tracking using queries and custom code
SSRS
Reports
Power BI Gateway
Sources
SQL Staging
Area
SQL DWH
SSAS Tabular
models
MetaModel driven code generation
Definition of:
Entities
Attributes
Relations
Sources
MetaModel

6. BI Solution Layers, which will help navigate through dependencies7
SSRS/Excel/PowerBI dashboards
SSAS Tabular Models 6
Hand written business logic
Views for Tabular Models 5 4
DWH views 3
DWH table valued functions
Generated 2
DWH tables
Staging tables 1
Sources

7. Datamodel for dependency tracking
Catalog
Report1
Dataset1
View1
View2
Table Valued Function
Tables
database
Report
Project
Object
Column c1 depends on table t1
View v1 depends on table t1
Calculated Column1 depends on C3
and C4 …
Dependency
Dependency
Child
Parent
Object
Tables, columns, views, stored procedures, functions, KPI’s, …
Objects
Catalog
SQL Databases, SSAS Databases, SSRS Projects
Catalog

8. Auto document your database objects
Functional description in MetaModel is attached to database objects by code generator using extended properties
Make use of Description field in VS Tabular Designer
Query those descriptions when loading meta data
Include object definitions
Include DAX expressions

9. Objects and Dependencies in SQL Server
Using system catalog queries
sys.objects, sys.columns,…(long list)..., sys.sql_modules
sys.extended_properties
sys.sql_expression_dependencies

10. Objects and Dependencies in SSAS Tabular
Using queries on SSAS DMV’s
SELECT * FROM
[$system].[MDSCHEMA_HIERARCHIES]
[$system].[MDSCHEMA_MEASURES]
[$system].[MDSCHEMA_KPIS]
[$system].[DISCOVER_CALC_DEPENDENCY] (see: note 1)
SELECT DISTINCT [DATABASE_NAME] as [DatabaseName], 'Model' as [SchemaName], [TABLE] as ChildObject, [OBJECT] as ChildField, [REFERENCED_TABLE] as ParentObject, [REFERENCED_OBJECT] as ParentField
FROM [$SYSTEM].[DISCOVER_CALC_DEPENDENCY]
WHERE ([OBJECT_TYPE] = 'CALC_COLUMN' OR OBJECT_TYPE = 'MEASURE' OR OBJECT_TYPE = 'KPI')
AND ([REFERENCED_OBJECT_TYPE] = 'MEASURE' OR [REFERENCED_OBJECT_TYPE] = 'CALC_COLUMN' OR [REFERENCED_OBJECT_TYPE] = 'COLUMN' OR [REFERENCED_OBJECT_TYPE] = 'HIERARCHY')
AND [REFERENCED_OBJECT] <> 'RowNumber'
(1) DISCOVER_CALC_DEPENDENCY is broken in SSAS 2016 in 1200 compatibility mode. Fixed in SSAS 2017 (I had not yet chance to verify this)

11. SSAS Tabular Query Extraction
SSAS database
Tabular table
Query
Database objects
SQL database
CompatibilityLevel < 1200: AMO
CompatibilityLevel >= 1200: TOM
Get the query
Create temporary stored procedure and paste the query in
Use sys.sql_expression_dependencies to get the dependencies
AMO = Analysis Services Management Objects
TOM = Tabular Object Model

12. Objects and Dependencies in SSRS
- Extraction by parsing the source files in the SSRS project
Report
Project
Report1
Query Extraction and parsing
Shared DataSet1
DataSource1
Query
Database objects
Database

13. Objects and Dependencies in SSIS
To do
I use the meta data in MetaModel to derive dependencies 
SSIS packages are generated using BIML
Could query SSIS DB
Could parse .dtsx files

14. What could you do with all this information?

15. Alle objecten worden gerubriceerd in een Catalog: SQL Database, SSAS Database, of een SSRS project

16. Drill down op de Catalog “Keuring” geeft een lijst met tabellen die er in zitten

17. …
Drill down op een tabel “Perceel” in Catalog “Keuring”. Geeft alle berekende kolommen, kolommen, measures

18. Toont de dependencies van een calculated column in een SSAS tabular table.
Je ziet hier ook de definitie van de formule van de calculated column

19. Deze schermafdruk toont de query achter een tabel in een SSAS database met al zijn afhankeljikheden.
De gele balk toont het (gegenereerde) commentaar regeltje.

20. Proof of the pudding: incremental deployments

21. Goal: Get a report or SSAS database into production
Partial deployment
SSAS database
Tabular table
Query
Database objects
SQL database
Deployable
Object
Goal: Get a report or SSAS database into production
Question: What do you need?
Answer the question by using object dependencies!
Deployment order is defined by object dependencies!
Question: How to make incremental changes?
SSAS / SSRS / SSIS: Deploy the whole project
SQL: Generate Change Scripts
Don’t need to deploy

22. How to generate a SQL database change script for a partial deployment?
By using some PowerShell glue!
Prepare deployment:
Choose your deployable objects
Add all dependent objects
Create a temporary database
Create all selected objects in dependency order (note 1)
Create a DACPAC using SqlPackage (note 2)
Execute deployment:
Generate a change script using SqlPackage with the DACPAC against the target database and run it (note 3)
Note 1: In the attached script files you will find a powershell script to script out all database objects and add them in TFS (source control)
Note 2: See:
Note 3: The auto generated change script will be run with the “don’t drop any objects” options set. So some complex changes cannot be done without manual written change scripts.

23. Heb je vragen? fvdgeer@vxcompany.com
Wil je een grafiek toevoegen.

24. Please fill in the evaluations
Please add this slide in your presentation