1. SSIS Data Integration Data Warehouse Acceleration
Fact and Dimension Table Patterns
Jim Miller – President BI Tracks Consulting
Product Presentation
June 8, 2019

2. What is Biml?
XML Schema used to define SSIS, and other Microsoft ‘BI Stack’ components
Use with BimlExpress to auto-generate SSIS Projects and Packages in Visual Studio (SSDT)
Who has used Biml? Who is developing a Data Warehouse?

3. Why Use Biml to Build SSIS Packages?
Code Reuse, Code Modularity
Code Reuse = Efficiency & Reliability
Code Modularity (Smaller Pieces ) promotes Code Reuse
Resource Repository Defined Outside of SSIS Projects
Connections, Parameters, Packages
Standardization of SSIS Project/Package Development
Framework Extensions, Package Design Styles
Automated Environment Management
Simplifies use of SSIS - Project/Package Creation
All made possible by dealing with just XML code. All XML code is generated by encapsulated SELECT statements.

4. Demo 1 – Generating SSIS Packages in Visual Studio using BimlExpress
BimlExpress will generate ‘pure biml tags’, or biml script (embedded C# code nuggets).
Who does not want to create higher quality work in less time?

5. Biml is just XML, so it’s free
Biml is just XML, so it’s free. BimlSnap Online, which creates Biml is free. BimlExpress which converts Biml into SSIS packages is a free VS add-in and it’s also free.

6. Biml and Biml Script Examples
Bimlsnap logic is all in the database layer, Biml Script uses embedded .NET code nuggets

7. BimlSnap (and SSIS) Workflow and Databases
bimlsnap_v2
(project config & metadata)
Define
Build
Deploy
Changes typically begin with a change to the database schema (metadata)
RRRR
SSIS_Data
(runtime)
SSIS Catalog
SSISDB
(runtime)
log
Log
Run-time

8. Demo 2 – How it Works (Select for XML)
Everything we do in BimlSnap to generate Biml is based on the “Select… for XML” technique.

9. BimlSnap Desktop - Components
Back-End SQL Express DB
Windows Front-End Application
SQL Server Data Tools (SSDT) with BimlExpress Add-in (Varigence)
SSIS Run-time Framework
Download:
Open Source Databases (bimlsnap_v2, SSIS_Data)
SSIS Project to update Metadata

10. ELT vs. ETL Paradigm

11. BimlSnap - Metadata Management (v2)
Built as a Data Mart using Type 1 and Type 2 Dimensions
Extends Information Schema Views
Front-end Client
Includes SSIS Project to Refresh Metadata

12. BimlSnap – Runtime Framework
Uses a free Back-end Database: (SSIS_Data)
Extends Functionality of the SSIS Catalog Database (SSISDB)
Integrates with BimlSnap Generated Packages to Provide:
Project restart ability
Project/Package row counts
Project/Package runtimes
alerts
Package error logging
Feeds a PowerBI Dashboard for Runtime Analytics

13. Demo 4 – Power BI Dashboard
Power BI Project available upon request.

14. BimlSnap Project Configuration
Select a Template
Alerts
Project Re-runability
Row Counts
Select a Package Protection Level
Encrypt Sensitive With User Key
Do Not Save Sensitive

15. Demo 2 - BimlSnap Package Configuration
DataFlow Pattern
Package Qualifier
Source Connection
Source Query
Destination Connection
Destination Table
Truncate Option
Identity Insert Options

16. Code Generators
Code Generators
Merge Statements
Table Partitioning

17. SSIS/Biml Project Build
Project Integration
Connections
Pattern Definition
Package
XML
Project
XML SP VS
Package Configuration SP
Package Template
Parameters
Framework Integration

18. Demo 3 - BimlSnap Project Walk-Through,and Package Creation
BTW, all SQL code needed to generate SSIS packages is in the database.

19. Demo 4 – Select with HTML Output
Documents all projects, packages in an conveniently linked HTML file

20. BimlSnap Foot Print

21. Links
Blog: Accelerated ELT using BI Markup Language (Biml) Project Vulcan: Varigence: Biml Express (VS Add-in): BimlSnap Website: ELT vs. ETL: Presentation on YouTube: (jump to minute: 6)