1. Populating a Data Warehouse
Data Warehouse Acceleration
Fact and Dimension Table Patterns
Jim Miller – President, BI Tracks Consulting
SQL Saturday – JACKSONVILLE, FL
May 5, 2018

2. Why Use Biml to Build SSIS Packages?
Code Reuse , Code Reuse , Code Reuse
Microsoft’s Project Vulcan was to bring Code Reusability to the MSBI Stack
Resource Repository Defined Outside of SSIS Projects
Connections, Parameters, Packages
Standardization of SSIS Project/Package Development
Accelerated Change Management

4. BimlSnap Workflow
Generate
Data Warehouse
Biml File
Populate
SSDT

5. BimlSnap Online - Components
Website:
SQL Server Data Tools (SSDT) with BimlExpress Add-in from Varigence
SSIS Run-time Framework (Optional)
Snap_Mart Metadata Solution (Optional) Download:
Back-end SQL Server Database
Front-end Client
Optional SSIS Project to update Metadata Database

6. BimlSnap Workflow with Metadata
Snap_Mart
Metadata
Upload Metadata
Populate
Data Warehouse
Biml
Populate
Changes

7. BimlSnap - Metadata Management
Stand alone application
Has its own Data Mart using Type 1 and Type 2 Dimensions
Uses a free Back-end Database:
Front-end Client
Extends Information Schema Views
Includes Optional SSIS Project to Refresh Metadata

8. BimlSnap Online – Getting Started w/Metadata
Steps
Download SSIS Metadata and Framework Products:
Run SQL Database Scripts (snap_mart, and optionally the ‘SSIS_Data’ run-time framework database)
Configure and Run SSIS Project: “snap_mart Refresh”
Use Metadata Front-end (SnapMart v1.2.exe) to designate databases for Metadata extracts
Re-run SSIS Project: “snap_mart Refresh”
Update Dimension Column Types for:
Business Keys (bk)
‘Type 2’ (historical) attributes
Export from local client and Import into

9. BimlSnap – Runtime Framework
Uses a free Back-end Database: Extends Functionality of the SSIS Catalog Database (SSISDB)
Integrates with BimlSnap Generated Packages to Provide:
Project restart ability
Project/Package row counts
Project/Package runtimes
alerts
Package error logging

10. Biml and Biml Script Examples

11. BimlSnap
Use SQL Server Procedures, and Tables to generate Pure Biml code
Includes a repository of re-usable ELT Patterns
Common Data Warehouse patterns are currently available
Integrated into an extensible ‘ELT Framework’

12. BimlSnap Project Configuration
Select a Template
Alerts
Project Re-runability
Row Counts
Select a Package Protection Level
Encrypt Sensitive With User Key
Do Not Save Sensitive

13. BimlSnap Package Configuration
DataFlow Pattern
Package Qualifier
Source Connection
Source Query
Destination Connection
Destination Table
Truncate Option
Identity Insert Options

14. SSIS Project Build Project Integration Connections Pattern Definition
Package
XML
Project
XML SP VS
Package Configuration SP
Package Template
Parameters
Framework Integration

15. Code Generators
Code Generators
Merge Statements
Table Partitioning

16. Links
Blog: Accelerated ELT using BI Markup Language (Biml) Project Vulcan: Varigence: Biml Express (VS Add-in): BimlSnap Online Video: For a copy of this presentation, please us at: