Populating a Data Warehouse Data Warehouse Acceleration Fact and Dimension Table Patterns Jim Miller – President, BI Tracks Consulting SQL Saturday – Cincinnati March 17, 2018

Why Use Biml to Build SSIS Packages? Code Reuse , Code Reuse , Code Reuse Microsoft’s Project Vulcan was to bring Code Reusability to the MSBI Stack Resource Repository Defined Outside of SSIS Projects Connections, Parameters, Packages Standardization of SSIS Project/Package Development Accelerated Change Management

BimlSnap Workflow Generate Data Warehouse Biml File Populate SSDT

BimlSnap Online - Components Website: https://bimlsnap.bitracks.net:2525/ SQL Server Data Tools (SSDT) with BimlExpress Add-in from Varigence SSIS Run-time Framework (Optional) Snap_Mart Metadata Solution (Optional) Download: http://www.bimlsnap.com/downloads/ Back-end SQL Server Database Front-end Client Optional SSIS Project to update Metadata Database

BimlSnap Workflow with Metadata Snap_Mart Metadata Upload Metadata Populate Data Warehouse Biml Populate Changes

BimlSnap - Metadata Management Stand alone application Has its own Data Mart using Type 1 and Type 2 Dimensions Uses a free Back-end Database: Front-end Client Extends Information Schema Views Includes Optional SSIS Project to Refresh Metadata

BimlSnap Online – Getting Started w/Metadata Steps Download SSIS Metadata and Framework Products: www.bimlsnap.com Run SQL Database Scripts (snap_mart, and optionally the ‘SSIS_Data’ run-time framework database) Configure and Run SSIS Project: “snap_mart Refresh” Use Metadata Front-end (SnapMart v1.2.exe) to designate databases for Metadata extracts Re-run SSIS Project: “snap_mart Refresh” Update Dimension Column Types for: Business Keys (bk) ‘Type 2’ (historical) attributes (optional) Export from local client and Import into www.bimlsnap.com

BimlSnap – Runtime Framework Uses a free Back-end Database: Extends Functionality of the SSIS Catalog Database (SSISDB) Integrates with BimlSnap Generated Packages to Provide: Project restart ability Project/Package row counts Project/Package runtimes Email alerts Package error logging

Biml and Biml Script Examples

BimlSnap Use SQL Server Procedures, and Tables to generate Pure Biml code Includes a repository of re-usable ETL Patterns Common Data Warehouse patterns are currently available Integrated into an extensible ‘ETL Framework’

BimlSnap Project Configuration Select a Template Email Alerts Project Re-runability Row Counts Select a Package Protection Level Encrypt Sensitive With User Key Do Not Save Sensitive

BimlSnap Package Configuration DataFlow Pattern Package Qualifier Source Connection Source Query Destination Connection Destination Table Truncate Option Identity Insert Options

SSIS Project Build Project Integration Connections Pattern Definition Package XML Project XML SP VS Package Configuration SP Package Template Parameters Framework Integration

Code Generators Code Generators Merge Statements Table Partitioning

Links Blog: Accelerated ELT using BI Markup Language (Biml) Project Vulcan: https://vulcan.codeplex.com/ Varigence: https://www.varigence.com/ Biml Express (VS Add-in): https://www.varigence.com/BimlExpress www.bimlsnap.com www.bitracks.com Need Help? Please email us at: info@bitracks.com