Optimizing the ETL Development process using BIML SQL Saturday Indianapolis | August 17, 2019

About me Data Services Manager with Republic Bank and Trust Co-Chapter Leader of the Louisville SQL and Power BI User Group SQL Saturday Louisville Organizer Friend of Redgate About me

Contact Info Twitter: @CodeNameSQL Email: Joshua.Higginbotham@Codename sql.com GitHub Account: Jhiggin GitHub Org: CodeNameSQL

SSIS Package Overview

Introduction to biml Designed by Scott Currie who is the founder and CEO of Varigence. BIML stands for Business Intelligence Markup Language. BIML is a dialect of the XML language that generates SSIS packages from Visual studio (BIML Express) or from an IDE (BIML Studio). Discuss Scott’s previous employeer. Discuss that Biml can build SSIS packages, SSAS models and Cubes. Generate SQL scripts.

Development tools BIML Express: Visual Studio add-in which all demos today will be covered in. Product is free for use but requires registering with Varigence. BIML Studio: Full-Feature IDE for BIML Development. Cost associated based of per user licensing. Biml Flex: Added features on top of Biml Studio. Cost is based off organization size. Biml Online: Web based IDE for BIML. Currently Free while in Beta. Discuss that as of Biml Express 2018, reverse engineering of SSIS package is available in the free version.

What can biml do? Generate Multi dimensional and Tabular SSAS Models Generate Staging and Destination Schema Generate dynamic SQL statements Generate frameworks for tasks like Logging or Auditing to help with code reuse and troubleshooting. Discuss that as of Biml Express 2018, reverse engineering of SSIS package is available in the free version.

ETL Framework - Overview Connections Databases Schemas Tables Tables_Flags Columns Mappings

Demo (Framework driven ETL)

Additional Resources Catherine Wilhelmsen : https://www.cathrinewilhelmsen.net/ Varigence : https://varigence.com Andy Leonard : https://andyleonard.blog/ The Biml Book : https://www.apress.com/us/book/9781484231340 Examples : https://varigence.com/Documentation/Samples/ Presentation resources: https://github.com/Jhiggin/BIML_Demos David Stein: http://www.made2mentor.com/