Building ETL/ELT Workloads with Azure Data Factory V2 cseferlis@pragmaticworks.com blog.bizdataviz.com @bizdataviz Linkedin.com/in/cseferlis Destination Azure: Building ETL/ELT Workloads with Azure Data Factory V2
Who Am I? Sr Principal Architect @ Pragmatic Works Former CIO cseferlis@pragmaticworks.com blog.bizdataviz.com @bizdataviz Linkedin.com/in/cseferlis Sr Principal Architect @ Pragmatic Works Former CIO Speaker, Blogger, General BI Enthusiast
Session Goals Mutually Beneficial Presentation cseferlis@pragmaticworks.com blog.bizdataviz.com @bizdataviz Linkedin.com/in/cseferlis Mutually Beneficial Presentation No single version of truth Canned Example of ETL Move to Azure Some Demo Lessons Learned
Who are you? cseferlis@pragmaticworks.com blog.bizdataviz.com Linkedin.com/in/cseferlis
ADFv2 Enhancements Pipeline Execution cseferlis@pragmaticworks.com blog.bizdataviz.com @bizdataviz Linkedin.com/in/cseferlis Pipeline Execution Control Flow Elements (Looping and Branching) Parameters Integration Runtimes Data Movement, Activity Dispatch, SSIS Package Execution Deploy SSIS Packages to Azure
Control Flow/Data Flow Key ADF Terms cseferlis@pragmaticworks.com blog.bizdataviz.com @bizdataviz Linkedin.com/in/cseferlis SSIS Package Control Flow/Data Flow Connection String SQL Agent Job Notification Task ADF Pipeline Activity Linked Service Trigger Logic App
Development & Lifecycle Multiple Environments Multiple Configurations ARM Templates VSTS GIT Repositories and Deployments
Example: Research Collection cseferlis@pragmaticworks.com blog.bizdataviz.com @bizdataviz Linkedin.com/in/cseferlis SQL Sources: ETL: DELIVERY:
cseferlis@pragmaticworks.com blog.bizdataviz.com @bizdataviz Linkedin.com/in/cseferlis
ADF Demo cseferlis@pragmaticworks.com blog.bizdataviz.com @bizdataviz Linkedin.com/in/cseferlis
Lessons Learned Still in Preview Documentation will improve cseferlis@pragmaticworks.com blog.bizdataviz.com @bizdataviz Linkedin.com/in/cseferlis Still in Preview Documentation will improve More Features Coming Azure Everyday: blog.pragmaticworks.com Do the Tutorials: https://github.com/kromer m/adflab Costs are relative