1. Azure Data Factory + SSIS: Migrating your ETLs to the Cloud
Presentation
Azure Data Factory + SSIS: Migrating your ETLs to the Cloud
Jose Chinchilla, MCSE

2. Jose Chinchilla, MCSE Data Analytics Lead, AgileThought jose
Jose Chinchilla, MCSE Data Analytics Lead, AgileThought linkedin.com/in/josechinchilla
@sqljoe

3. Agenda Azure Data Factory (ADF) Integration Runtime (IR)
ETL migration scenarios to ADFv2
Demo:
Configuring ADF-SSIS IR
Deploying, executing and monitoring an SSIS Project to ADF
Executing a Copy Activity

4. What is Azure Data Factory (ADF)?

5. Azure Data Factory ADFv1 and ADFv2 (current)
Data Integration / ETL web service
70+ data source connectors
Drag and drop UI to author data pipelines
Schedule (trigger), run and monitor pipeline executions
SSIS-like Control and Data Flow (preview)

6. Azure Data Factory
~ SSIS + SQL Agent

7. Azure Data Factory (ADF)
Feature
ADFv1
ADFv2
GUI development
Limited
SSIS-like
Activities
Limited, required custom
More out-of-the-box
Linked services
SSIS Package execution
Not supported*
Full support
On-prem sources
Requires VPN or ER
Self-hosted IR
Compare versions:

8. Azure Data Factory (ADF)
Feature
ADFv1
ADFv2
GUI development
Limited
SSIS-like
Activities
Limited, required custom
More out-of-the-box
Linked services
SSIS Package execution
Not supported*
Full support
On-prem sources
Requires VPN or ER
Self-hosted IR
Compare versions:

9. ADFv2 Features Pipelines Activities Datasets (Source & Sink)
Connections
Linked Services
Integration Runtime
Triggers

10. ADFv2 Pipeline with single activity
Linked Service
Linked Service
ADFv2
Pipeline
Copy Activity
Source Dataset
Sink Dataset

11. Integration Runtime

12. Integration Runtime
Provides data integration capabilities across different network environments
Data movement: Move data between data stores in public network and data stores in private network (on-premises or virtual private network). It provides support for built-in connectors, format conversion, column mapping, and performant and scalable data transfer.
Activity dispatch: Dispatch and monitor transformation activities running on a variety of compute services such as Azure HDInsight, Azure Machine Learning, Azure SQL Database, SQL Server, and more.
SSIS package execution: Natively execute SQL Server Integration Services (SSIS) packages in a managed Azure compute environment.

13. Integration Runtime Azure-SSIS
Deploy, monitor and manage SSIS packages
Integration Services Catalog (SSISDB)
Azure
Azure to Azure or other public networks
Always one by default aka AutoResolve
Self-Hosted
Azure to private networks
On-prem or private virtual network
Linked Self-Hosted
Shared IR with other ADFv2

14. https://docs. microsoft

15. https://docs. microsoft

16. ETL Migration Scenarios

17. Common Migration Strategies
Rehost "as is" (lift and shift)
Replatform (lift, tinker and shift)
Refactor
Rearchitect
Rebuild
Replace
More intrusive

18. Migration goals Eliminate infrastructure overhead Reduce costs
Scale up/down as needed
Eliminate re-writing and re-architecting ETLs
…what else ?

19. ETL Migration Scenarios
Lift & shift SSIS packages
Deploy, monitor and run SSIS packages from ADF-SSIS IR
Change environment variable values (connection managers, credentials, etc.)
Lift and shift is a strategy for moving an application or operation from one environment to another – without redesigning the app. In the lift-and-shift approach, certain workloads and tasks can be moved from on-premises storage to the cloud…

20. ETL Migration Scenarios
Replatform / Rearchitect
Run stored procedure based ETLs using ADF CopyActivity
Stage on-prem data in a Data Lake or Blob storage

21. Demo

22. Q&A

23. www. linkedin.com/company/AgileThought @AgileThought
Stay Connected
www. linkedin.com/company/AgileThought
@AgileThought
If you have questions or would like more information, feel free to contact me via

24. Links and References
Azure Data Factory Documentation
Create a trigger that runs a pipeline in response to an event trigger