THR1171 Azure Data Integration: Choosing between SSIS, Azure Data Factory, and Azure Databricks Cathrine Wilhelmsen, Inmeta @cathrinew | cathrinew.net

You are a data engineer

You are starting a new project

Which tool do you pick?

SQL Server Integrations Services (SSIS)? Azure Data Factory (ADF)? Azure Databricks? All of the above?

It Depends ™

What does it depend on?

Let's take a look!

SQL Server Integration Services

SQL Server Integration Services Data Integration Extract, Transform, Load (ETL) On-Premises

Azure Data Factory

Azure Data Factory Data Movement & Orchestration Extract, Load, Transform (ELT) Hybrid

Azure Databricks

Azure Databricks Apache Spark-based Analytics Service Collaborative Notebooks Cloud

Data Volume Velocity Variety

Volume Medium High High

Velocity Batch Batch Streaming Batch Streaming Real-Time

Variety Structured Structured Unstructured Structured Unstructured

Development Tools Interface Languages

Development Tools SSDT Browser Browser

Development Interface Drag and Drop Drag and Drop Code Code

Development Languages VB / C# Biml .NET Python PowerShell SQL Python R Scala

Usage Platform Pricing Purpose

Platform On-Premises Own Hardware Scale Out Hybrid Managed Scale Up Cloud Managed Autoscale

Pricing License Pay as you go Pay as you go

Purpose Integration Transformation ETL Movement Orchestration ETL / ELT Preparation Collaboration AI / ML

Modern Data Warehouse

Modern Data Warehouse Ingest Store Prep & Train Model & Serve Visualize

SSIS ♥ ADF Lift and Shift SSIS Execute SSIS in ADF

ADF ♥ Databricks Databricks Activities Data Flows run on Databricks

…everywhere a data flow… Here a flow… …there a flow …everywhere a data flow…

Cathrine Wilhelmsen, Inmeta @cathrinew | cathrinew.net Thank you! :) Cathrine Wilhelmsen, Inmeta @cathrinew | cathrinew.net

Please evaluate this session Your feedback is important to us! 1/1/2019 9:51 PM Please evaluate this session Your feedback is important to us! Please evaluate this session through MyEvaluations on the mobile app or website. Download the app: https://aka.ms/ignite.mobileApp Go to the website: https://myignite.techcommunity.microsoft.com/evaluations © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

1/1/2019 9:51 PM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.