Platinum Sponsors Titanium Sponsors
ETL Tool (SSIS, etc) EDW (SQL Svr, Teradata, etc) Extract Original Data Load Transformed Data Transform BI Tools Data Marts Data Lake(s) Dashboards Apps
ETL Tool (SSIS, etc) EDW (SQL Svr, Teradata, etc) Extract Original Data Load Transformed Data Transform BI Tools Ingest (EL) Original Data Data Marts Data Lake(s) Dashboards Apps
ETL Tool (SSIS, etc) EDW (SQL Svr, Teradata, etc) Extract Original Data Load Transformed Data Transform BI Tools Ingest (EL) Original Data Scale-out Storage & Compute (HDFS, Blob Storage, etc) Transform & Load Data Marts Data Lake(s) Dashboards Apps Streaming data
ETL Tool (SSIS, etc) EDW (SQL Svr, Teradata, etc) Extract Original Data Load Transformed Data Transform BI Tools Ingest (EL) Original Data Scale-out Storage & Compute (HDFS, Blob Storage, etc) Transform & Load Data Marts Data Lake(s) Dashboards Apps Streaming data
Azure Data Factory
Compose storage, movement, and processing services into data pipelines Initial data sources SQL Server, SQL Server in IaaS Virtual Machines, Azure SQL Database, Azure Blobs, and Azure Tables Initial processing services Hive, Pig, C# code running on HDInsight hybrid data movement PowerShell developer experience for pipeline composition and deployment Cluster management for on-demand or bring your own Rich visual monitoring experience for a single view of all pipelines and datasets that provides lineage dependencies, health monitoring, and error identification Consume datasets by BI tools and applications Capabilities at Public Preview
Example: customer profiling, game analytics
Copy new users to blob storage Join and aggregate activity per week and user table Weekly Privacy: Refresh: Xbox New Users Game New Users Game Activity Per Week Daily New User Activity Per Week Contains PII Weekly, Mon by 8AM
All data, and all systems