Moving your on-prem data warehouse to cloud. What are your options? Asanka Padmakumara Moving your on-prem data warehouse to cloud. What are your options?

Who am I? Asanka Padmakumara Business Intelligence Consultant, More than 8 years in BI and Data Warehousing Blog: asankap.wordpress.com Linked In: linkedin.com/in/asankapadmakumara Twitter: @asanka_e Facebook: facebook.com/asankapk

Thanks to our Sponsors SILVER: PASS: VENUE: Global Alliance Partner Sri Lanka

Why you need to pay attention now? Cleaver

How do you define a DW? Central Organized Integrated Read Optimized Reporting/ Analysis SMP/MPP

Symmetric Multi Processing(SMP) Multiple processors, common OS and Memory Shared Everything Architecture Scale Up NUMA -> Dedicated Memory area for a processor NUMA-non-uniform memory access

Massively Parallel Processing (MPP) Multiple processors, own OS and Memory Shared Nothing Architecture Scale Out

SMP or MPP? SMP MPP Best suited for small to medium data sets Many small read/write operations Multiple row-by-row operations Best suited for big data, analytical, batch-oriented workloads Complex Queries If your data sizes already exceed 1 TB and are expected to continually grow

For your Data Warehousing needs SMP Azure SQL Database SQL Server in a VM MPP Azure Data Warehouse Apache Hive on HDInsight Interactive Query (Hive LL AP) on HDInsight symmetric multiprocessing massively parallel processing

Azure SQL Database SQL Database as a Service No CDC,CLR Primary file group only No database mail No cross data base queries Automatic Backup Managed Service No PolyBase Can’t pause Support In-Memory Dynamic Scalability SQL Database as a Service One or more databases in a logical sever Single/Elastic Pool/Managed Instance Max Size : 4 TB Pricing: DTU (Database Transaction Units) Premium 5 to 4000 DTUs

SQL Server in a VM SQL server installed and configured in a VM Can pause the VM High no of concurrent connections Support In-Memory Not a managed Service No Dynamic Scalability Seamless migration SQL server installed and configured in a VM Windows Server, Linux (Red Hat ,SUSE ,Ubuntu) MAX Size : 256 TB Pricing: VCore and RAM basic

Azure SQL DW PaaS One Control Node- Multiple Compute Nodes Distributed Computing Distributed Storage DWUs (Data Warehouse Units) Parallel Data Warehouse Distributed or replicated tables 60 distribution if 400DWUs- 4 compute nodes , each 15 Distributions – max 60 compute nodes Hash/Replicated/Round Robin Global temporary tables that begin with ## are not supported.

Azure Data Warehouse is not for Everyone Should I use Azure Data Warehouse? www.sqlchick.com/ No In-Memory No Real Time Limited Concurrency Data Encryption Managed Can pause Dynamic Scalability Faster Processing /Computation Melissa Coates No Primary key, Foreign keys, Unique

Demo Behavior of a Small DW in Azure SQL Database and Azure SQL Data Warehouse

Apache Hive on HDInsight Hadoop Data warehouse system Convert SQL like Queries (Hive QL) to MapReduce, Tez , or Spark jobs Batch Processing Works well with Semi-Structured Data Pausing -> Delete and re-create No Dynamic Scalability

Hive Interactive Query Also called Hive LLAP, or Live Long and Process In-Memory Caching Suite for real time analytics

I want to pause my resources when not using I have a real time reporting needs I want to manage my own server. I need to support a large number of concurrent users and connections . My data is mostly unstructured. Azure SQL Database Azure SQL Data warehouse SQL Server in a VM Apache Hive

Azure Data Lake Not a traditional DW, But … Highly scalable data storage and analytics service Largely intended for big data storage and analysis Consists of three services: Analytics Storage HDInsight (“managed clusters”) Azure Data Lake Analytics Azure Data Lake Storage

A DW solution using ADL Curated Data Staging Area Persisted(Raw) Data Azure Data Lake Store Staging Area Curated Data Persisted(Raw) Data Sandbox End user/Report authors Business Analysts (exploratory analysis) Azure Data Factory Azure Data Lake Analytics Azure Analysis Services Data Warehouse

Your Feedback is Important Paste Feedback QR Code here

Thanks to our Sponsors SILVER: PASS: VENUE: Global Alliance Partner Sri Lanka