Azure SQL Data Warehouse Scaling: Configuration and Guidance Microsoft 2016 9/21/2018 5:22 AM BRK3291 Azure SQL Data Warehouse Scaling: Configuration and Guidance JRJ & Kevin SQLDW PMs jrj@microsoft.com kevin@microsoft.com © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Agenda Compute Scaling Storage Sizing Pricing Scenarios Q&A TechReady 23 9/21/2018 5:22 AM Agenda Compute Scaling Storage Sizing Pricing Scenarios Q&A © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Elastic Scale Concepts

Separation of compute from storage Microsoft Build 2016 Separation of compute from storage 9/21/2018 5:22 AM Compute Control Premium Storage © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Independently scale compute Microsoft Build 2016 Independently scale compute 9/21/2018 5:22 AM Compute Control Premium Storage © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Pause and resume workload Microsoft Build 2016 Pause and resume workload 9/21/2018 5:22 AM Compute Control Premium Storage © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Geo-redundancy Microsoft Build 2016 9/21/2018 5:22 AM © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Introducing Data Warehouse Units CPU DWU DW100 DW200 DW300 DW400 DW500 DW600 DW1000 DW1200 DW1500 DW2000 DW3000 DW6000 ALTER DATABASE ContosoDW MODIFY (service_objective = 'DW1000' ) ; RAM I/O

9/21/2018 5:22 AM Compute scaling © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Starting point: Sizing by capacity 0-4 TBs 4-8 TBs 8-12 TBs 12-16 TBs 16-20 TBs 20-36 TBs 36-48 TBs 48-60 TBs 60-80 TBs 80-160 TBs >160 TBs 100 200 300 400 500 600 1000 1200 1500 2000 3000 6000 Recommended starting point Flexibility to select any range of DWUs

Compute scaling factors Concurrency Transaction size Load Memory Data Warehouse Unit (DWU)

Concurrency: queries QID 32 active queries open sessions 1024

Concurrency: slots TechReady 23 9/21/2018 5:22 AM © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Concurrency slot consumption TechReady 23 9/21/2018 5:22 AM Concurrency slot consumption 1 2-3 4-5 © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Concurrency impact: queries & slots

Transaction Size Microsoft Build 2016 9/21/2018 5:22 AM © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Use mediumrc+ for high DWU loads Microsoft Build 2016 9/21/2018 5:22 AM Data loading DWU Max External Readers Max Writers DW100 8 60 DW200 16 DW300 24 DW400 32 DW500 40 DW600 48 DW1000 80 DW1200 96 DW1500 120 DW2000 160 DW3000 240 DW6000 480 Exception 60 Writers Target Table = CI or NCI Load user is defaultrc TAKE AWAY Use mediumrc+ for high DWU loads © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Load scaling 30-32 MB/Sec/Node 110-115 GB/Hour/Node

Loading delimited text A compressed text file cannot be read in parallel Splitting data across multiple files maximises load performance

Memory Management (MB per distribution) Service admin account

Memory grant sizing factors Target rows in the rowgroup Table Overhead #columns #short string character typed columns #long string character typed columns

Est. mem grant example #1: 10 column table

Est. mem grant example #2: 30 column table

Demo: Memory Sizing JRJ

Storage Sizing

Storage tiers Local storage Premium storage (remote) Blob storage (remote and geo redundant)

Data locations Local Premium Blob Tempdb Data files Geo-backup Billed to customer Local Premium Blob Tempdb Data files Geo-backup Log files Snapshots

Local Storage: Tempdb sizing Microsoft Build 2016 9/21/2018 5:22 AM Local Storage: Tempdb sizing ~399GB per DW100 © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Premium storage: Capacity limits TechReady 23 9/21/2018 5:22 AM Premium storage: Capacity limits 240TB File capacity 5x CCI compression >1PB Db capacity © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Premium Storage: Database Size TechReady 23 9/21/2018 5:22 AM Premium Storage: Database Size Unallocated Unused Total DB Size Reserved Space Index Used Space Data Log © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Premium Storage: Unallocated Space TechReady 23 9/21/2018 5:22 AM Premium Storage: Unallocated Space 4TB Unallocated 12TB Reserved 4TB Reserved 12TB Unallocated 16 TB © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Premium Storage: Snapshots TechReady 23 9/21/2018 5:22 AM Premium Storage: Snapshots Frequency Retention 4 hours 7 days RPO : 8 Hours © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Blob Storage: Geo-redundant backups TechReady 23 9/21/2018 5:22 AM Blob Storage: Geo-redundant backups Frequency and Retention 1 Geo-backup 24hr RPO © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Capping storage capacity Microsoft Build 2016 9/21/2018 5:22 AM Capping storage capacity CREATE DATABASE MyDB COLLATE SQL_Latin1_General_CP1_CI_AS ( EDITION = 'DataWarehouse' , SERVICE_OBJECTIVE = 'DW400' , MAXSIZE = 10240 GB ); ALTER DATABASE MyDB MODIFY (MAXSIZE = 245760 GB); © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Storage sizing summary TechReady 23 9/21/2018 5:22 AM Storage sizing summary Database size: sp_spaceused Table sizing: DMVs Snapshot size: Total storage size (portal) – database size (sp_spaceused) Free space (unallocated): © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Pricing

Compute pricing: DWU $900 Per DWU100 Per Month

Storage sizing: Premium storage $122.88 TiB / Month Data files Log files Snapshots

Storage pricing: Geo-redundant backups $0.12 GB / Month

Pricing scenarios Microsoft Build 2016 9/21/2018 5:22 AM GA price: Storage: $122.88 DWU: $1.21 © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Pricing scenarios Microsoft Build 2016 9/21/2018 5:22 AM © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Summary

Summary of the session Session Objective(s): Key takeaways: TechReady 23 9/21/2018 5:22 AM Summary of the session Session Objective(s): Elastic scale concepts Understand compute scaling factors Understand all components of the storage model Explain how compute and storage affects pricing with SQL Data Warehouse Key takeaways: Understand compute scaling and analyze memory requirements Compute scaling is the primary cost driver Elasticity empowers cost optimization and different utilization patterns Storage is not just data in the database © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Free IT Pro resources To advance your career in cloud technology Microsoft Ignite 2016 9/21/2018 5:22 AM Free IT Pro resources To advance your career in cloud technology Plan your career path Microsoft IT Pro Career Center www.microsoft.com/itprocareercenter Cloud role mapping Expert advice on skills needed Self-paced curriculum by cloud role $300 Azure credits and extended trials Pluralsight 3 month subscription (10 courses) Phone support incident Weekly short videos and insights from Microsoft’s leaders and engineers Connect with community of peers and Microsoft experts Get started with Azure Microsoft IT Pro Cloud Essentials www.microsoft.com/itprocloudessentials Demos and how-to videos Microsoft Mechanics www.microsoft.com/mechanics Connect with peers and experts Microsoft Tech Community https://techcommunity.microsoft.com © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Please evaluate this session 9/21/2018 5:22 AM Please evaluate this session Your feedback is important to us! From your PC or Tablet visit MyIgnite at http://myignite.microsoft.com From your phone download and use the Ignite Mobile App by scanning the QR code above or visiting https://aka.ms/ignite.mobileapp © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

9/21/2018 5:22 AM © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.