Download presentation
Presentation is loading. Please wait.
1
Azure SQL Data Warehouse Scaling: Configuration and Guidance
Microsoft 2016 9/21/2018 5:22 AM BRK3291 Azure SQL Data Warehouse Scaling: Configuration and Guidance JRJ & Kevin SQLDW PMs © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
2
Agenda Compute Scaling Storage Sizing Pricing Scenarios Q&A
TechReady 23 9/21/2018 5:22 AM Agenda Compute Scaling Storage Sizing Pricing Scenarios Q&A © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
3
Elastic Scale Concepts
4
Separation of compute from storage
Microsoft Build 2016 Separation of compute from storage 9/21/2018 5:22 AM Compute Control Premium Storage © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
5
Independently scale compute
Microsoft Build 2016 Independently scale compute 9/21/2018 5:22 AM Compute Control Premium Storage © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
6
Pause and resume workload
Microsoft Build 2016 Pause and resume workload 9/21/2018 5:22 AM Compute Control Premium Storage © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
7
Geo-redundancy Microsoft Build 2016 9/21/2018 5:22 AM
© 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
8
Introducing Data Warehouse Units
CPU DWU DW100 DW200 DW300 DW400 DW500 DW600 DW1000 DW1200 DW1500 DW2000 DW3000 DW6000 ALTER DATABASE ContosoDW MODIFY (service_objective = 'DW1000' ) ; RAM I/O
9
9/21/2018 5:22 AM Compute scaling © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
10
Starting point: Sizing by capacity
0-4 TBs 4-8 TBs 8-12 TBs 12-16 TBs 16-20 TBs 20-36 TBs 36-48 TBs 48-60 TBs 60-80 TBs TBs >160 TBs 100 200 300 400 500 600 1000 1200 1500 2000 3000 6000 Recommended starting point Flexibility to select any range of DWUs
11
Compute scaling factors
Concurrency Transaction size Load Memory Data Warehouse Unit (DWU)
12
Concurrency: queries QID 32 active queries open sessions 1024
13
Concurrency: slots TechReady 23 9/21/2018 5:22 AM
© 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
14
Concurrency slot consumption
TechReady 23 9/21/2018 5:22 AM Concurrency slot consumption 1 2-3 4-5 © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
15
Concurrency impact: queries & slots
16
Transaction Size Microsoft Build 2016 9/21/2018 5:22 AM
© 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
17
Use mediumrc+ for high DWU loads
Microsoft Build 2016 9/21/2018 5:22 AM Data loading DWU Max External Readers Max Writers DW100 8 60 DW200 16 DW300 24 DW400 32 DW500 40 DW600 48 DW1000 80 DW1200 96 DW1500 120 DW2000 160 DW3000 240 DW6000 480 Exception 60 Writers Target Table = CI or NCI Load user is defaultrc TAKE AWAY Use mediumrc+ for high DWU loads © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
18
Load scaling 30-32 MB/Sec/Node GB/Hour/Node
19
Loading delimited text
A compressed text file cannot be read in parallel Splitting data across multiple files maximises load performance
20
Memory Management (MB per distribution)
Service admin account
21
Memory grant sizing factors
Target rows in the rowgroup Table Overhead #columns #short string character typed columns #long string character typed columns
22
Est. mem grant example #1: 10 column table
23
Est. mem grant example #2: 30 column table
24
Demo: Memory Sizing JRJ
25
Storage Sizing
26
Storage tiers Local storage Premium storage (remote)
Blob storage (remote and geo redundant)
27
Data locations Local Premium Blob Tempdb Data files Geo-backup
Billed to customer Local Premium Blob Tempdb Data files Geo-backup Log files Snapshots
28
Local Storage: Tempdb sizing
Microsoft Build 2016 9/21/2018 5:22 AM Local Storage: Tempdb sizing ~399GB per DW100 © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
29
Premium storage: Capacity limits
TechReady 23 9/21/2018 5:22 AM Premium storage: Capacity limits 240TB File capacity 5x CCI compression >1PB Db capacity © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
30
Premium Storage: Database Size
TechReady 23 9/21/2018 5:22 AM Premium Storage: Database Size Unallocated Unused Total DB Size Reserved Space Index Used Space Data Log © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
31
Premium Storage: Unallocated Space
TechReady 23 9/21/2018 5:22 AM Premium Storage: Unallocated Space 4TB Unallocated 12TB Reserved 4TB Reserved 12TB Unallocated 16 TB © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
32
Premium Storage: Snapshots
TechReady 23 9/21/2018 5:22 AM Premium Storage: Snapshots Frequency Retention 4 hours 7 days RPO : 8 Hours © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
33
Blob Storage: Geo-redundant backups
TechReady 23 9/21/2018 5:22 AM Blob Storage: Geo-redundant backups Frequency and Retention 1 Geo-backup 24hr RPO © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
34
Capping storage capacity
Microsoft Build 2016 9/21/2018 5:22 AM Capping storage capacity CREATE DATABASE MyDB COLLATE SQL_Latin1_General_CP1_CI_AS ( EDITION = 'DataWarehouse' , SERVICE_OBJECTIVE = 'DW400' , MAXSIZE = GB ); ALTER DATABASE MyDB MODIFY (MAXSIZE = GB); © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
35
Storage sizing summary
TechReady 23 9/21/2018 5:22 AM Storage sizing summary Database size: sp_spaceused Table sizing: DMVs Snapshot size: Total storage size (portal) – database size (sp_spaceused) Free space (unallocated): © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
36
Pricing
37
Compute pricing: DWU $900 Per DWU100 Per Month
38
Storage sizing: Premium storage
$122.88 TiB / Month Data files Log files Snapshots
39
Storage pricing: Geo-redundant backups
$0.12 GB / Month
40
Pricing scenarios Microsoft Build 2016 9/21/2018 5:22 AM GA price:
Storage: $122.88 DWU: $1.21 © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
41
Pricing scenarios Microsoft Build 2016 9/21/2018 5:22 AM
© 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
42
Summary
43
Summary of the session Session Objective(s): Key takeaways:
TechReady 23 9/21/2018 5:22 AM Summary of the session Session Objective(s): Elastic scale concepts Understand compute scaling factors Understand all components of the storage model Explain how compute and storage affects pricing with SQL Data Warehouse Key takeaways: Understand compute scaling and analyze memory requirements Compute scaling is the primary cost driver Elasticity empowers cost optimization and different utilization patterns Storage is not just data in the database © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
44
Free IT Pro resources To advance your career in cloud technology
Microsoft Ignite 2016 9/21/2018 5:22 AM Free IT Pro resources To advance your career in cloud technology Plan your career path Microsoft IT Pro Career Center Cloud role mapping Expert advice on skills needed Self-paced curriculum by cloud role $300 Azure credits and extended trials Pluralsight 3 month subscription (10 courses) Phone support incident Weekly short videos and insights from Microsoft’s leaders and engineers Connect with community of peers and Microsoft experts Get started with Azure Microsoft IT Pro Cloud Essentials Demos and how-to videos Microsoft Mechanics Connect with peers and experts Microsoft Tech Community © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
45
Please evaluate this session
9/21/2018 5:22 AM Please evaluate this session Your feedback is important to us! From your PC or Tablet visit MyIgnite at From your phone download and use the Ignite Mobile App by scanning the QR code above or visiting © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
46
9/21/2018 5:22 AM © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
Similar presentations
