1. SQL Storage Fundamentals for Azure IaaS
Bob Duffy, MCA, MCM, MVP

2. Bob Duffy 22 years in database sector, 250+ projects
SQL Server MCA, MCM, MVP
SSAS Maestro
Senior Data Platform Consultant with Microsoft
Database Architect at Prodata SQL Centre of Excellence

3. Agenda Azure Storage Fundamentals Azure Sizing and Throttling
Optimising Storage Configuration
Storage Requirements and Capacity Planning
SQL Configuration
Storage Benchmarking

4. What is Azure Storage Takeaways: Dont worry about RAID 1+
Geo redundancy/replication
Takeaways:
Dont worry about RAID 1+
Dont do geo-replication for SQL IaaS
Write penalty (expect WRITE_LOG)
LRS
ZRS
GRS
RA-GRS
Total Copies 3 6
SLA
99.9%
99.99% read
99.9% write
Warning: No write ordering on geo replication

5. Storage Choices ? VHD on HDD VHD on SSD Direct to Blob Store
Throughput MB/Sec 60
200
Max IOPS
500
5,000
Latency
10-40ms
<5ms
???
Cost per TB/Month €
17.28
113.99
VM SLA
None
99.9%
CREATE DATABASE MyAzureDB ON
( NAME = MyAzureDBData,
FILENAME = ' )
LOG ON
( NAME = MyAzureLog,
FILENAME = '

6. Storage Host Caching
Discuss – why do not care so much about storage cache?
Best Practise

7. VM Storage Limits (SSD)

8. SQL IO Characteristics (AVG IO Size)
Workload
Type
IO Size
Log Writes
Sequential
Up to 60k
Checkpoint
Random
Up to 256k
Table Scan
64-512K (on EE)
Index Seek 8k
64k (read ahead)
Backup
64k-4MB
500 = 4MB/sec
500 = 31MB/sec
MB/Sec
80,000 64k = 5GB/Sec
Conclusion – ignore iop limit on ssd vms and worry about throughput !

9. VM Throughput Limits DS-Series, ACU: 210-250 Cores RAM VM “IOPS”
Max MB/sec
Effective
VM Cost €/Month in sql EE 2 14
6,400 96
1,536
119 4 28
12,800
192
3,072
237 8 56
25,600
384
6,144
476 16
112
50,000*
768
12,288
954 20
140
64,000
960
15,360
1,190
48 MB/Sec per core
GS-Series, ACU: 2 28
5,000
125
2,000
439 4 56
10,000
250
4,000
878 8
112
20,000
500
8,000
1,757 16
224
40,000
1,000
16,000
3,514 32
448
80,000
32,000
6,267
€1.23 euro per MB/Sec
62.5 MB/Sec per core

10. The Three Levels of Throttling
STORAGE ACCOUNT 20,000 iops or 1,250 MB/Sec
4 Core VM
386 MB/Sec
8 Core VM
768 MB/Sec
32 Core VM
2,000 MB/Sec
1TB SSD 200 MB/Sec
1TB SSD 200 MB/Sec
1TB SSD 200 MB/Sec
X 2
X 4
X 10
Also 10 Gbps

11. Managed Disks Eliminates limits of storage account ;-) P10 P20 P30
Eliminates limits of storage account ;-)
P10
P20
P30
Disk Size (GB)
128
512
1,204
IOPS
500
2,300
5,000
Throughput
100
150
200
Effective Throughput 8
143
200+ (313)
Cost €/month 18 68
125
Cost € per MB/Sec
2.25
0.46
0.625
Cost € per TB
144
136
P10 is only 8 MB/Sec

12. An Example of designing storage for elasticity
We need a 3TB SQL but want to “max” storage performance on an 8 core server and burst up to 20 core at night
8 Core VM
384 MB/Sec
1TB SSD 200 MB/Sec
1TB SSD 200 MB/Sec
1TB SSD 200 MB/Sec
600 MB/Sec
20 Core VM
960 MB/Sec
512GB SSD 150 MB/Sec
X 6 = 900 MB/Sec

13. Azure Portal Storage Configuration Wizard
Advice: configure the disks yourself

14. Adding and Combining Disks
Combine azure disks using storage spaces
Interleave defaults to 256Kb (data warehouse)
May need 64k (OLTP)
set column count = number of physical disks
For > 8 disks need to use powershell
Adding disks to VMs
$diskConfig = New-AzureRmDiskConfig -AccountType $storageType -Location $location -CreateOption Empty -DiskSizeGB $diskSizeGB
$vm = Add-AzureRmVMDataDisk -VM $vm -Name $myDiskName -CreateOption Attach -ManagedDiskId $dataDisk1.Id -Lun $i
Creating storage spaces
New-StoragePool –FriendlyName SQLVMStoragePool2 –StorageSubsystemFriendlyName "Windows Storage*" -PhysicalDisks (Get-PhysicalDisk -CanPool $True)
New-VirtualDisk -StoragePoolFriendlyName SQLVMStoragePool2 -FriendlyName SQLVMVirtualDisk2 `
-Interleave 256KB -NumberOfColumns 1 -UseMaximumSize -ResiliencySettingName simple

15. The Magnificent Seven Capacity Planning Metrics
Perfmon
dm_io_virtual_file_stats
IOPS
Logical Disk\Reads/Sec
Logical Disk\Writes/Sec Y
OLTP, DW, Mixed
Logical Disk\ Avg Disk Bytes/Read
Kinda
Throughput
Logical Disk\Read Bytes/Sec
Size of data, log, tempdb
MSSQL$:Databases(x)\Data File Size (KB)
MSSQL$:Databases(x)\Log File Size (KB)
% of workload in temp
Derived
RW Ratio
Latency
Logical Disk\Avg Disk Secs/Read
[sys].[dm_io_virtual_file_stats]

16. Capacity Planning Tools
Tools to capture metrics
Tools to analyse metrics

17. SQL Configuration - Tempdb
The usual practices apply
Pre-size and set growth on tempdb
Use multiple files
Only use T1117 and T1118 before SQL 2016
Tune to lower tempdb utilisation
Should we use Local SSD – Maybe ;-)
Side effect of permissions/startup issues
Can reduce latency on write intensive tempdb
Only one local disk and limited size

18. SQL Configuration - Data and Log Files
Create Two disk pools (Log, data+tempdb)
Best for isolation and recovery
Does log NEED dedicated IOPS
One large disk pool (one logical drive)
Best for maximising throughput
Separate Logical drives (F: and L:)
Compromise
Normal File and Filegroup apply in IaaS
Avoid use of primary group
Use File groups for data warehouses:
Partial recovery
Reduce fragmentation => increase IO size
Hot/cold data=>one per table=> one per partition
Use multiple files to improve concurrency in OLTP
Set proper file sizing and growth
Data
Log
Data + Log
Data
Log

19. SQL Configuration – Other Settings to improve Storage Performance
Lock Pages in Memory
Instant File Initialisation
Use these features as appropriate
Page Compression
Column Store Indexes
Delayed Durability
In Memory Tables
Stretch Tables

20. Benchmarking Use diskspd.exe to validate throughput and or IOPS
Consider making a IO saturation chart
Run diskspd as ever increasing thread and outstanding IO
If you are lazy use CrystalDiskMark

21. 8 Core single 1TB Disk

22. 8 Core single 1TB Disk x 2

23. 16 Core VM – 4 x 1TB disks

24. Lessons Learned Worry more about throughput than IOPS
Choose your own disks to get target throughput
Build your own disk pools
Plan your VM Size based on throughout needed
Wish List for Azure
Provisioned IOPS!
Fixing the bugs with tempdb on local SSD

25. Meet me at the Community Zone
Thank You ! – 4 x 1TB disks
Meet me at the Community Zone
After this session, you can speak with me in the Community Zone
WE MIGHT
Discuss additional questions
Review parts of my session in more detail
Network
Take selfies… 
Play Jenga and drink beer
IMPORTANT : ONLY SHOW THIS SLIDE IF YOU HAVE COMMITED TO BE AT THE COMMUNITY ZONE

26. Please evaluate all sessions! QR / LINK on posters and in program

27. THANKS to all Sponsors! EVENT SPONSORS EXPO SPONSORS
EXPO LIGHT SPONSORS