Storage spaces direct Hyper converged

Storage spaces direct Hyper converged
Kent

Mikael Nystrom Johan Arwidmark
MVP MVP 8” 5 ¼” Mikael Steak Steak

Agenda Software Defined Storage and Hyper-Converged

Windows Server Management Marketing
11/12/2017 Industry trends What is Software-defined Storage (SDS) ? Software intelligence delivering feature-rich cloud scale storage and economics built on industry standard hardware Cloud-inspired infrastructure and design Using Industry-standard hardware Integrating cloud design points in software Driving cloud cost efficiencies Data explosion Device proliferation Modern apps Unstructured data analytics Evolving technologies Flash is transforming storage Network delivering extreme performance Maturity in software-based solutions Virtual machines and containers Scale out with simplicity Integrated solutions Rapid time to solution Policy-based management © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

What is a SAN, really? Connectivity adaptors Controllers
Resilient connectivity to external sources via iSCSI, FC, FCoE, NFS, SMB. Controllers The brains of the SAN—typically now with x86 CPU, memory, and provides enterprise features like thin provisioning, deduplication, storage tiering, etc. Multiple controllers provide resiliency. Physical disks Flash-based (SSD) or spinning media (HDD) to provide the raw storage capacity for your data. Pooled by the controllers, and sliced into LUNs (simple, mirrored, parity, etc.).

What about Microsoft and storage?
Connectivity adaptors Windows Server File Servers have resilient connectivity to external sources using regular 1GBE, 10GBE network adaptors. Support for up to 56GB, 100GB RDMA adaptors. Support via iSCSI, SMB 3.0, and NFS connectivity. Windows Server is now the controller Clustered Windows Server File Servers (SOFS) create disk pools, then slices them into storage spaces. Spaces can be thin provisioned, tiered, and support deduplication. Spaces can be simple, mirrored, or parity. Physical disks Multiple options for low cost and low complexity. HDD/SSD mix can exist in external JBOD shelf connected via SAS, or within the file server (controller) chassis itself.

Windows Server 2012 R2 architecture
One Marketing Template 11/12/2017 Windows Server 2012 R2 architecture 1 Industry-standard JBOD, filled with SSD and HDD on a 1:4 ratio. Additional JBODs added for capacity. 4 2 Up to 8 industry-standard x86 servers, running Windows Server 2012 R2, connected to JBOD via 6 GB/12 GB SAS. Scale-Out File Server 3 3 Build Windows Server cluster Create Storage pool Create Storage Spaces from pool Create Scale-Out File Server Create continuously available file shares on the spaces 2 SSDs & HDDs SSD 1 4 File shares provide storage for Hyper-V hosts, accessed over SMB 3.0. Highest performance delivered via SMB Direct (RDMA) and SMB Multichannel. Supports 56 GB+ speeds. © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Windows Server 2016—new architecture
One Marketing Template 11/12/2017 Windows Server 2016—new architecture Converged (disaggregated) architecture with Storage Spaces Direct Hyper-V cluster Industry standard x86 servers, with local SSD and HDD. Servers are connected together with 10GBE. SATA and NVMe drives supported. Build Windows Server cluster Enable Storage Spaces Direct Create Storage pool Create Storage Spaces from pool Create Scale-Out File Server Create Continuously Available file shares on the Spaces Optimize for Storage Spaces Direct 1 2 SMB storage fabric Storage Spaces Direct with Scale-Out File Server 1 File shares provide storage for Hyper-V hosts, accessed over SMB 3.0. Highest performance delivered via SMB Direct (RDMA) and SMB Multichannel. Supports 56 GB+ speeds. 2 Architecture allows for scaling Hyper-V clusters (compute) and Scale-Out File Server cluster (storage) independently © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

11/12/ :25 AM Storage Spaces Direct Software-defined storage using industry standard servers with local storage Hyper-V cluster Cloud design points and management Standard servers with local storage New device types such as SATA and NVMe SSD Prescriptive hardware configurations Deploy, manage, and monitor with SCVMM, SCOM, and PS Reliability, scalability, flexibility Fault tolerance to disk, enclosure, node failures Scale pools to large number of drives Simple and fine grained expansion Fast VM creation and efficient VM snapshots SMB storage fabric Storage Spaces Direct with Scale-Out File Server Use cases Hyper-V IaaS storage Storage for backup and replication targets Hyper-converged (compute and storage together) Converged (compute and storage separate) © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Windows Server 2016—new architecture
One Marketing Template 11/12/2017 Windows Server 2016—new architecture Hyper-converged storage and compute with Storage Spaces Direct Hyper-converged stack Hyper-V virtual machines Cluster share volumes ReFS file system Storage spaces Storage pools Software storage bus C:\Cluster storage SMB network Industry standard x86 servers, with local SSD and HDD. Servers are connected together with 10GBE. SATA and NVMe drives supported. Build Hyper-V cluster Enable Storage Spaces Direct Create Storage pool Create Storage Spaces from pool Create Cluster Shared Volumes Optimize for Storage Spaces Direct 1 Compute and storage resources scale and are managed together. Typically small to medium sized scale-out deployments. 2 © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Cosmos Darwin

Hardware

Building a 4 node cluster

RDMA or not RDMA?

The cache

11/12/ :25 AM Storage Quality of Service (QoS) Control and monitor storage performance Simple out-of-box behavior Enabled by default for Scale-Out File Server Automatic metrics per VHD, VM, host, volume Includes normalized IOPs and latency Scale-Out File Server cluster Policy Manager Hyper-V cluster Virtual machines Rate limiters I/O sched Flexible and customizable policies Policy per VHD, VM, service, or tenant Define minimum and maximum IOPs Fair distribution within policy Management System Center VMM and Ops Manager PowerShell built in for Hyper-V and SoFS © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Storage QoS policies Understanding policies
Define on Scale-Out File Server Apply to Hyper-V virtual disk The rest is automatic Policies Virtual machines Sample Policy Hyper-V cluster Name SilverVM PolicyID 8d f acf36 MinimumIOPs 100 MaximumIOPs 200 Type Multi-instance Rate limiters Rate limiters Rate limiters Rate limiters Scale-Out File Server cluster Silver policy Policy Manager I/O sched I/O sched I/O sched Gold policy

Types of Storage QoS policies
MaximumIOPs = 200 Single instance Resource distributed among VMs Ideal for representing a clustered workload, application, or tenant MaximumIOPs = 200 Multi-instance All VMs perform the same Ideal for creating per-VM performance tiers

Policies with PowerShell
11/12/2017 Policies with PowerShell # Deployment - Create policy (on File Server) New-StorageQosPolicy –CimSession FS -Name SilverVM -PolicyType MultiInstance - MaximumIops 200 # Deployment - Assign policy to VMs (on Hyper-V Host) $Policy = Get-StorageQosPolicy –CimSession FS -Name SilverVM Get-VM -Name VMName* | Get-VMHardDiskDrive | Set-VMHardDiskDrive –QoSPolicy $Policy # Monitoring - Retrieve all flows (on File Server) Get-StorageQosFlow # Monitoring - Retrieve flows using the policy (on File Server) Get-StorageQosPolicy -Name SilverVM | Get-StorageQosFlow Be sure to call out © 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Storage QoS monitoring
11/12/2017 Storage QoS monitoring Installed with the SOFS role If upgrading from 2012R2, enable it Tracks usage for all VMs On by default Virtual machines Available data VHD Path Storage Node Name VM Name VM Host Name Storage Node IOPS VM IOPS Storage Node Latency VM Latency Hyper-V cluster Rate limiters Rate limiters Rate limiters Rate limiters #Performance of all VMs using this file server Get-StorageQoSFlow #Performance of each volume on this file server Get-StorageQosVolume Have video Scale-Out File Server cluster Policy Manager I/O sched I/O sched I/O sched © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Storage Replica Protection of key data and workloads
Synchronous replication Storage agnostic mirroring of data in physical sites with crash- consistent volumes ensuring zero data loss at the volume level. Increase resilience Unlocks new scenarios for metro-distance cluster-to-cluster disaster recovery and stretch failover clusters for automated high availability. Complete solution End-to-end for storage and clustering, including Hyper-V, Storage Replica, Storage Spaces, cluster, Scale-Out File Server, SMB3, deduplication, Resilient File System (ReFS), NTFS, and Windows PowerShell. Streamlined management Graphical management for individual nodes and clusters through Failover Cluster Manager and Azure Site Recovery. Stretch cluster and cluster-to-cluster Site 1 Site 2

Storage replica Stretch cluster Cluster-to-cluster Single cluster
Manhattan DC Jersey City DC NODE1 in HVCLUS NODE2 in HVCLUS NODE3 in HVCLUS NODE4 in HVCLUS The Hudson River SR over SMB3 Stretch cluster Single cluster Automatic failover Synchronous Manhattan DC Jersey City DC NODE1 in FSCLUS NODE2 in FSCLUS NODE3 in FSCLUS NODE4 in FSCLUS NODE1 in DRCLUS NODE2 in DRCLUS NODE3 in DRCLUS NODE4 in DRCLUS The Hudson River SR over SMB3 Cluster-to-cluster Two separate clusters Manual failover Synchronous or asynchronous

Storage tiering for Spaces
Optimizing storage performance on Storage Spaces Disk pool consists of both high performance SSDs and higher capacity HDDs Hot data is moved automatically to SSD and cold data to HDD using sub-file-level data movement With write-back-caching, SSD absorb random writes that are typical in virtualized deployments Admins can pin hot files to SSDs manually to drive high performance New PowerShell cmdlets are available for the management of storage tiers Storage Space Cold data Hot data HDD tier—4 TB, 7200 RPM SAS SSD tier—400 GB EMLC SAS SSD SSD

Storage Spaces reliability
One Marketing Template 11/12/2017 Storage Spaces reliability Mirror resiliency 2-copy mirror, tolerates one drive failure 3-copy mirror, tolerates two drive failures Suitable for random I/O Parity resiliency Lower cost storage using LRC encoding Tolerates up to 2 drives failures Suitable for large sequential I/O Enclosure awareness Tolerance for entire drive enclosure failure Parallel rebuild Pseudo-random distribution weighted to favor less used disks Reconstructed space is spread widely and rebuilt in parallel Drive Failure Mirror space Parity space Data copy 1 Data copy 2 © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Storage Quality of Service (QoS) Building blocks
11/12/ :25 AM Storage Quality of Service (QoS) Building blocks Profiler and rate limiter on Hyper-V compute nodes 1 Virtual machines I/O scheduler distributed across the storage nodes 2 Hyper-V cluster 1 Rate limiters Rate limiters Rate limiters Rate limiters Centralized Policy Manager on Scale-Out File Server cluster 3 Scale-Out File Server cluster 2 3 Policy Manager I/O sched I/O sched I/O sched © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Responding to changing demand The policy process
1. Measure current capacity at the compute layer Virtual machines 2. Measure current capacity at the storage layer Hyper-V cluster 4 1 Rate limiters Rate limiters Rate limiters Rate limiters 3. Use algorithm to meet policies at the Policy Manager Scale-Out File Server cluster 2 4. Adjust limits and enforce them at the compute layer 3 Policy Manager I/O sched I/O sched I/O sched

Storage Replica Synchronous and asynchronous modes
One Marketing Template 11/12/2017 Storage Replica Synchronous and asynchronous modes Mode Deployment Diagram Steps Synchronous Zero data loss RPO Mission critical apps On-premises or metro setup Short distance (<5 ms, more likely <30 km) Usually dedicated link Bigger bandwidth Application write Log data written and the data is replicated to remote site Log data written at the remote site Acknowledgement from the remote site Application write acknowledged t, t1: Data flushed to the volume, logs always write through Asynchronous Near zero data loss (depends on multiple factors) RPO Non-critical apps Across region/country Unlimited distance Usually over WAN Log data written Data replicated to the remote site t, t1: Data flushed to the volume, logs always write through Applications (primary) Server cluster (SR) 1 5 t 2 Log Data 4 Applications (remote) t1 3 Applications (primary) Server cluster (SR) 1 3 t 2 Log Data 4 6 Applications (remote) t1 5 © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Recommendations for synchronous
One Marketing Template 11/12/2017 Recommendations for synchronous Network latency ≤5 ms round trip average Assuming the light speed vacuum ideal, 5 ms is ~1,500 km round trip Reality: Optical fiber reduces by ~35%, you cross switches, routers, firewalls, etc. Financial limits, availability End result: Most customers end up 30–50 km Network bandwidth ≥1 Gbps network—end-to-end—between servers is a starting point (Windows Server logo requires 1 GB NIC) It depends on your IO and sharing of the pipe (SR may not be the only traffic for the DR site) Learn your IOPS math (125 MB/s of IO = ~1 GB/s network usage) Log volume performance and size Flash (SSD, NVME, etc.) Larger logs allow faster recovery from larger outages and less rollover, but cost space © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Traditional SAN or Microsoft SDS
TechEd 2013 11/12/ :25 AM Traditional SAN or Microsoft SDS Traditional SAN Microsoft SDS Block protocol fabric Copy offload, snapshots Low latency network with FC Storage tiering Management of LUNs Persistent write-back cache Data deduplication Scale up RAID resiliency groups Storage QoS Pooling of disks Replication High availability Firmware updates File protocol fabric SMB copy offload, snapshots Low latency with SMB3Direct Performance with tiering Persistent write-back cache Management of shares Automatic scale-out rebalancing Data deduplication Storage QoS Flexible resiliency options Storage Replica Rolling cluster upgrades Pooling of disks Storage Spaces Direct Continuous availability Azure-consistent storage NEW IN R2 NEW IN 2016 Hyper-V compute nodes FC/SAS disk shelf SAN/NAS FC/iSCSI fabric (block) Hyper-V compute nodes Shared SAS JBOD or DAS Scale-Out File Server with Storage Spaces SMB3 fabric (file) © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Storage spaces direct Hyper converged

Similar presentations

Presentation on theme: "Storage spaces direct Hyper converged"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Storage spaces direct Hyper converged

Similar presentations

Presentation on theme: "Storage spaces direct Hyper converged"— Presentation transcript:

Similar presentations

About project

Feedback