WS2012 File System Enhancements: ReFS and Storage Spaces Rick Claus Sr. Technical WSV316

Ease of storage administration  Storage Management PowerShell with SMI-S integration Minimize Op-Ex Efficient capacity utilization  Thin provisioning integrationMaximize Existing Hardware Dynamic data movement  Offloaded data transfers Storage virtualization and resiliency  Storage SpacesMinimize Cap-Ex Maximize Availability Next-generation scale, availability, and integrity  Resilient File System (ReFS) Maximize data availability  Modern NTFS CHKDSK BenefitCapabilityFeature

Cost-efficient Storage (New)Traditional Storage Block Access File Access JBODS App Servers SAS JBODS CA File Servers SAS App Servers File \db App Servers FC iSCSI CA File Servers FC iSCSI App Servers File \db

Storage Spaces Storage Virtualization and Resiliency

Windows Server 2012 Capabilities File and Storage Manager Failover Clustering & CSV v2 ReFS & NTFS Hyper-V SMB v3 RDMA Storage Mgmt. PowerShell Virtualized Resilient Storage Mirror Space Parity Space Mirror Space Parity Space Mirror Space Parity Space Cost-efficient shared SAS JBOD arrays Physical or Virtualized Servers

OR SAS Shared SAS

Optimized and flexible capacity utilization Resiliency to drive failure Operational simplicity with support for multi-tenancy High-availability and scalability Workload-optimized performance via heterogeneous media

60TB Space Company A Storage Pool 60TB Space Company A Storage Pool Thinly provisioned storage spaces Industry standard commodity storage 36 TB across 72 disks48 TB across 96 disks

Mirror Space Parity Space Mirror Space Parity Space Mirror Space Resilient Spaces Hot Spare Industry standard storage

Mirror Space Parity Space Simple Space Workload Admin Tenant Admin Company A Pool Mirror Space Parity Space Simple Space Workload Admin Tenant Admin Company B Pool Hoster Admin Granular management control Multi-tenant deployment

Industry standard commodity storage Diverse spaces for diverse workloads 15K RPM Home Space Silver 7200 RPM Archive Space Bronze Solid state Finance Space ERP Space Gold

Space Storage spaces with a diverse set of workloads Standalone Deployment

Standalone Dell R710 server with dual LSI e SAS adapters Dell MD bay JBOD array with 300GB 15K RPM disks

Space Clustered Storage Spaces \CRM Unified Cluster Shared Volume Namespace Enterprise Workloads 2-4 servers Shared SAS storage Deployment Element

Physical View (64 servers max) Space Clustered Pools and Storage Spaces \Finance\CRM\File-Shares Unified Cluster Shared Volume Namespace Logical View (Highly available, scalable, cost- efficient) Enterprise Workloads.\db.\VHD VMs.\Home CA File Server Deployment Element 2-4 servers Shared SAS storage Deployment Element

60-bay SAS array 60-bay SAS array 60-bay SAS array 60-bay SAS array 3TB Near-line SAS HDDs 720TB High-Cap 720TB High-Cap 600GB 15K SAS HDDs 144TB Mid-Perf 144TB Mid-Perf 400GB SAS SSDs 96TB High-Perf 96TB High-Perf

ItemCount Deployment Elements16 Servers per Element4 Total number of servers64 JBODs per Element1 Total number of disks242 HDDs and SSDs 40Gbps networks per server4 Virtual Machines4000

Resilient File System (ReFS) Next-generation scale, availability, and integrity

All metadata corruption is always detected. Optionally, user data corruption is also detected. ReFS utilizes Storage Spaces’ resiliency to auto-correct. Integrity ReFS efficiently scales to PB datasets comprising very large files. Scalability

ReFS maximizes data availability. If corruption occurs and a redundant copy is unavailable, the non-corrupt portions remain unaffected and online. Availability Architecture enables efficient evolution for new storage devices and new access patterns. Architectural Evolution

Allocate-on-write eliminates corruption from errors associated with in-place update Checksums detect whether the data on media has changed since it was last written File system metadata is always protected, maximizing online operation Flexible user data protection is configurable on a per-volume, per-directory, and per-file basis

Data Volume - NTFS Scrubber Data Volume - ReFS Storage Pool Data Copy0 Data Copy1 Data Copy2 On media failure, NTFS will get an alternate copy of the data With NTFS, Scrubber verifies all copies can be read Checksum is inspected on all copies of the data If corruption detected, trigger auto-correct and return a healthy copy of the data With ReFS, Scrubber verifies checksums on all copies and auto-corrects as needed Copy1 Data Copy2 Data Copy0 Data Mirror Space

Application accesses data ReFS volume detects corruption and automatically replaces the corrupt data with correct data Mirror space offers all data copies to the file system ReFS

ReFS uses a highly scalable, flexible, B+ tree data structure for all metadata Ensures maximum scalability and volume sizes beyond NTFS File system abstractions are implemented using key-value tables built from the B+ trees Capacity allocation is handled by a hierarchy of allocators Ability to allocate at different sized granularities

NTFS.SYS Upper API Layer On-media Store ReFS.SYS Upper API Layer MinStore

Legacy  Short Names  TxF NTFS-specific  Reparse Index  Named Streams  MFT Superseded  NTFS Encryption  NTFS Quotas  NTFS Compression Other  Hard Links  Extended Attributes  Object IDs

Deployment Element 2-4 servers Shared SAS storage Space Clustered Storage Spaces Home Dirs VHD Library Mark’t Dept. ReFS volumes with integrity streams \Docs\Home\VHDs Highly Available File Server VHDs

Ease of storage administration  Storage Management PowerShell with SMI-S integration Minimize Op-Ex Efficient capacity utilization  Thin provisioning integrationMaximize Existing Hardware Dynamic data movement  Offloaded data transfers Storage virtualization and resiliency  Storage SpacesMinimize Cap-Ex Maximize Availability Next-generation scale, availability, and integrity  Resilient File System (ReFS) Maximize data availability  Modern NTFS CHKDSK BenefitCapabilityFeature

Virtualizing Storage for Scale, Resiliency, and Efficiency How to Configure Clustered Storage Spaces in Windows Server Storage Spaces FAQ