MDC323B SMB 3 is the answer Ned Pyle Sr. PM, Windows Server © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Agenda Why SMB 3? SMB 3 in WS 2012 What’s new in SMB 3.02 for WS 2012 R2 What SMB 3 brings to real life workloads Troubleshooting SMB 3

Why SMB 3?

SMB lineage in Microsoft SMB 1 – LANMan-XP SMB 2 – Vista/2008 SMB 2.1 – 7/2008 R2 SMB 3.0 – 8/2012 SMB 3.02 8.1/2012 R2 BTW: You can remove SMB 1 in WS 2012 R2        

Windows Server 2012 brought a new vision Dramatically lowering the costs and effort of delivering Infrastructure as a Service storage services Separate compute and storage Independently manage and scale at each layer Industry standard servers, networking and storage Inexpensive networks Inexpensive shared JBOD storage Hyper-V Clusters SMB Scale-Out File Server Clusters Storage Spaces Virtualization and Resiliency Shared JBOD Storage

This required a new storage stack Windows Server 2012 Storage Spaces 64-node clusters SMB3 Scale-Out File Server 8,000 VMs per Cluster Offload Data Transfer Dedup Hyper-V Replica Virtual Fibre Channel VM Prioritization ReFS Cluster-Aware Updating iSCSI Target Server VM Storage Migration VHDX SM API SMI-S Storage Service NFS 4.1 NTFS Trim / Unmap CSV online CHKDSK

SMB 3 mandates Enable the IaaS compute and storage scenario Become the go-to remote file protocol for business Modernize for information workers One ring to rule them all!

The industry is onboard EMC and NetApp have SMB 2 and subset SMB 3 implementations now http://www.emc.com/collateral/white-papers/h11383-vnxe-introduction-wp.pdf http://www.netapp.com/us/media/tr-4172.pdf “NetApp is committed to providing support for SMB 3.0” – NetApp, 2012 “[We have a] commitment to support Windows Server 2012 - including Server Message Block 3.0” – EMC, 2012 Even Apple prefers SMB now http://images.apple.com/osx/preview/docs/OSX_Mavericks_Core_Technology_Overview.pdf “SMB2 is superfast, increases security, and improves Windows compatibility.” – Apple, 2013

The workloads

Historically, FS cluster failover disrupts File Server Cluster File Server Node A File Server Node B \\fs1\share 1 2 3 SQL Server Failover share and connections and handles lost Normal operation Administrator intervention needed to recover Failovers are not transparent Works ok for traditional file server Most user client software resumes gracefully Server applications expect continuously available storage Connection and file handles are lost on share failover Application disruption Administrator intervention required to recover

Hyper-V over SMB What is it? Store Hyper-V files in shares over the SMB 3.0 protocol Standalone and cluster File storage used as cluster shared storage Hyper-V Cluster Hyper-V Hyper-V Hyper-V Hyper-V Hyper-V Hyper-V Hyper-V Hyper-V Hyper-V SQL Server SQL Server SQL Server IIS IIS IIS VDI Desktop VDI Desktop VDI Desktop File Server Cluster File Server File Server Shared Storage

SQL over SMB What is it? Store SQL database files in shares over the SMB 3.0 protocol Standalone and cluster File storage used as cluster shared storage SQL Cluster Hyper-V Hyper-V Hyper-V Hyper-V SQL Server SQL Server Hyper-V SQL Server Hyper-V SQL Instance SQL Instance SQL Instance SQL Instance SQL Instance SQL Instance SQL Instance SQL Instance SQL Instance File Server Cluster File Server File Server Shared Storage

Advantages Increases flexibility Eases provisioning, management and migration Leverages converged network Reduces capital and operational expenses

How SMB gets you there

SMB Transparent Failover Failover transparent to server application Zero downtime – small IO delay during failover Supports planned and unplanned failovers Hardware/Software Maintenance Hardware/Software Failures Load Rebalancing Resilient for both file and directory operations Requires File Servers configured as Windows Failover Cluster Windows Server 2012 on both the servers running the application and file server cluster nodes Shares enabled for “continuous availability” (default configuration for clustered file shares) Works for both classic file server clusters (cluster disks) and scale-out file server clusters (CSV) 1 Normal operation Failover share - connections and handles lost, temporary stall of IO 2 Connections and handles auto-recovered Application IO continues with no errors 3 Hyper-V 1 3 \\fs\share \\fs\share 2 File Server Cluster File Server Node A File Server Node B

SMB Scale-Out Targeted for server app storage Key capabilities Example: Hyper-V and SQL Server Increase available bandwidth by adding nodes Leverages Cluster Shared Volumes (CSV) Key capabilities Active/Active file shares Fault tolerance with zero downtime Fast failure recovery CHKDSK with zero downtime Support for app consistent snapshots Support for RDMA enabled networks Optimization for server apps Simple management Hyper-V Cluster (Up to 64 nodes) Datacenter Network (Ethernet, InfiniBand or combination) Single Logical File Server (\\FS\Share) Single File System Namespace Cluster Shared Volumes File Server Cluster (Up to 8 nodes)

SMB Direct (SMB over RDMA) Advantages Scalable, fast and efficient storage access High throughput , low latency, minimal CPU Load balancing, automatic failover and bandwidth aggregation via SMB Multichannel Scenarios High performance remote file access for application servers Required hardware RDMA-capable network interface (R-NIC) Three types: iWARP, RoCE and InfiniBand RDMA NICs use SMB Multichannel, not teaming SMB Client SMB Server Application User Kernel SMB Client SMB Server Network w/ RDMA support Network w/ RDMA support NTFS SCSI R-NIC R-NIC Disk NIC Throughput 1Gb Ethernet ~0.1 GB/sec 10Gb Ethernet ~1.1 GB/sec 40Gb Ethernet ~4.5 GB/sec 32Gb InfiniBand (QDR) ~3.8 GB/sec 56Gb InfiniBand (FDR) ~6.5 GB/sec HBA Throughput 3Gb SAS x4 ~1.1 GB/sec 6Gb SAS x4 ~2.2 GB/sec 4Gb FC ~0.4 GB/sec 8Gb FC ~0.8 GB/sec 16Gb FC ~1.5 GB/sec

SMB Multichannel Full Throughput Automatic Failover Bandwidth aggregation with multiple NICs Multiple CPUs cores engaged when NIC offers Receive Side Scaling (RSS) Automatic Failover SMB Multichannel implements end-to-end failure detection Leverages NIC teaming (LBFO) if present, but does not require it Automatic Configuration SMB detects and uses multiple paths Requires least one of these configs Multiple network adapters One or more NICs that support RSS One or more NICs configured with teaming One or more NICs that support RDMA Sample Configurations Single 10GbE RSS-capable NIC Multiple 1GbE NICs Multiple 10GbE in LBFO team Multiple RDMA NICs SMB Client SMB Client SMB Client SMB Client LBFO NIC 10GbE NIC 1GbE NIC 1GbE NIC 10GbE NIC 10GbE NIC 10GbE/IB NIC 10GbE/IB Switch 10GbE Switch 1GbE Switch 1GbE Switch 10GbE Switch 10GbE Switch 10GbE/IB Switch 10GbE/IB SMB Server NIC 10GbE SMB Server NIC 1GbE NIC 1GbE SMB Server NIC 10GbE NIC 10GbE SMB Server NIC 10GbE/IB NIC 10GbE/IB LBFO Vertical lines are logical channels, not cables

SMB Encryption & Signing End-to-end encryption of SMB data in flight Protects data AES-CCM SMB Signing updated Prevents packet tampering AES-CMAC Both leverage AES-NI CPUs (Nehalem+) for better performance Note required, but highly recommended No need for IPSec or WAN accelerators Configured per share or server Application workload over unsecured networks Branch Offices over WAN networks Another good reason to remove SMB 1 Server Client SMB Encryption

Demo

What about SMB 3.02?

Automatic Scale-Out Rebalancing Hyper-V host SoFS clients redirected to the “best” node for access to a specific share Avoids unnecessary redirection traffic Driven by ownership of Cluster Shared Volumes SMB connections managed per share, not server Clients move as CSV volume ownership changes Clustering now balances CSV automatically Automatic behavior, no administrator action \\SOFS\Share1 \\SOFS\Share2 SMB Scale-out File Server File Server 1 File Server 2 Share1 Share2 Share1 Share2 Storage Spaces

SMB Direct v2 Performance 50% improvement for small IO workloads with SMB over RDMA Increased 8KB IOPs from ~300K IOPS to ~450K IOPS per interface Efficiency Increased efficiency and density of hosting workloads with small I/O’s such as OLTP database in a VM Optimizes 40Gbps Ethernet and 56Gbps InfiniBand

Hyper-V Live Migration over SMB SMB as a transport for Live VM Migration RDMA (SMB Direct) Streaming over multiple NICs (SMB Multichannel) Provides highest bandwidth and lowest latency and CPU TCP/IP Compression SMB w/RDMA (no compression)

SMB Bandwidth Management File Server for library Storage Common Infrastructure SMB leveraged for VMs to access storage, distribution from VM library, and live migration Desire to manage bandwidth of different types of SMB communication Live Migration Limit = 500 MB/s Default Limit = 100 MB/s Hyper-V host 1 Hyper-V host 2 Control Configurable SMB bandwidth limits per category Default, VirtualMachine and LiveMigration Storage No Limit Scale-out File Server VHDX

Multiple SMB Instances Hyper-V Host 1 Hyper-V Host 2 Additional instance on each node in a SoFS for CSV traffic Default instance handles incoming traffic from SMB clients accessing shares Other instance handles only inter-node CSV traffic (metadata access or redirected traffic) Separate data structures (locks/queues) for regular client traffic and inter-node traffic Improves scalability and reliability of inter-node traffic between CSV nodes SMB Client SMB Client Scale-Out File Server File Server 1 File Server 2 Default Instance SMB Server SMB Server CSV Instance SMB Client SMB Client SMB Server CSV Instance Default Instance SMB Server CSV1 (Metadata Owner) CSV2 (Not Metadata Owner) CSV1 (Not Metadata Owner) CSV2 (Metadata Owner) Shared SAS Storage

Hyper-V over SMB (this is important) Hyper-V supports SMB version 3.0 only WS2012 and later A few 3rd parties have a subset of SMB 3 now HVBPA will confirm Active Directory is required Continuously Available shares recommended Virtual Machine Manager 2012 SP1 and 2012 R2 support Hyper-V over SMB File Server and Hyper-V must be separate servers If using Failover Clusters, File Server and Hyper-V must be on separate clusters

IW and SoFS (this is also important) Scale-out file server designed for App workloads Long running data operations Few metadata operations Write-through requirement Traditional Information Worker Many features incompatible with CSV, SoFS Opening files, closing files, creating new files, or renaming existing files are slower on SoFS User client apps often very resilient to brief interruption Still should cluster, just not as continuous availability!

File Server Configurations Dual-node File Server Low cost for continuously available shared storage Limited scalability (up to a few hundred disks) Multi-node File Server Highest scalability (up to thousands of disks) Higher cost, but still lower than connecting all Hyper-V hosts with FC Single-node File Server Lowest cost for shared storage Shares not continuously available Hyper-V Parent 1 Hyper-V Parent N Hyper-V Parent 1 Hyper-V Parent N Hyper-V Parent 1 Hyper-V Parent N Child 1 Child N Child 1 Child N Child 1 Child N Config Config Config Config Config Config Disk Disk Disk Disk Disk Disk VHD VHD VHD VHD VHD VHD File Server File Server 1 File Server 2 FS 1 FS 2 FS 3 FS 4 Share1 Share2 Share1 Share2 Share1 Share2 Share1 Share2 Share3 Share4 Disk Disk Shared SAS Storage Fibre Channel Storage Array Disk Disk Disk Disk Disk Disk Disk Disk Disk Disk Disk A B C

Network Configurations All 1GbE Mixed 1GbE/High Speed All High Speed (10GbE/40GbE/56GbIB) Clients Clients Clients 1GbE 1GbE 1GbE 1GbE High Speed High Speed Clients Hyper-V 1 Hyper-V 2 Hyper-V 1 Hyper-V 2 Hyper-V 1 Hyper-V 2 High Speed High Speed Hyper-V 1 Hyper-V 2 1GbE 1GbE High Speed High Speed High Speed High Speed File Server 1 File Server 2 File Server 1 File Server 2 File Server 1 File Server 2 File Server 1 File Server 2 A D B C

Enterprise IW

IW Scenario The cloud is coming But our new name is Cloud & Enterprise The on-premise information worker is still king Businesses still run on home folders and unstructured data shares, to the tune of 800PB and growing Windows 7 is an unstoppable juggernaut

Our WS2012 R2 IW File Server stack TechReady 17 4/25/2017 Our WS2012 R2 IW File Server stack SMB 2.X/3.X Encryption Signing File Leasing Directory Leasing Energy efficiency SMB Multichannel Durable handles Work Folders AD RMS DFSR DFSN iSCSI NFS Dynamic Access Control Storage Spaces StorSimple BranchCache & CSC DirectAccess File Classification Infrastructure Traditional Cluster © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

SMB2.1 is no slouch for end users HP StoreEasy 5000 results using WS2012+Windows 7

File Server Capacity Tool 1.2 Validate and compare hardware Locate existing bottlenecks Planning for expansion in advance of exhaustion Throughput capacity Maximum number of operations per second Maximum number of users supported http://www.microsoft.com/en-us/download/details.aspx?id=27284

Troubleshooting SMB3 in WS2012 R2

Improved SMB Diagnosibility in WS2012 R2 Consistent design philosophy Scenario-based events containing inter-machine info Useful events on by default Include details on configuration and troubleshooting guidance. Less noisy event don’t wrap often Common misconfigurations logged at boot up

Peel the onion (expect some tears) Is it enabled? SMB Windows PowerShell Client and Server It’s often not SMB Application? Storage? When it is SMB, it’s often still not SMB Network? Security? Use all the logs in the stack SMB, System, CSV, Spaces, Cluster Message Analyzer Vendor? Make them prove themselves

Message Analyzer Time to learn a new tool  But it’s a really great tool!  Download from: http://connect.microsoft.com/site216 Designed with SMB3 troubleshooting in mind

Demo

Net cap or it didn’t happen! TechReady 17 4/25/2017 Net cap or it didn’t happen! SMB2_GLOBAL_CAP_DFS - server supports the Distributed File System. SMB2_GLOBAL_CAP_LEASING - server supports leasing. SMB2_GLOBAL_CAP_LARGE_MTU - Connection.SupportsMultiCredit is TRUE. SMB2_GLOBAL_CAP_MULTI_CHANNEL - Connection.Dialect is "3.000", IsMultiChannelCapable is TRUE, and SMB2_GLOBAL_CAP_MULTI_CHANNEL set SMB2_GLOBAL_CAP_DIRECTORY_LEASING if Connection.Dialect is "3.000", server supports directory leasing, and SMB2_GLOBAL_CAP_DIRECTORY_LEASING set SMB2_GLOBAL_CAP_PERSISTENT_HANDLES if Connection.Dialect is "3.000", SMB2_GLOBAL_CAP_PERSISTENT_HANDLES set, and server supports persistent handles. SMB2_GLOBAL_CAP_ENCRYPTION if Connection.Dialect is "3.000", server supports encryption, and SMB2_GLOBAL_CAP_ENCRYPTION is set © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Don’t disable SMB2/3 I mean it, don’t disable SMB2/3 Really Seriously, don’t do it Client IW issues in metadata caching, not all SMB2/3 HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Lanmanworkstation\Parameters Try setting FileNotFoundCacheLifetime to ZERO Try setting FileInfoCacheLifetime to ZERO Try setting DirectoryCacheLifetime to ZERO http://technet.microsoft.com/en-us/library/ff686200(v=WS.10).aspx All hurt perf, and directorycachelifetime really hurts perf – think SMB1 Ok, if you have to disable, use it as a test, not a solution

4/25/2017 5:28 PM Related sessions MDC331A - Storage Spaces: What's New in Windows Server 2012 R2 MDC211 - Introduction to Windows Server 2012 R2 MDC325C - Windows Server 2012 Storage Efficiencies Demonstrated © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Related content Windows File Server Stack Tech Ready 15 4/25/2017 Related content Windows File Server Stack Windows File Server Team - http://blogs.technet.com/filecab Jose Barreto - http://blogs.technet.com/josebda What’s new in WS2012 R2 - http://technet.microsoft.com/en-us/library/dn250019.aspx WS2012 R2 - http://blogs.technet.com/b/josebda/archive/2013/06/26/updated-links-on-windows-server-2012-r2-file-server-and-smb-3-0.aspx WS2012 - http://blogs.technet.com/b/josebda/archive/2013/05/05/updated-links-on-windows-server-2012-file-server-and-smb-3-0.aspx SOFS- http://technet.microsoft.com/en-us/library/hh831349.aspx DFSR - http://technet.microsoft.com/en-us/library/dn281957.aspx NFS - http://technet.microsoft.com/en-us/library/jj592688.aspx iSCSI - http://technet.microsoft.com/en-us/library/dn305893.aspx . © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Related content Windows File Server Stack Tech Ready 15 4/25/2017 Related content Windows File Server Stack DAC - http://technet.microsoft.com/en-us/library/hh831717.aspx Work Folders - http://technet.microsoft.com/en-us/library/dn265974.aspx AD RMS - http://technet.microsoft.com/en-us/library/cc771234(v=ws.10).aspx FCI - http://technet.microsoft.com/en-us/library/dd758761(v=ws.10).aspx BranchCache - http://technet.microsoft.com/en-us/library/hh831696.aspx Storage Spaces - http://technet.microsoft.com/en-us/library/hh831739.aspx StorSimple - http://microsoft.com/storsimple Certified Windows JBODs http://windowsservercatalog.com/results.aspx?&chtext=&cstext=&csttext=&chbtext=&bCatID=1573&cpID=0&avc=38&ava=0&avq=0&OR=1&PGS=25&ready=0 . © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Resources Learning TechNet Developer Network 4/25/2017 5:28 PM Resources Learning Sessions on Demand http://channel9.msdn.com/Events/TechEd/Australia/2013 Virtual Academy http://www.microsoftvirtualacademy.com/ TechNet Developer Network Resources for IT Professionals http://technet.microsoft.com/en-au/ Resources for Developers http://msdn.microsoft.com/en-au/ © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

4/25/2017 5:28 PM © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.