WinHEC 2015 9/9/2018 1:06 AM © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE.

Windows Networking: Offloads and Optimizations
WinHEC 2015 9/9/2018 1:06 AM Windows Networking: Offloads and Optimizations Don Stanwyck Sr PM, Windows Core Networking © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Introduction and Agenda
Tech Ready 15 9/9/2018 Introduction and Agenda Session Introduction Networking Offloads and Optimizations: How we make Windows Networking faster and more efficient Session Agenda: The networking offloads of Windows, especially Windows Server The networking features of Windows Server 1709 and Windows 10 Fall Creators Update Our vision of networking in the next months © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Offloads and Optimizations
WinHEC 2015 9/9/2018 1:06 AM Offloads and Optimizations Q: What’s an offload? A: It’s when the NIC does the processing for the OS Q: What’s an optimization? A: It’s when the OS does something to make processing faster or more efficient Q: Is something ever both? A: Yes. Some features are software-hardware coordination, an optimization with an offload © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Hardware Offloads and Capabilities
WinHEC 2015 9/9/2018 1:06 AM Hardware Offloads and Capabilities Address checksum offload Interrupt moderation Jumbo frames Large send offload (LSO) Receive Side Coalescing (RSC) © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Software controlled Hardware features
WinHEC 2015 9/9/2018 1:06 AM Software controlled Hardware features Data Center Bridging (DCB) Enhanced Transmission Service (ETS) Priority Flow Control (PFC) Encapsulation offloads NV-GRE VxLAN IPsec Task Offload RMDA Native host (mode 1) Converged NIC (mode 2) Guest RDMA (mode 3) RSS SR-IOV VLAN VMQ VMMQ © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Software features (no HW support needed)
WinHEC 2015 9/9/2018 1:06 AM Software features (no HW support needed) ACLs, extended ACLs, and SDN ACLs NIC Teaming (LBFO, SET) vmQoS and SDN QoS Virtual RSS (vRSS) © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

NIC Evolution The original NICs were simple devices
WinHEC 2015 9/9/2018 1:06 AM NIC Evolution The original NICs were simple devices Loaded and unloaded one byte at a time Calculated FCS (CRC) Performed no protocol logic © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

NIC Evolution Over time they got more sophisticated
WinHEC 2015 9/9/2018 1:06 AM NIC Evolution Over time they got more sophisticated Loaded and unloaded one packet at a time Understood some protocol logic Could generate IP, TCP, and UDP header checksums under the right conditions Memory and computing power were getting cheaper © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

WinHEC 2015 9/9/2018 1:06 AM NIC Evolution WS2008, WS2008 R2 NICs developed the ability to manage distinct queues Queues could be set to interrupt independent CPUs Protocol headers could be processed enough to direct packets to different queues RSS for native traffic (WS2008) 5-tuple hash created Hash values mapped to different queues VMQ for Hyper-V traffic (WS2008 R2) Filters set to match destination MAC addresses Each MAC address got it’s own queue Default SET OF QUEUES © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Packets processed sequentially
Without RSS TCP/IP stack One queue Packets processed sequentially NIC arriving packets

With RSS Many queues One queue Packets processed in parallel
TCP/IP stack TCP/IP stack Many queues Packets processed in parallel on associated cores One queue Packets processed sequentially NIC Place in queue Generate hash arriving packets arriving packets

WinHEC 2015 9/9/2018 1:06 AM NIC Evolution WS2012, WS2012 R2 To support the industry standard for SR-IOV, NICs began to get onboard switches VMQ evolved under the covers from filters to switch-port routes (when a switch was present) Windows Terminology: NIC-Switch port  “vPort” (can be a vPort of a PF or can be a VF) vPort SWITCH PF VF © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

WinHEC 2015 9/9/2018 1:06 AM NIC Evolution WS2012, WS2012 R2 In parallel, Remote DMA (RDMA) arrived on the scene High speed full packet processing in the NIC DMA between host and NIC at both sides Skips packet processing in the stack High throughput, low overhead, using the processing power of the NIC DataCenter Bridging (DCB) also arrived The cable becomes several virtual cables called “traffic classes” Bandwidth is managed on per TC basis Individual TCs can be paused to prevent switch buffer issues vm vm vm vm Host RDMA engine PF VF VF VF VF vPort vPort vPort vPort vPort vPort vPort vPort vPort vPort vPort SWITCH © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

VMQ (filter-based) Hyper-V Switch Hyper-V Switch A A A B C D E F Def
Match filter No B C D E F A Def Match filter NIC_switch Get MAC/VLAN Get MAC/VLAN arriving packets arriving packets

Accelerating the guest - vRSS
The software path between the Hyper-V switch and the guest became a multi-lane highway vRSS is built on VMQ Hardware calculates the Toeplitz hash (RSS hash) on each incoming packet and stores it with the packet vRSS unloads a VMQ and reads the RSS Hash vRSS assigns packet to vmBus queue/core and issues software interrupt

Virtual RSS Packet is delivered over vmBus channel to VM
9/9/2018 1:06 AM Virtual RSS Packet is delivered over vmBus channel to VM VM maps vmBus channel to RSS vCPU and processes packets in parallel vCPU processes vmBus channels and vCPUs are independent vRSS runs on both variants of VMQ © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

vRSS vmBus channels VM “F” 1 2 Hyper-V Switch 3 vRSS B C D E F A Def
vCPU 1 vmBus 2 Hyper-V Switch 3 channels vRSS B C D E F A Def Match filter NIC_switch Get MAC/VLAN arriving packets

Windows Server 2016 Windows Server 2016 brought several new items:
WinHEC 2015 9/9/2018 1:06 AM Windows Server 2016 Windows Server 2016 brought several new items: Converged NIC (RDMA to the host vNIC) Microsoft Azure VFP Switch Extension SDN v2 (NVGREv2 and VxLAN offloads) SDN QoS VMMQ Switch Embedded Teaming © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Windows Server 2016 – Converged NIC
WinHEC 2015 9/9/2018 1:06 AM Windows Server 2016 – Converged NIC VM Storage Host partition VM VM VM Live Migration Mgmt SMB Other Stuff vmNIC vmNIC vmNIC SMB Multichannel & SMB Direct Hyper-V Switch NIC Team NIC NIC NIC NIC (DCB) (DCB) Windows Server 2012 R2 © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Windows Server 2016 – Converged NIC
WinHEC 2015 9/9/2018 1:06 AM Windows Server 2016 – Converged NIC VM Storage VM Storage Host partition Host partition VM VM VM VM VM VM Live Migration Live Migration SMB Mgmt SMB Management& Cluster TCP/IP Other Stuff RDMA vmNIC Other Stuff vmNIC vmNIC vmNIC vmNIC vmNIC SMB Multichannel & SMB Direct Hyper-V Switch Hyper-V Switch (SDN) NIC Team With embedded teaming NIC NIC NIC NIC NIC NIC (DCB) (DCB) (DCB) (DCB) Windows Server 2012 R2 Windows Server 2016 © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

ACLs, Metering, Security
SDN Switch Extension Known in Azure as the Virtualization Filtering Platform (VFP) Acts as a virtual switch inside the Hyper-V vmSwitch Provides core SDN functionality for Azure networking services, including: Address Virtualization for VNET VIP -> DIP Translation for SLB ACLs, Metering (QoS), and Security Guards Bandwidth management/control (QoS) Uses programmable rule/flow tables to perform per- packet actions Supports all Azure data plane policy at 40GbE+ with offloads Available to private cloud in Windows Server 2016 VM VM vNIC vmNIC vmNIC VM Switch VFP ACLs, Metering, Security VNET SLB (NAT)

Flow Tables: the Right Abstraction for the Host
VMSwitch exposes a typed Match-Action-Table API to the controller Controllers define policy One table per policy Key insight: Let controller tell switch exactly what to do with which packets e.g. encap/decap, rather than trying to use existing abstractions (tunnels, …) VNet Description Tenant Description Controller VNet Routing Policy ACLs NAT Endpoints Host: VFP Flow Action TO: 10.2/16 Encap to GW TO: Encap to TO: !10/8 NAT out of VNET Flow Action TO: DNAT to TO: !10/8 SNAT to Flow Action TO: /24 Allow 10.4/16 Block TO: !10/8 NIC VM1 VNET LB NAT ACLS

Table Typing/Flow Caching are Critical to Performance
COGS in the cloud is driven by VM density: 50GbE and 100GbE are here 60 to 100 VMs/host is common, 200+ VMs/host have been seen on customer sites First-packet actions can be complex Established-flow matches must be typed, predictable, and simple hash lookups First Packet VNET LB NAT ACLS Flow Action TO: 10.2/16 Encap to GW TO: Encap to TO: !10/8 NAT out of VNET Flow Action TO: DNAT to TO: !10/8 SNAT to Flow Action Flow Action Flow Action TO: /24 Allow 10.4/16 Block TO: !10/8 NIC Blue VM1 Connection Action , ,80,9876 DNAT + Encap to GW , ,80,9876 Encap to , ,6754,80 SNAT to VFP Subsequent Packets

HNVv2 – VxLAN, NV-GRE Customers asked for VxLAN – we delivered!!
WinHEC 2015 HNVv2 – VxLAN, NV-GRE 9/9/2018 1:06 AM Customers asked for VxLAN – we delivered!! But we still do NV-GRE for those who like that option All HNV policies are handled in the SDN Extension Network Controller (NC) plumbs the policies to the gateways and hosts Either SCVMM or NRP program the NC A semi-hidden feature automatically adjusts the MTU on the wire to accommodate the encapsulation overhead Better performance than splitting packets due to length of encapsulation © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

SDN QoS A more reliable, more performant vmQoS
WinHEC 2015 SDN QoS 9/9/2018 1:06 AM A more reliable, more performant vmQoS Compatible with RDMA work loads Compatible with DCB Supports Egress reservations (minimum guaranteed bandwidth) Supports Egress limits (maximum permitted bandwidth) Works well even with very different policies for different VMs Works on all vmSwitch ports (host or guest) Managed by Network Controller Implemented in the VFP © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Virtual Machine MultiQueue (VMMQ)
Built on VMQ (vPort) and vRSS Associates vPorts with one or more hardware queues Distributes traffic between queues based on RSS Hash Allows for multiple queues on the default vPort Useful for very network intensive VMs or when VMs outnumber the number of VMQ queues

VMMQ vmBus channels VM “F” 1 2 Hyper-V Switch 3 B C D E F A Def
vCPU 1 vmBus 2 Hyper-V Switch 3 channels B C D E F A Def Match filter NIC_switch Get MAC/VLAN arriving packets

VMMQ Discussion Still limited by number of cores available
Still limited by number of queues available Not much advantage below 25 Gbps A fast processor can keep up with 6-10 Gbps by itself Very useful when number of VMs exceeds number of queues Increasing the number of queues that the default vPort can use helps all the default vPort users (the ones that don’t have their own VMQ)

Switch Embedded Teaming (SET)
WinHEC 2015 Switch Embedded Teaming (SET) 9/9/2018 1:06 AM Legacy NIC Teaming isn’t going away It isn’t compatible with the SDN switch extension Long-term direction is to integrate full teaming functionality into the Hyper-V Switch WS2016 is the v1 edition of integrated teaming Focused on the needs of the SDN Extension and Converged NIC Has a number of limitations/restrictions in order to focus on doing the right things right © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Switch Embedded Teaming (SET)
WinHEC 2015 Switch Embedded Teaming (SET) 9/9/2018 1:06 AM What it does Switch independent teaming Dynamic or HyperVPort modes of load distribution RDMA/DCB aware SR-IOV teaming Teams of up to 8 ports The limitations All team members must be identical make/model/driver/features No LACP No Active/Passive teaming © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

WinHEC 2015 Windows Server 1709 9/9/2018 1:06 AM Two notable features in this Semi-Annual Channel (SAC) release Dynamic VMQ/VMMQ (also known as RSSv2) Guest RDMA © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Dynamic VMQ/VMMQ (RSSv2)
dVMQ from WS2012R2 had challenges Spreading was too slow Coalescing was too fast RSS spreading management was redesigned Spreading to more cores happens aggressively Coalescing to fewer cores happens conservatively VMQ and VMMQ make use of RSSv2 © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Guest RDMA RDMA works in Native Hosts – why not Guests?
WinHEC 2015 9/9/2018 1:06 AM Guest RDMA RDMA works in Native Hosts – why not Guests? Guest RDMA uses SR-IOV Throughput as good as with native RDMA No noticeable load on Guest LPs © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Demo configuration ..42.100 Host partition Host B VM SMB ..42.110
VM Storage Host B VM SMB Management vmNIC VF vNIC vNIC Hyper-V Switch SR-IOV C1 C1 TOR

Help us do RDMA right Customers complain: RoCE is too hard to deploy
WinHEC 2015 9/9/2018 1:06 AM Help us do RDMA right Customers complain: RoCE is too hard to deploy They are right DCB is hard – and it has real issues in larger deployments If you do RoCE, please help us with tools to verify and validate switch settings, NIC settings, etc. Fortunately iWarp just works New RDMA Deployment Guide available at: © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Review, Our vision, Our future plans
WinHEC 2015 9/9/2018 1:06 AM Review, Our vision, Our future plans Windows Server 2012 RDMA SR-IOV NIC Teaming Software QoS Windows Server 2016 Converged NIC VXLAN VMMQ SET teaming Future releases Hardware QoS Tenant DCB QUIC Crypto GFT Accelerating the host Accelerating the guest Windows Server 2012 R2 Dynamic VMQ HNVv1 Windows Server, version 1709 Guest RDMA Dynamic VMQ/VMMQ © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Looking ahead Vision: Accelerated Networking everywhere
Acceleration for nested Hyper-V (e.g., Container Host) Acceleration for untrusted tenants Acceleration for low-latency apps in guests/tenants RDMA everywhere SMB, pMEM, NVMe, etc.

Call to action Offloads are the key to high performance networking
Tech Ready 15 9/9/2018 Call to action Offloads are the key to high performance networking Please implement as many of the Windows Offloads as possible Tell us what we other offloads we should be exploring If you do RoCE – we need tools, diagnostics, etc. Look at security offloads – design with network security in mind © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Thank you for being our partner
WinHEC 2015 9/9/2018 1:06 AM Thanks! Thank you for being our partner Questions? © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Please Complete An Evaluation Your input is important!
Tech Ready 15 9/9/2018 Please Complete An Evaluation Your input is important! Multiple ways to access Evaluation Forms: CommNet stations located throughout conference venues Via WinHEC app on your Windows Phone and Windows device Via BYOD browser from any wired or wireless internet connection to <link> © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

WinHEC 2015 9/9/2018 1:06 AM © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE.

Similar presentations

Presentation on theme: "WinHEC 2015 9/9/2018 1:06 AM © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

WinHEC 2015 9/9/2018 1:06 AM © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE.

Similar presentations

Presentation on theme: "WinHEC 2015 9/9/2018 1:06 AM © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE."— Presentation transcript:

Similar presentations

About project

Feedback