Technical Overview of Microsoft’s NetDMA Architecture Rade Trimceski Program Manager Windows Networking & Devices Microsoft Corporation

Presentation Goals Describe the NetDMA architecture Demonstrate NetDMA hardware/driver Provide information about stateless offload technologies for Windows Server codename “Longhorn”

Presentation Outline Background and motivation High level overview of NetDMA NetDMA Provider Interface Using NetDMA in the Windows networking stack Stateless offload technologies update for Windows Server Longhorn

Stateless Offload Stateless offload An approach to improving network scalability Offload interface does not require state to be stored in the offload engine Scales with number of connections Network DMA (NetDMA) Latest addition to stateless offload technologies supported on Windows platform

Motivation Eliminate/reduce CPU cost of copy from NIC receive buffers to application buffers NDIS.SYS Miniport driver Network Interface Card TCPIP.SYS AFD.SYS Winsock Application DMA to system memory Copy to application memory Prepost model

High Level Overview NetDMA provider Driver for a DMA engine that performs memory-to- memory copy NetDMA OS component that manages NetDMA providers NetDMA client An OS component that uses NetDMA to copy data NetDMA NetDMA Provider NetDMA Hardware NetDMA Client

NetDMA Provider Interface Supports multiple providers Each provider provides one or more DMA channels Each DMA channel is an independent copy unit Interaction between NetDMA and providers Channel allocation and CPU affinity Setting up DMA descriptors for copy operations DMA copy completion notification Plug and Play & power management NDIS.SYS NetDMA TCPIP.SYS NetDMA Provider C0 U0 C0 U0 C0 U0 C0 U0 System Memory

NDIS Framing Layer Network Layer Transport Layer TCP/IP Usage of NetDMA TCP uses NetDMA to copy packets from system memory to application buffers TCP overlaps NetDMA copy with TCP processing Time Packet receive processing with NetDMA TCPIP.SYS NetDMA Asyn. Copy Miniport Driver Indicate I/O completion

NetDMA Hardware and Driver in Action Kevin Cassidy Wayne Swick LAN Access Division Intel Corporation

Intel’s Approach to Network Acceleration Stateless offloads Advantages Frees CPU from making multiple trips to memory Naturally scales with new hardware With CPU, memory and I/O speeds Easy integration with Microsoft Windows network stack *Other names and brands may be claimed as the property of others.

Intel ® I/O Acceleration Technology Implements Network Acceleration on the 2006 Intel Enterprise Platform *Other names and brands may be claimed as the property of others. Intel I/OAT Advantages Increases data throughput and lowers CPU & memory overhead Uses the native Microsoft Windows* network stack Scales seamlessly to multi-GbE ports Standard platform feature with Intel Bensley platform ESB2 MCH Intel Xeon Processor Data Movement Engine MAC Intel LOM PHY Intel NIC (optional) Memory Intel Xeon Processor Stateless TCP Offloads CPU: Network stack utilizing Intel I/OAT hardware features Chipset: TCP Data moved via DMA hardware (NetDMA HW + provider) LAN: Stateless hardware acceleration in LAN silicon and NICs (Checksum, LSO)

Intel I/OAT + NetDMA Demo Intel I/OAT with Windows Server 2003 SNP Port scalability improves with I/OAT 2 Bensley Servers One with I/OAT enabled one with Standard GbE Demonstrates higher performance with Intel I/OAT Increased throughput Lower CPU utilization Microsoft Windows Server 2003 SP1 Microsoft Windows Server 2003 SP1 with SNP Client PC for statistics (thru-put, CPU utilization) *Other names and brands may be claimed as the property of others.

Intel I/OAT + NetDMA Demo

Source: Intel Labs May 2006 Ixia IxChariot* 6.0, Bi-dir Test; High Perf. Throughput script, 6 clients per port Bensley server platform, Microsoft Windows Server 2003 SNP; 4 GB RAM Performance with Intel I/OAT Demonstrating Port Scalability *Other names and brands may be claimed as the property of others.

Release Schedule NetDMA ships as part of Windows Server 2003 Scalable Networking Pack (SNP) NetDMA interface documentation is available as part of the SNP DDK

Stateless Offload in Windows Server “Longhorn” NetDMA Header Data Split Large Send Offload v2 (LSO v2) IPsec task offload v2 (IPsec v2)

Planned Enhancements to NetDMA Improve plug and play support for NetDMA provider drivers No reboot Utilize NetDMA in presence of: TCP or IP options Firewalls IPsec policy on the system

Header/Data Split Motivation Splitting received Ethernet frame offers additional performance Reduces cache misses Improves Translation Lookaside Buffer (TLB) hit rate

Header/Data Split Implementation Receive frame to be split At the beginning of TCP or UDP or other upper layer protocol header At the beginning of TCP or UDP payload Miniport driver indicates if a frame is IPv4 or IPv6 TCP or UDP Split at the beginning of protocol header or payload

Enhancements to Task Offload Technologies Large Send Offload (LSO) v2 Adds support for IPv6 Adds support for larger (> 64K) segmentation offload IPsec Task Offload v2 Adds support for IPv6 Adds new authentication and encryption algorithms: AES, SHA-256

Call To Action Consider the benefits of NetDMA in your product plans Implement Header/Data split functionality in your NIC Implement LSO v2 and IPsec task offload v2 in your NIC

Additional Resources Web Resources: Whitepapers available on the NDIS 6.0 Beta place: NDIS 6.0 Task Offload v2 NDIS 6.0 IPsec Task Offload v2 Standardized INF Keywords for Ethernet advanced properties NDISTest tools available on the NDISTest Beta place NDIS 6.0 questions: NDISTest questions:

© 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.