Presentation is loading. Please wait.

Presentation is loading. Please wait.

Implementing Convergent Networking: Partner Concepts

Similar presentations


Presentation on theme: "Implementing Convergent Networking: Partner Concepts"— Presentation transcript:

1 Implementing Convergent Networking: Partner Concepts
Uri Elzur Broadcom Corporation Director, Advanced Technology Brian Hausauer Neteffect Inc. Chief Architect

2 Convergence In The Data Center: Convergence Over IP
Uri Elzur Broadcom Corporation Director, Advanced Technology

3 Agenda Application requirements in Data Center
Data flows and Server architecture Convergence Demo Hardware and software challenges and advantages Summary

4 Enterprise Network Today IT: Get ready for tomorrow’s Data Center, today
Multiple networks drive Total Cost of Ownership (TCO) up Consolidation, convergence, virtualization requires Flexible I/O Higher speeds (2.5G, 10G) requires more Efficient I/O Issue: Best use of Memory and CPU resources Additional constraints: Limited power, cooling and smaller form factor

5 Convergence Over Ethernet
Multiple networks and multiple stacks in the OS are used to provide these services Wire protocols; e.g., Internet Small Computer System Interface (iSCSI) and iWARP (Remote Direct Memory Access {RDMA}) enable the use of Ethernet as the converged network Direct Attach storage migrates to Networked Storage Proprietary clustering can now use RDMA over Ethernet The OS supports one device servicing multiple stack with the Virtual Bus Driver To accommodate these new traffic types, Ethernet’s efficiency must be optimal CPU utilization Memory BW utilization Latency Sockets Applications Windows Sockets Windows Socket Switch Storage Applications RDMA Provider User Mode File System Kernel Mode Partition TCP/IP Class Driver NDIS iSCSI Port Driver NDIS IM Driver (iscsiprt sys) . iSCSI NDIS Miniport RDMA Driver Miniport HBA NIC RNIC

6 Data Center Application Characteristics
Database Application Servers Cluster Web Servers IP/Ethernet Storage Cluster Management DAS Load Balancers

7 The Server In The Data Center
Web Servers Application Servers Database Cluster DAS Load Balancers Server network requirements – Data, Storage, Clustering, and Management Acceleration required for Data = TCP, Storage = iSCSI, Clustering = RDMA Application requirements: More transactions per server, Higher rate, Larger messages (e.g., ) IP/Ethernet Storage Cluster Management Long Lived connection Short Lived connection

8 Traditional L2 NIC Rx Flow And Buffer Management
Application Application pre-posts buffer Data arrives at Network Interface Adapter (NIC) NIC Direct Memory Access (DMA) data to driver buffers (Kernel) NIC notifies Driver after a frame is DMA’d (Interrupt moderation per frame) Driver notifies Stack Stack fetches headers, processes TCP/IP, strip headers Stack copies data from driver to Application buffers Stack notifies Application 1 8 TCP Stack 7 6 5 Driver 3 4 L2 NIC 2 Minimum of one copy

9 iSCSI iSCSI provides a reliable high performance block storage service
Microsoft Operating System support for iSCSI accelerates iSCSI’s deployment Microsoft iSCSI Software Initiator iSCSI HBA iSCSI HBA provides for Better performance iSCSI Boot iSER enablement Storage Applications File System Partition Manager Class Driver iSCSI Port Driver (iscsiprtysys) iSCSI Miniport HBA

10 The Value Of iSCSI Boot Storage consolidation – lower TCO
Easier maintenance, replacement No need to replace server blade for a HD failure No disk on blade/motherboard – space, power savings Smaller blades, higher density Simpler board design, no need for HD specific mechanical restrainer Higher reliability Hot replacement of disks if a disk fails RAID protection over boot disk Re-assign disk to another server in case of server failure

11 WSD And RDMA Kernel by pass attractive for High Performance Computing (HPC), Databases, and any Socket application WSD model supports RNICs with RDMA over Ethernet (a.k.a., iWARP) As latency improvements are mainly due to kernel by-pass, WSD is competitive with other RDMA-based technologies, e.g., Infiniband Traditional Model WSD Model Socket App Socket App WinSock WinSock WinSock Switch WinSock SPI TCP/IP WinSock Provider TCP/IP WinSock Provider RDMA service Provider User Kernel TCP/IP Transport Driver TCP/IP Transport Driver RDMA provider Driver Microsoft WSD Module OEM WSD Software NDIS OEM SAN Hardware NDIS Miniport SAN NDIS Miniport Private interface NIC RNIC

12 L2 Technology Can’t Efficiently Handle iSCSI And RDMA
iSCSI HBA implementation concerns iSCSI Boot Digest overhead – CRC-32C Copy overhead – Zero Copy requires iSCSI protocol processing RDMA RNIC implementation concerns Throughput – high Software overhead for RDMA processing MPA – CRC-32C, Markers every 512B DDP/RDMA – protocol processing, zero copy, User mode interaction, special queues Minimal latency – Software processing doesn’t allow for kernel bypass Thus, for optimal performance specific offload is required

13 Convergence Over Ethernet: TOE, iSCSI, RDMA, Management
Legacy model New model Networking Converges functions Multiple functions (SAN, LAN, IPC, Mgmt.) can be consolidated to a single fabric type Blade server storage connectivity (low cost) Clustering (IPC, HPC) NIC Host Application layer Presentation layer Session layer Transport layer Network layer Data Link layer Physical layer C-NIC TOE RSS Host Cluster IPC Convergence over Std. Ethernet LAN Storage Management Remote Management Storage Networking Block Storage File Storage Consolidates ports Leverage Ethernet pervasiveness, knowledge, cost leadership and volume Consolidate KVM over IP Leverage existing Standard Ethernet equipment Lower TCO – one technology for multiple purposes

14 C-NIC Demo

15 C-NIC Hardware Design – Advantages/Challenges
Performance – wire speed Find the right split between Hardware and Firmware Hardware for Speed – e.g., connection look up, frame validity, buffer selection, and offset computation Hardware connection look up is significantly more efficient than software IPv6 address length (128-bits) exacerbates it Flexibility Firmware provides flexibility, but maybe slower than hardware… Specially optimized RISC CPU – it’s not about MHz… Accommodate future protocol changes: e.g., TCP ECN Minimal latency From wire to application buffer (or from application to wire for Tx) Not involving the CPU Flat ASIC architecture for minimal latency Scalability – 1G, 2.5G, 10G Zero Copy architecture – a match to server memory BW and latency; additional copy or few copies in any L2 solution Power goals – under 5W per 1G/2.5G, under 10W per 10G CPU consumes 90W

16 C-NIC Software Design – Advantages/Challenges
Virtual Bus Driver Reconcile requests from all stacks Plug and Play Reset Network control and speed Power Support of multiple stacks Resource allocation and management Resource isolation Run time – priorities Interfaces separation Interrupt moderation per stack Statistics

17 Summary C-NIC Advantages
TCP Offload Engine in Hardware – for better application performance, lower CPU, and improved latency RDMA – for Memory BW and ultimate Latency iSCSI – for networked storage and iSCSI Boot Flexible and Efficient I/O for the data center of today and tomorrow

18 Brian Hausauer Chief Architect NetEffect, Inc BrianH@NetEffect.com
WinHEC 2005 Brian Hausauer Chief Architect NetEffect, Inc

19 Myrinet, Quadrics, InfiniBand, etc.
Today’s Data Center NAS Users LAN Ethernet networking adapter Applications ▪ ▪ ▪ ▪ ▪ ▪ ▪ switch Clustering Myrinet, Quadrics, InfiniBand, etc. clustering adapter ▪ ▪ ▪ ▪ ▪ ▪ ▪ switch SAN Block Storage Fibre Channel block storage adapter ▪ ▪ ▪ ▪ ▪ ▪ ▪ switch

20 Datacenter Trends: Traffic Increasing 3x Annually
2006 I/O traffic requirements Front-end web servers Mid-tier application servers Back-end database servers Network Heavy (5-10 Gb/s) Intermediate ( Mb/s) Low (<200 Mb/s) Storage (1.5-4 Gb/s) (<100 Mb/s) (3-6 Gb/s) Clustering IPC None (2-4 Gb/s) Application requirements pervasive standard, plug-n-play interop concurrent access, high throughput, low overhead fast access, low latency 6.5 – 14.0 Gb/s 2.3 – 4.6 Gb/s 5.2 – 10.2 Gb/s Typical for each server Sources: 2006 IA Server I/O Analysis, Intel Corporation; Oracle Scaling 3-fabric infrastructure expensive and cumbersome Server density complicates connections to three fabrics Successful solution must meet different application requirements

21 High Performance Computing: Clusters Dominate
Clusters in Top 500 Systems 300 Clusters continue to grow in popularity and now dominate the Top 500 fastest computers 294 clusters in top 500 computers 250 200 Ethernet is the interconnect for over 50% of the top clusters 150 Ethernet continues to increase share as the cluster interconnect of choice for the top clusters in the world 100 50 1997 1998 1999 2000 2001 2002 2003 2004 Ethernet-based Clusters All Other Clusters Source:

22 Next-generation Ethernet can be the solution
Why Ethernet? pervasive standard multi-vendor interoperability potential to reach high volumes and low cost powerful management tools/infrastructure Why not? Ethernet does not meet the requirements for all fabrics Ethernet overhead is the major obstacle The solution: iWARP Extensions to Ethernet Industry driven standards to address Ethernet deficiencies Renders Ethernet suitable for all fabrics at multi-Gb and beyond Reduces cost, complexity and TCO

23 Overhead & Latency in Networking
Sources Solutions Packet Processing Intermediate Buffer Copies Command Context Switches user application app buffer OS buffer driver buffer app buffer I/O cmd % CPU Overhead I/O cmd I/O library 100% application to OS context switches 40% kernel context switch context switch device driver OS device driver OS I/O cmd I/O cmd server software 60% application to OS context switches 40% Intermediate buffer copies 20% TCP/IP 40% transport processing 40% application to OS context switches 40% Intermediate buffer copies 20% software Transport (TCP) offload hardware I/O adapter I/O cmd RDMA / DDP TCP/IP standard Ethernet TCP/IP packet adapter buffer User-Level Direct Access / OS Bypass

24 Introducing NetEffect’s NE01 Ethernet Channel Adapter (ECA)
A single chip supports: Transport (TCP) offload RDMA/DDP OS bypass / ULDA Meets requirements for: Clustering (HPC, DBC,…) Storage (file and block) Networking Reduces overhead up to 100% Strategic advantages Patent-pending virtual pipeline and RDMA architecture One die for all chips enables unique products for dual 10Gb / dual 1Gb host CPUs NetEffect ECA Ethernet ports (10 Gb or 1 ; Copper or fibre) host interface transaction switch protocol engine DDR2 SDRAM controller MAC server chipset memory external DRAM with ECC PCI Express or PCI-X

25 Future Server Ethernet Channel Adapter (ECA) for a Converged Fabric
O/S Acceleration Interfaces Existing Interfaces RDMA Accelerator Clustering OS/driver s/w Block Storage OS/driver s/w Networking OS/driver s/w TCP Accelerator (WSD, DAPL, VI, MPI) iSER iSCSI iSCSI iWARP TOE NIC NetEffect ECA Transaction Switch Ethernet fabric(s) NetEffect ECA delivers optimized file and block storage, networking, and clustering from a single adapter

26 NetEffect ECA Architecture
host interface Basic Networking Accelerated Networking Clustering Block Storage crossbar ... MAC MAC

27 NetEffect ECA Architecture Networking
Sockets (SW Stack) WSD, SDP, TCP Accelerator Basic & Accelerated Networking Related software standards Sockets Microsoft WinSock Direct (WSD) Sockets Direct Protocol (SDP) TCP Accelerator Interfaces iWARP TOE Basic Networking

28 NetEffect ECA Architecture Storage
Connection Mgmt iSER, R-NFS iSCSI, NFS Block Storage Related software standards File system NFS DAFS R-NFS Block mode iSCSI iSER iWARP TOE Basic Networking Setup/Teardown and Exceptions only

29 NetEffect ECA Architecture Clustering
Connection Mgmt MPI, DAPL, Clustering Related software standards MPI DAPL API IT API RDMA Accelerator Interfaces iWARP TOE Basic Networking N/A Setup/Teardown and Exceptions only

30 Myrinet, Quadrics, InfiniBand, etc.
Tomorrow’s Data Center Separate Fabrics for Networking, Storage, and Clustering Users LAN iWARP Ethernet LAN Ethernet NAS ▪ ▪ ▪ ▪ ▪ ▪ ▪ switch Applications networking storage clustering networking adapter Block Storage Fibre Channel Storage iWARP Ethernet switch networking storage clustering storage adapter ▪ ▪ ▪ ▪ ▪ ▪ ▪ SAN clustering networking storage clustering adapter Clustering iWARP Ethernet Clustering Myrinet, Quadrics, InfiniBand, etc. switch ▪ ▪ ▪ ▪ ▪ ▪ ▪

31 Fat Pipe for Blades & Stacks Converged Fabric for Networking, Storage & Clustering
Users LAN iWARP Ethernet ▪ ▪ ▪ ▪ ▪ ▪ ▪ switch NAS Applications Storage iWARP Ethernet networking storage clustering adapter switch ▪ ▪ ▪ ▪ ▪ ▪ ▪ SAN Clustering iWARP Ethernet switch ▪ ▪ ▪ ▪ ▪ ▪ ▪

32 Take-Aways Multi-gigabit networking is required for each tier of the data center Supporting multiple incompatible network infrastructure is becoming increasingly more difficult as budget, power, cooling and space constraints tighten With the adoption of iWARP, Ethernet for the first time meets the requirements for all connectivity within the data center NetEffect is developing a high performance iWARP Ethernet Channel Adapter that enables the convergence of clustering, storage and networking

33 Call to Action Deploy iWARP products for convergence of networking, storage and clustering Deploy 10 Gb Ethernet for fabric convergence Develop applications to RDMA-based APIs for maximum server performance

34 Open Group Authors of ITAPI, RNIC PI & Sockets API Extensions
Resources NetEffect iWARP Consortium Open Group Authors of ITAPI, RNIC PI & Sockets API Extensions DAT Collaborative RDMA Consortium IETF RDDP WG

35 Community Resources Windows Hardware and Driver Central (WHDC)
Technical Communities Non-Microsoft Community Sites Microsoft Public Newsgroups Technical Chats and Webcasts Microsoft Blogs


Download ppt "Implementing Convergent Networking: Partner Concepts"

Similar presentations


Ads by Google