Cisco Confidential 1 © 2010 Cisco and/or its affiliates. All rights reserved. Fiber Channel over Ethernet Marco Voi – Cisco Systems – Workshop CCR INFN GRID 2011 Il protocollo e le applicazioni.

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 2 What Why Designs and applications

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 3 FCoE is I/O consolidation of FC storage traffic over Ethernet FC traffic shares Ethernet links with other traffics Requires a lossless Ethernet fabric Fibre Channel Traffic Ethernet

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 4 From a Fibre Channel standpoint it’s FC connectivity over a new type of cable called… Ethernet From an Ethernet standpoints it’s Yet another ULP (Upper Layer Protocol) to be transported FC-0 Physical Interface FC-1 Encoding FC-2 Framing & Flow Control FC-3 Generic Services FC-4 ULP Mapping Ethernet Media Access Control Ethernet Physical Layer FC-2 Framing & Flow Control FC-3 Generic Services FC-4 ULP Mapping FCoE Logical End Point Fiber Channel over Ethernet Protocol Mapping

Cisco Confidential 5 © 2010 Cisco and/or its affiliates. All rights reserved. 1, 2, 4, 8, 10, 16, (32) Gbps1, Gbps10, 20 GbpsFCoE

Cisco Confidential 6 © 2010 Cisco and/or its affiliates. All rights reserved. FC Traffic FC HBA Fewer CNAs (Converged Network adapters) instead of NICs, HBAs and HCAs Limited number of interfaces for Blade Servers All traffic goes over 10GE CNA FC Traffic FC HBA NIC LAN Traffic NIC LAN Traffic NIC Mgmt Traffic NIC Backup Traffic IPC Traffic HCA

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 7 Ethernet Economic Model Embedded on Motherboard Integrated into O/S Many Suppliers Mainstream Technology Widely Understood Interoperability by Design FC Economic Model  Always a Stand-Up Card  Specialized Drivers  Few Suppliers  Specialized Technology  Special Expertise  Interoperability by Test Ethernet Model has Proven Benefits

Cisco Confidential 8 © 2010 Cisco and/or its affiliates. All rights reserved. Ethernet Header FCoE Header FC Header FC Payload CRC EOF FCS 12 bytes (MAC addresses) + 4 bytes (802.1Q tag) 16 bytes 24 bytes Up to 2112 bytes 4 bytes 1 byte (EOF) + 3 bytes (padding) 4 bytes Total: 2180 bytes

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 9 Ethernet Header FCoE Header FC Header FC Payload CRC EOF FCS Destination MAC Address Source MAC Address (IEEE 802.1Q Tag) ET = FCoEVerReserved SOF Encapsulated FC Frame (with CRC) EOFReserved FCS Byte 0 Byte 2197 FCoE Frame Format Bit 0 Bit 31  FCoE is a standard - June 3rd 2009, the FC-BB-5 working group of T11 completed its work and unanimously approved a final standard for FCoE 10Gbps Ethernet Lossless Ethernet Matches the lossless behavior guaranteed in FC by B2B credits Ethernet jumbo frames Max FC frame payload = 2112 bytes

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 10 Standard / FeatureStatus of the Standard IEEE 802.1Qbb Priority-based Flow Control (PFC) In Sponsor Ballot IEEE 802.3bd Frame Format for PFC In Sponsor Ballot IEEE 802.1Qaz Enhanced Transmission Selection (ETS) and Data Center Bridging eXchange (DCBX) Under recirculation in order to go to Sponsor Ballot after the May interim IEEE 802.1Qau Congestion NotificationDone! IEEE 802.1Qbh Port ExtenderIn its first task group ballot  Developed by IEEE Data Center Bridging Task Group (DCB)  All technically stable  Final standards expected by mid 2010 CEE (Converged Enhanced Ethernet) is an informal group of companies that submitted initial inputs to the DCB WGs.

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 11 Fibre Channel Drivers Ethernet Drivers Operating System PCIe Ethernet Fibre Channel 10GbE Link Ethernet Driver bound to Ethernet NIC PCI address FC Driver bound to FC HBA PCI address  Replaces multiple adapters per server, consolidating both Ethernet and FC on a single interface  Appears to the operation system as individual interfaces (NICs and HBAs)

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 12 CostCompatibility Virtualization PCIe x16 10GbE/FCoE vNICs Eth 0 FC 1 QP 2 FC 3 Eth GbE/FCoE PCIe Bus FC 10GbE Software FCoE “Free” SAN Access for Any Ethernet Equipped Host Existing Driver Stacks VM I/O Virtualization and Consolidation

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 13 Expansion Module 32 x Fixed ports 1/10 GE or 1/2/4/8 FC Expansion Module Unified Ports Expansion Module Unified Ports 8 ports 1/10 GE + 8 ports1/2/4/8 G FC 8 ports 1/10 GE + 8 ports1/2/4/8 G FC 16 ports 1/10 GE 16 ports 1/10 GE

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 14 Ethernet LAN SAN B SAN A LegacyI/O Consolidation with FCoE SAN B LAN SAN A Nexus 5000 FCFCoE 2x 4x

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 15 Nexus GE FEX SAN B SAN A Nexus GE FEX Single Hop Design Extending the FCoE Edge – Nexus 2232 Nexus 5000 FCF-A Nexus 5000 FCF-B Nexus 2232 Nexus 5548

Cisco Confidential 16 © 2010 Cisco and/or its affiliates. All rights reserved. Multi-Tier FCoE The goals are: To expand I/O consolidation into the backbone To introduce the support for native FCoE Storage array SAN ASAN B Backbone Native FCoE Storage array

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. Nexus 5010 Nexus 5020 Nexus 4000 Nexus 2232 NX-OS & DCNM Nexus 5548UP NEW Nexus 5596UP NEW MDS 9500 Nexus 7000 NEW F1 32-portFCoE 8-port Director Class Solutions Fixed Solutions NEW

Cisco Confidential 18 © 2010 Cisco and/or its affiliates. All rights reserved. Processor Memory LAN Storage IPC Processor Memory I/O Subsystem LAN Storage IPC I/O IPC: Inter Process Communication

Cisco Confidential 19 © 2010 Cisco and/or its affiliates. All rights reserved. A single system to unify compute, network and storage resources access under a single management Chassis Mgmt Unified Network Mgmt LAN SAN A SAN B

Thank you.