The Limits of Switch Bandwidth [Add Presentation Title: Insert tab > Header & Footer > Notes and Handouts] 4/19/2017 The Limits of Switch Bandwidth Scott Kipp February 28, 2011 © 2010 Brocade Communications Systems, Inc. CONFIDENTIAL—For Internal Use Only

Increasing Bandwidth Switch designs are limited by: ASICS that are getting faster with process improvements 45 nm processes can support 25Gbps lanes Backplanes and connectors are getting faster and support 25 Gbps FR4 will probably need to be replaced with Nelco13 Front panel is set by 19” rack width and limits the port count The height is optimized on 1.75” (1RU) increments and is a differentiator Power of blade – optics usually only dissipate 50W of power max Higher speed designs will need new optical module and electrical interface standards 25-28 Gbps electrical signals is needed to increase the bandwidth of the switch

10GbE ASIC and Module Design 1st Generation 2nd Generation 3rd Generation Electrical / Optical Interface XSBI / 300 pin MSA XAUI / X2, XENPAK, XPAK XFI, SFI / XFP, SFP+, QSFP+ Electrical Lanes and Speeds 16 lanes at 644 MHz 4 lanes at 3.125Gbps 1 lane at 10.3125 Gbps 10GE Ports/ASIC 3 12 48 - 128 Throughput/ASIC 30 Gbps 120 Gbps 480 to 1280 Gbps XENPAK XPAK XFP 300 Pin MSA QSFP – 4 lanes of 10GbE SFP+ X2

100GbE ASIC and Module Design 1st Generation 2nd Generation 3rd Generation Electrical / Optical Interface CAUI - CFP, CPPI – CXP, CXP2? CAUI-4 - CFP2, CPPI-4 – QSFP, CFP4 ? Electrical Interface 10 lanes at 10 Gbps 4 lanes at 25Gbps 100GE Ports/ASIC 4 with CFP 32 with CXP 12 Throughput/ASIC 480 to 1280 Gbps 1200 to 2560 Gbps Under Development By CFP MSA CXP CXP2? CFP QSFP+ CFP2 CFP4

Switch Design Switches are typically designed for a 19” rack and this allows about 16-17” of port space for ports Highest densities so far are 64 mini SFP+ (mSFP) ports at 10GE or 48 SFP+ and 4 QSFP ports Blades inserted horizontally or vertically Modular Switch with 48 SFP+ ports Modular Switch with 64 mSFP+ ports 1U Switch

Switch Designs with 10Gbps interfaces 48 SFP+ Design - 480 Gbps of throughput, 48 Watts* of power Limited by size of SFP+ with 15mm pitch 4 CFP Design - 400 Gbps of throughput, 60 Watts** of power Limited by ASICs now, but soon limited by size of CFP Port with 84mm pitch CFP 40 QSFP+ Design - 1600 Gbps of throughput, 80 Watts*** of power Limited by ASICs now, size of QSFP+ with 21mm pitch 32 CXP Design - 3200 Gbps of throughput, 80 Watts**** of power Limited by ASICs, CXP has 27mm pitch *Based on 1 Watt of power/SFP per SFF-8431 *** Based on 2 Watts of power/QSFP+ **Based on 15 Watts/CFP for 10km links ****Based on 2.5 Watts of power / CXP

Summary of 10 Gbps Switch Designs Support of single-mode applications requires larger modules because of higher power dissipation Multimode modules support less power and are denser Throughput Max Power / Module 10km Support SFP+ 48*10Gbps = 480 Gbps 1.5 W Yes CFP for 100GbE 4*100Gbps = 400 Gbps 20 W for 10km 32W for 40km QSFP+ for 40GbE 40*40Gbps = 1.6Tbps 2 W Not without exotic cooling CXP for 100GbE 32*100Gbps = 3.2 Tbps

Switch Designs with 25-28 Gbps interfaces 48 SFP+ Design - 1.3 Tbps of throughput at 32GFC, 48 Watts* of power Limited by size of SFP+ with 15mm pitch 8 CFP2 Design - 800 Gbps of throughput, 96 Watts** of power Limited by ASICs now, but soon limited by size of CFP2 Port with 42mm pitch CFP2 CFP2 40 QSFP+ Design - 4 Tbps of throughput, 80 Watts*** of power Limited by ASICs and power, QSFP+ pitch is 21mm 16 CFP4 Design - 1.6 Tbps of throughput, 96 Watts**** of power Limited by ASICs, CXP pitch is 27mm *Based on 1 Watt of power/SFP per SFF-8431 **Based on 12 Watts/CFP2 for 10km links ***Based on 2 Watt of power/QSFP ****Based on 6 Watts/CFP2 for 10km links

Summary of 25Gbps Switch Designs Support of single-mode applications requires larger modules because of higher power dissipation Multimode modules support less power and are denser Throughput Max Power / Module 10km Support SFP+ for 32GFC 48*28Gbps = 1.3Tbps 1.5 W Yes, but challenging at 1.5W CFP2 for 100GbE 8*100Gbps = 800 Gbps 6 W Yes QSFP+ for 100GbE 40*100Gbps = 4Tbps 2 W Not without exotic cooling CFP4 for 100GbE 16*100Gbps = 1.6 Tbps 3 W Probably 2014

Tbps Bandwidths require faster speeds 10Gbps lanes will limit the throughput of the switch/blade in the next few years Difficult to expand beyond 128 ports / ASIC Module size limits bandwidth and causes long trace lengths In the next few years, 25-28Gbps signaling needed to support Tbps switch/blade bandwidths 4X25Gbps easier to support than 10X10Gbps in near future Cost of 25-28G signaling compared to 10Gbps will be an interesting battle in the next few years Terabit Ethernet will require major innovation 100x10G or 40x25G not practical for pluggable modules 10X100G not practical from signaling perspective

[Add Presentation Title: Insert tab > Header & Footer > Notes and Handouts] 4/19/2017 Thank You © 2010 Brocade Communications Systems, Inc. CONFIDENTIAL—For Internal Use Only