2006 © SWITCH 1 TNC'06 Panel Presentation Myths about costs of circuit vs. packet switching Simon Leinen

2006 © SWITCH 2 Common Wisdom About Backbone Costs “Bandwidth is almost free” – thanks to DWDM “Routing is very expensive” – in particular, high-capacity (>= 10 Gb/s) interfaces/line cards The only way to build cost-effective high-speed backbones: – build DWDM network – use one lambda for “legacy” packet (IP) network – use circuit switching to provide “end-to-end” connections to sites

2006 © SWITCH 3 Free Bandwidth on DWDM Networks? The incremental cost for additional bandwidth can be low – Just add another wavelength The up-front investment in a DWDM network can be quite high – Fibers (IRU/construction) – Amplifiers, dispersion compensation etc. – Little things like synchronization (for SDH/SONET) The overall operational expenditure (OpEx) can also be high – Fiber lease/maintenance – Device maintenance – Housing

2006 © SWITCH 4 Free Bandwidth on DWDM Networks Howto: “Creative Accounting”: – Blame the up-front and overall costs on the “legacy IP” service: “The first wavelength is quite expensive. We use it for the IP backbone.” – Additional wavelengths (for point-to-point connections) are now very cheap.

2006 © SWITCH 5 Costs of router OC-192/STM-64 POS – Old carrier-class router: 135k (½ 2-port linecard) – 375k (LR) – Newer carrier-class router: 83k (¼ 4-port linecard) – 188k (LR) – Glorified L3 switch: 215k+XFP – 350k (LR)

2006 © SWITCH 6 Costs of router OC-192/STM-64 POS – Old carrier-class router: 135k (½ 2-port linecard) – 375k (LR) – Newer carrier-class router: 83k (¼ 4-port linecard) – 188k (LR) – Glorified L3 switch: 215k+XFP – 350k (LR) 10GE – Old carrier-class router: 85k – 165k (1-port linecard) – Newer carrier-class router: 14k+XENPAK (1/8 8-port linecard) – Glorified L3 switch: 5k+XENPAK (¼ 4-port linecard)  XENPAK cost: 600 (CX4) – (DWDM, replaces transponder!) All of these platforms support IPv4/IPv6 unicast/multicast, MPLS. Compare with transponders for WDM/TDM switch platforms: 45k-70k for 10Gb/s (but with tunable transponders available)

2006 © SWITCH 7 Chassis/switching fabric costs Old carrier-class router – 38k (4-slot 80Gb/s) – 355k (16-slot 1.28 Tb/s) Newer carrier-class router – 225k (8-slot 640? Gb/s) – 450k (16-slot 1.28? Tb/s) Glorified L3 switch – 16k (4-slot 32 Gb/s) – 34k (4-slot 720 Gb/s) – 97k (9-slot 720 Gb/s large/red.)

2006 © SWITCH 8 Can a glorified L3 switch be a router? “If it walks like a duck...” – Switch/router terminology is largely a marketing device – L3 switches forward on L3 addresses, thus I consider them routers – They must participate in routing protocols (OSPF, BGP-4 etc.)  Many L3 switches don't implement IS-IS, some lack MPLS features etc. – With a L3 switch, you are mostly stuck with what the hardware supports, but  With forwarding based on TCAMs, many forwarding modes can be supported  Replacing the switching logic is cheap! (especially with centralized forwarding)

2006 © SWITCH 9 Costs per...? Aggregate bps – circuit-switched wins – but seamless upgrades are more difficult Number of endpoints – ? Number of connections sustained by the network – packet-switched wins: no connection state in the network – limitations of today's WDM and WDM/TDM networks (5-low 100s of channels) Connection setup rate – packet-switched wins: connection creation between endpoints – Note that setup delay may be low for CO network; but throughput will be limited Resilience – packet-switched wins: no connection state in the network (fate sharing)

2006 © SWITCH 10 Building Cost-Effective IP Backbones Cost drivers for IP networks: – Carrier-class equipment  Redundancy within the device  Excessive device scalability (e.g. hundreds of interfaces per box)  High development costs (ASIC) over low sales volumes o e.g. CRS-1: USD 500M dev. project, sold 100 units in first 18 months – Complex feature sets (integration bugs) – Bleeding-edge performance – Large buffers – Large routing tables  small routing tables are OK (TCAM vs. CAMs in Ethernet/MPLS switches)

2006 © SWITCH 11 Building Cost-Effective IP Backbones How to avoid these cost drivers: – Use high-volume (but high-performance) equipment  e.g. glorified campus switches  10 GbE will get cheaper still (replacement of SAN and cluster interconnects)  Provide redundancy through multiple chassis – Avoid complex features, especially in the core  This will help lots for OpEx and availability  A minimum feature set could include o IPv4/IPv6 unicast/multicast w/BGP+IGP (IS-IS/OSPF) o diffserv support, ACLs and control-plane protection o basic management features (SNMP, decent CLI) o tunneling for basic VPNs (EoL2TPv3 or EoMPLS)

2006 © SWITCH 12

2006 © SWITCH 13

2006 © SWITCH 14 Even more cost-effective packet networks Learn to stop worrying and love small router buffers – Recent studies show some conditions under which this works well  Develop (end/edge) mechanisms to fix those cases that don't work well Humble suggestion for the network layer after IP(v6): – Move path selection to the endpoints (hosts)  Makes the network more useful (path-agile applications, controlled multipath etc.)  If done right (strict source routes), this eliminates routing tables from routers