1. Circuit switching in the Internet Ph.D. oral examination Pablo Molinero-Fernández 21 st May 2002

2. 2 Traffic doubles every year Cost and complexity of five times as many central offices is prohibitive GAP OF 5x!!

3. 3 Optics removes bandwidth constraints But we cannot buffer light!

4. 4 Circuit switching Link bandwidth is reserved Need signaling No buffering No processing in data path

5. 5 Packet switching Link bandwidth is shared Buffering Per-packet processing

6. 6 Contributions The Internet needs circuit switchin g in the core Provision ing of fat pipes in an all- optical backbon e TCP Switching: how to integrate circuit switching in the core

7. 7 How we think the Internet is

8. 8 Why the Internet uses packet switching Efficient use of expensive links: –“Circuit switching is rarely used for data networks,... because of very inefficient use of the links” – Bertsekas & Gallager ‘92 Resilience to failure of links & routers: –”For high reliability,... [the Internet] was to be a datagram subnet, so if some lines and [routers] were destroyed, messages could be... rerouted” – Tanenbaum ‘96

9. 9 What the Internet is really like today SONET/SDH DWDM

10. 10 What the Internet is really like today Modems, DSL

11. 11 Users Performance criteria Carriers Cost: Bandwidth efficiency Reliability and stability Low complexity Response time Service Level Agreement (QoS)

12. 12 Users Response time Service Level Agreement (QoS) What users want Carriers Cost: Bandwidth efficiency Reliability and stability Low complexity

13. 13 Response time of packets and circuits Packet swCircuit sw 10 Mb/s1 Gb/s Flow bandwidth 1 s0.51 s Avg response time 1 s Max response time All but one of circuits finish earlier File = 10Mbit x 100 1 server 100 clients 1 Gb/s

14. 14 10.99 sec10.99 s Max response time Packet swCircuit sw 10Mb/s;1Gb/s1 Gb/s Flow bandwidth 1.10 sec10.50 s Avg response time A big flow can kill CS if it blocks the link File = 10Gbit/ 10Mbit x 99 1 server 100 clients 1 Gb/s Response time when blocking occurs

15. 15 Response time with flow rate limits Packet swCircuit sw 1 Mb/s Flow bandwidth 10,000 sec 10,000 s Max response time 109.9sec 109.9s Avg response time No difference between CS and PS in core x 99 File = 10Gbit/ 10Mbit 1 server 100 clients 1 Gb/s 1 Mb/s

16. 16 Analytical modeling Fluid model: Many independent arrivals => Poisson Service policies: –Packets: Processor Sharing –Circuits: FCFS Service time distribution: –Flow size variance: Bimodal –Realistic flow size distrib: Pareto

17. 17 Fluid model: M/BiModal/N Access link is as fast as the core link Hogging by long flows Flow size variance +

18. 18 Fluid model: M/BiModal/N Flow rate limited by access link (1/N) Same response time regardless of flow size variance, if N is large Flow size variance + N=2 k

19. 19 Fluid model: M/Pareto/N Link hogging becomes very bad with heavy-tailed traffic, if ratio N=1

20. 20 Fluid model: M/Pareto/N Response time similar to IP if core/access ratio N is large Typically N >> 1,000 N =2 k 512

21. 21 Users see little difference in response time Simulation of full networks N is large => same response time Circuit switching Packet switching

22. 22 Service Level Agreements Packet switching: –Algorithms (WFQ, DRR, …) => not used –Thus we must overprovisioning => used and it works Circuit switching: –Simple QoS: guaranteed BW => no jitter

23. 23 Users Response time Service Level Agreement (QoS) What carriers want Carriers Cost: Bandwidth efficiency Reliability and stability Low complexity

24. 24 Cost: Bandwidth efficiency Argument: packets share all link BW => statistical multiplexing gain => more throughput with bursty traffic Reality: –Internet avg. link utilization: 5-20% [Coffman&Odlyzko’02] –Phone avg. link utilization: ~33% [Odlyzko’99] –There is a glut of BW in the core [WSJ’00] Result: –Packets more efficient, but BW is no longer a scarce resource

25. 25 OC-12OC-3 OC-48 Carriers peak allocate their network When over- provisioning, link BW is virtually peak allocated That is exactly what circuit switching does Source: Chuck Fraleigh ‘02

26. 26 Reliability and stability (I) Argument: because of the state, rerouting a circuit is more costly than with packets Reality: –Internet average availability: 1220 min/year down time [Labovitz’99] –Phone average availability: 5 min/year down time [Kuhn’97]

27. 27 Reliability and stability (II) Reality (cont.): –IP recovers in about 3 min (median), sometimes it takes over 15 min [Labovitz’01] –SONET required to recover in less than 50 ms Result: –No evidence packet switching is more robust

28. 28 Low complexity (I) Argument: No per-flow state => packet switching is simpler Reality: –PS: 8M lines of code in core router [Cisco’s IOS ‘00] –CS: 18M lines of code in telephone switch [AT&T/Lucent 5ESS ‘96] –CS: 3M lines of code in transport switch [’01] Result: –Packet switching does not seem inherently less complex than circuit switching

29. 29 Functions in a packet switch Interconnect scheduling Route lookup TTL process ing Buffer ing Buffer ing QoS schedul ing Control plane Ingress linecard Egress linecardInterconnect Framing Data path Control path Scheduling path

30. 30 Functions in a circuit switch Interconnect scheduling Control plane Interconnect Framing Ingress linecard Egress linecard Data path Control path

31. 31 Low complexity (II) Argument: IP does not have the signaling of circuits switches => Routers go faster Reality: –IP does almost same operations on every packet as a circuit switch on the circuit establishment –CS has no work to do once circuit is established Result: –The fastest commercially-available circuit switches [Ciena ’01, Lucent ‘01] have 5x the capacity of the fastest routers [Cisco ’01, Juniper ’02]

32. 32 Network architecture LANs & wireless WANs Use packet switching Better response time (ratio N small) Efficient use of the spectrum Use circuit switching More capacity, reliability Similar response time & QoS MANs If metro-to-access BW ratio (N) is small => use packets Otherwise use what costs less

33. 33 Contributions The Internet needs circuit switchin g in the core Provision ing of fat pipes in an all- optical backbon e TCP Switching: how to integrate circuit switching in the core

34. 34 Integration of circuits and packets Create a separate circuit for each user flow IP controls circuits Optimize for the most common case –TCP (90-95% of traffic) –Data (9 out of 10 pkts) TCP Switching

35. 35 TCP Switching exposes circuits to IP TCP Switches IP routers

36. 36 TCP “creates” a connection Router Destina- tion Source SYN SYN+ACK DATA Packets

37. 37 Let TCP leave state behind Boundary TCP-SW Core TCP-SW Boundary TCP-SW Destina- tion Source SYN SYN+ACK DATA Create circuit One CircuitPackets Create circuit

38. 38 What is a typical flow? Most traffic are TCP connections: –Taking less than 10 s, 12 packets and 4 KBytes –Obtaining less than 100 Kbps –~40% of the flows continue transmitting ACKs after sending a FIN (asymmetrical closures)

39. 39 State management feasibility Amount of state –Minimum circuit = 56 Kb/s. –178,000 circuits for OC-192. Update rate –About 51,000 entries per sec for OC-192 Implementable in hardware or software.

40. 40 TCP Switching can be implemented in software TCP Switching boundary router: Kernel module in Linux 2.4 1GHz PC Forwarding latency –Forward one packet: 21  s. –Compare to: 17  s for IP. –Compare to: 95  s for IP + QoS. Time to create new circuit: 57  s. Source: Byung-Gon Chun ‘01

41. 41 Slow Start Congestion Avoidance Flow duration Inactivity timeout Circuit BW Instantaneous bandwidth time Bandwidth inefficiencies Compromise: inactivity timeout of few seconds

42. 42 Related work IP Switching –Uses ATM virtual circuits (i.e. packets) –Became MPLS (but no longer user flows) Generalized Multi-Protocol Label Switching (GMPLS) –Coarse circuits –Heavy weight signaling

43. 43 Contributions The Internet needs circuit switchin g in the core TCP Switching: how to integrate circuit switching in the core Provision ing of fat pipes in an all- optical backbon e

44. 44 New networking scenario Optical Switches IP routers

45. 45 New networking scenario Optical Crossconnect IP Linecards

46. 46 Circuit creation is slow We need a safeguard to avoid running out of BW => inefficiency A slow signaling requires a larger BW safeguard

47. 47 Controlling coarse circuits with user flows Should use the fastest optical switching elements Should avoid ACKs => no RTT

48. 48 Conclusions Circuits should be used in the core, packets in the edges TCP Switching integrates circuits and packets in an evolutionary way User flows can be used to control an all-optical network

49. 49 Papers PMF, Nick McKeown, "TCP Switching: Exposing Circuits to IP“, IEEE Micro, 2002 PMF, NM, "TCP Switching: Exposing Circuits to IP“, Hot Interconnects, 2001 PMF, NM, “Study of routing behavior trough traffic analysis and traceroute measurements”, NAT Times, 2001 PMF, NM, Hui Zhang, "Is IP going to take over the world (of communications)?“, submitted

50. Thank you

51. 51 Generalized Multi- Protocol Label Switching Extension of MPLS mapping label paths to circuits Use of true circuits –Heavy signaling –Slow (~ ms-s) No definition of control algorithms

52. 52 Controlling fat pipes using packets Counting packets is very noisy => hard to make a decision

53. 53 Controlling fat pipes counting flows Counting flows is easier than counting packets

54. 54 Cost Argument: because of their simplicity and efficiency IP routers are less expensive Reality: –Only part of the picture (amortization period, operations cost, …) –A router is not as simple, requires more chips than circuit switch, requires memory Result: –More components means more cost (but given the margins, final price can be the same)

55. 55 Overprovisioning limits bandwidth efficiency In order to provide QoS to the users, link BW is virtually peak allocated That is exactly what circuit switching does Source: Chuck Fraleigh ‘02

56. 56 Controlling coarse circuits with user flows Should use the fastest optical switching elements Should avoid ACKs

57. 57 Controlling coarse circuits with user flows Should use the fastest optical switching elements Should avoid ACKs