Where Does Circuit Switching Make Sense In the Internet? Pablo Molinero-Fernández High Performance Networking Group Stanford University

Outline Motivation Packet switching Circuit switching 6/26/2018 1:16 AM Outline Motivation Packet switching Description and myths Circuit switching Where it can be useful Research: TCPSwitch Goals, architecture and QoS Conclusion I am going to start talking about packet switching, I will describe its characteristics and why researchers thought in the late 60s and early 70s that it would be the perfect technology for computer networks I will revisit those ideas and reevaluate in the current context of the Internet, and I will show that there are some myths and misconceptions Next I will describe where circuit switching can be useful to the current Internet, and where it just adds complexity, without providing much benefit Finally I will be talking about my current research, a TCP Switch for the core of the Internet

How We Think the Internet Is

How the Internet Really Is 6/26/2018 1:16 AM How the Internet Really Is $41Bn $12Bn Market Size in 2003: Data from RHK Market Size in 2003 (Data from RHK) IP routers SONET/SDH

What Dictates Internet’s Performance Processing power Link speed

Fast Links, Slow Routers 6/26/2018 1:16 AM Fast Links, Slow Routers Processing Power Link Speed (Fiber) SPEC95Int: doubles every 2 ¼ years FO: doubles every 2 years DWDM: triples every year, i.e. Doubles every 7 months Need to optimize router processing not bandwidth efficiency Optical technology requires: No buffering Slow switching Little logic Source: SPEC95Int; Prof. Miller, Stanford Univ., 2000

Fast Links, Slow Routers 6/26/2018 1:16 AM Fast Links, Slow Routers Processing Power Link Speed (Fiber) 2x / 2 years 2x / 7 months SPEC95Int: doubles every 2 ¼ years FO: doubles every 2 years DWDM: triples every year, i.e. Doubles every 7 months Need to optimize router processing not bandwidth efficiency Optical technology requires: No buffering Slow switching Little logic Source: SPEC95Int; Prof. Miller, Stanford Univ., 2000

Can We Build All Optical Routers? Packet switches require buffering We cannot buffer light We need other techniques

Why was Internet Packet-Switched in the First Place? 6/26/2018 1:16 AM Why was Internet Packet-Switched in the First Place? Gallager: “Circuit switching is rarely used for data networks, ... because of very inefficient use of the links” Tanenbaum: ”For high reliability, ... [the Internet] was to be a datagram subnet, so if some lines and [routers] were destroyed, messages could be ... rerouted” Breaking message into packets allows parallel transmission across all links, reducing network latency. In summary the benefits that of packet switched networks can be summarized as follows: They use the bandwidth efficiently, meaning that a trunk link uses less resources than the sum of its tributaries, as they multiplex and conserve bandwidth They have little state in the intermediate nodes They are robust, some claim that they were designed to withstand a nuclear attack They do not have a central authority from whom we need permission to run experiments

Statistical Multiplexing 6/26/2018 1:16 AM Statistical Multiplexing A B C A 2x 1.7x A+B B Statistical multiplexing Important if traffic is bursty Less expensive Contention requires buffering Variable delay => no QoS guarantees

How Was Internet Used in the 70s and 80s? 6/26/2018 1:16 AM How Was Internet Used in the 70s and 80s? Applics: email, news, ftp, telnet Peer-to-peer network Long lasting and bursty flows (telnet, large ftp’s) Interactive applications (telnet) could consume pkts as they arrived Many things have changed since then Bursty flows=> statistical multiplexing is good

How Is It Different Today? Client-server population Traffic dominated by short http flows Average: 5 s, 10 pkts TCP flows Connection setup: dominated by RTT Burst of data: dominated by data rate Nature of user expectations has changed telnet: each pkt was useful Web: only useful when all pkts have arrived

How the Internet Was Used Then Host A R1 R2 R3 Host B

How the Internet Is Used Today Tresponse Client A R1 R2 R3 Server B

How Current Internet Works Servers Internet Clients

Paradox: 99% of Circuits Finish Earlier 6/26/2018 1:16 AM An Example 1 server 100 clients 1 Mbps File = 1Mbit Paradox: 99% of Circuits Finish Earlier 99% of all clients using a circuit get a better performance than the ones using packets Shortest Job First obtains the minimum delay, and this is what this example is showing (packet switching is doing processor sharing) Circuit sw Packet sw Bandwidth 1 Mbps 10 Kbps Average latency 50 sec 100 sec Worst case latency 100 sec

Myths about BW Efficiency 6/26/2018 1:16 AM Myths about BW Efficiency Most networks lightly utilized on average Backbone links: 10-15% Private lines: 3-5% Ethernet links: extremely low Links are not congested BW efficiency not needed In the core ½ BW devoted to redundancy This link utilization data from work by Andrew Odlyzko and Kerry Coffman at AT&T Research. ” Data networks are lightly utilized, and will stay that way,” A. M. Odlyzko. And “ The size and growth rate of the Internet,” K. G. Coffman and A. M. Odlyzko, Andrew Odlyzko and Kerry Coffman from AT&T Research did several studies of the Internet traffic and capacity growth, and they observed that the link utilization average over days ranges between 10 to 15% for backbone links, to 3 to 5 % for private lines, and the almost negligible use of the Ethernet links. Most of us are not continuously sending and receiving data, even when we are sitting in front of a computer If we compare it to the 30% of average utilization of voice lines, we see that links in the Internet are lightly utilized, so in fact there is not that much statistical multiplexing going on in the Internet Growth in traffic limited by source (storage and modems), not by network capacity. Most data processed locally In private lines, network designers want rapid response during peak times ( Average is average over a full week Reasons for low usage: Data lines are symmetric, traffic is not Data traffic is burstier, and gets less statistical multiplexing Data traffic growth is less predictable than voice Lumpy network capacity and sub linear price Source: A. Odlyzko and K. Coffman, AT&T Research

Myths about Robustness 6/26/2018 1:16 AM Myths about Robustness Link/router failure is rare 0.5% reroute prob. of TCP flow Rerouting circuits is not hard (if there aren’t too many!) 50 ms reroute delay for SONET vs. over 30 s for OSPF and BGP Current products can reroute 1,000’s of STS-1 circuits Robustness can also be achieved with circuits

If we were to start again with what we know today, could we use circuit switching?

This Is Where We Are IP routers SONET/SDH 6/26/2018 1:16 AM Market Size in 2003 (Data from RHK) IP routers SONET/SDH

Where We Can Go From Here 6/26/2018 1:16 AM Where We Can Go From Here PS CS Big routers replace circuit switches MPLS replaces circuit switches Circuit and packet switching coexist, as today We are are a point of transition Pure CS: TCP connection is a circuit, Very short lived ccts TCP becomes the center of the hourglass (cornerstone) Like a telephone network, where one has several lines and one can have several conversations in parallel MP[lambda]S replaces TDM circuit switches Pure circuit switched Internet,

All Packet Switched Network Is Unlikely Decision making frequency (for OC-192) Complexity Packet switching Per-packet (32 ns) # pkts (~10,000) [#conn (~100,000)] Circuit Per-connection (50 μs) # connections (~100,000)

All Circuit Switched Network Is Unlikely Hard to change all end hosts Hard to change mentality of OS and application developers Too much investment in packet switched LANs

Circuits Make Sense in the Core 6/26/2018 1:16 AM Circuits Make Sense in the Core Where: electronics cannot keep up with link speeds performance is more important than price Broad Picture Is Unlikely to Change However, provisioning of circuits is primitive Let’s see how we could change the circuit switches Both extremes look unlikely

What Other People Are Proposing Burst switching Mega packets or mini circuits with explicit tear-down Automatic monitoring of traffic Queues, utilization, … Both cases need new provisioning protocol

We Want to Propose a Different Approach 6/26/2018 1:16 AM We Want to Propose a Different Approach Expose circuits to IP Optimize for the common case TCP (90-95% of traffic) Data (9 out of every 10 pkts) TCPSwitch 90-95% of all traffic is TCP How do they interoperate with existing IP routers? What signal and circuit setup mechanisms need to be used for the circuit switches Not the same fate as IPv6 Replace SS7 and other proprietary SONET control protocols

TCPSwitch Objectives Architecture QoS Results Simulation Implementation

Objectives of TCPSwitch 6/26/2018 1:16 AM Objectives of TCPSwitch Ability to interoperate with: Normal IP routers Unmodified end-hosts Optics and off-the-shelf switches QoS and best-effort Not the same fate as IPv6 Reduce blocking probability

TCPSwitch Exposes Circuit Switching to IP 6/26/2018 1:16 AM TCPSwitch Exposes Circuit Switching to IP What we change from today’s picture Granularity of the circuit: TCP flow Replace SONET control protocol Access Network: Ethernet, packet-switched Best-effort only Like the taxi coming from the airport Outer backbone: TCP Switches Best-effort and reserved traffic Like checking-in and boarding the plane Inner backbone: Lambda- and SONET switches Reserved traffic only, circuit switched Like the airplane itself IP routers TCPSwitches

TCP “Creates” a Connection Router Destina-tion Source SYN SYN+ACK DATA Permanent Circuits Packets

Let TCP Leave State Behind 6/26/2018 1:16 AM Let TCP Leave State Behind Ingress TCP-SW Core TCP-SW Egress TCP-SW Destina-tion Source SYN SYN+ACK DATA ACCEPTED Circuits Packets Connection establishment using TCP SYN Admission control No buffering in the core

Complexity of Switches 6/26/2018 1:16 AM Complexity of Switches Ingress: Like a packet-sw’ed router Admission control Outgoing port creates circuits Core: Regular TDM/Optical switch Egress: Like core, but with packet reassembly Ingress has all the buffering

Design Issues Layer-4 lookup Path rerouting Exact match 6/26/2018 1:16 AM Design Issues Layer-4 lookup Exact match Path rerouting SYN-packet not seen Hard state vs. soft state Soft state, because less than 0.5% of cases there is a reroute (little leakage) Admission control based on (priority, BW) pair given by an oracle (SLA database or TCP options) Circuit establishment via out-of-band circuit (less processing when idle, because there is no polling of idle channels). However this limits the circuit establishment rate

Design Issues (2) Admission control Circuit establishment/tear-down 6/26/2018 1:16 AM Design Issues (2) Admission control Priority+BW request blocking prob. vs dropping prob. Circuit establishment/tear-down In-band vs out-of-band Minimum granularity Soft state, because less than 0.5% of cases there is a reroute (little leakage) Admission control based on (priority, BW) pair given by an oracle (SLA database or TCP options) Circuit establishment via out-of-band circuit (less processing when idle, because there is no polling of idle channels). However this limits the circuit establishment rate

Simplifies QoS In PS: In CS: 6/26/2018 1:16 AM Simplifies QoS In PS: Very complex schemes for isolation, jitter and reserved BW In CS: dedicated BW Jitter and isolation fall naturally Can mix premium customers (QoS) as well as best-effort in a single network QoS: Result of BW reservation Easy to explain to customers (nice SLA) Admission control is easier problem than QoS with packets (done on a per-connection basis rather than a per-packet basis)

Simulation Results In ns-2 Paradox verified using TCP and http traffic 6/26/2018 1:16 AM Simulation Results In ns-2 Paradox verified using TCP and http traffic Avg response: 50% and 90% improvement, for some blocking prob (0%-3%) Some TCP inefficiencies noticeable with very short flows Work in progress Avg delay smaller, the bigger the BW reservation Connection establishment will retry for 3min (that is what RFC says) However Microsoft waits for only 45 sec. TCP inefficiencies are noticeable for short flows But what we want is low response time Can have starvation if client is rejected immediately

Implementation Results 6/26/2018 1:16 AM Implementation Results Ingress TCPSwitch Linux 2.4 router using netfilter (iptables) and iproute2 Module in user space for proof of concept Big penalty for crossing kernel-user boundary Data switching: similar performance as a router with a classifier (e.g. policer) Now being ported into kernel Penalty of the order of 3-4 ms for the connection setup Penalty of the order of 60 us for the data forwarding (we are doing some classification now)

Conclusion Circuit switching is already there 6/26/2018 1:16 AM Conclusion Circuit switching is already there CS makes sense in the core: Electronics can keep up with link speed Use of Internet fits well with CS TCPSwitch Exposes circuits to IP Optimized for TCP and data CS unlikely to go away TCPSwitch: a circuit-aware core with fine circuit granularity Core, where performance is more important than cost Cost amortized over many users, or connections Lower complexity at the edges, more complexity in the core Can use optical switches Provides QoS at the cost of BW inefficiencies TCPSwitch , a circuit-aware core