Why can’t I innovate in my wiring closet? Nick McKeown MIT, April 17, 2008 The Stanford Clean Slate Program Funded by: Cisco, DT DoCoMo, NEC, Xilinx

2 Outline  Routing doesn’t belong in routers  Substrates to enable innovation  OpenFlow as a streamlined substrate

3 Internet Innovation End to end arguments led to the principle: Move function “up” out of the network to the end-points Led to streamlined substrate of links and routers Even congestion control done at end-points Led to enormous innovation on top , WWW, VOIP, P2P, Social networks, …

4 A fixed substrate Fierce protection against overloading network, unless necessary Change happens slowly or surreptitiously IPv6, Multicast, NAT IP, Intel instruction set, ms-dos Stable fixed substrates Standards that bred innovation It’s hard to imagine it having been any other way…. An era of fixed substrates

5 How streamlined is the substrate? Feature lists as long as your arm Routers with >10M lines of source code, datapath with 100M gates & 1Gbyte RAM. Many complex functions in the infrastructure OSPF, BGP, multicast, differentiated services, Traffic Engineering, NAT, firewalls, MPLS, redundant layers, … Uh-oh, looks like another GENI talk… Claim Most complexity is about routing: picking paths, managing location, identity and access. The current network controls the routing. Routing doesn’t belong in the boxes. If we control routing, we can innovate.

6 Routing inside routers It made sense at the time  Early on, seemed necessary for scalability & robustness  Routing doesn’t need to be in the network Trivial example: source routing  Routing doesn’t belong in the network Aside: Electricity grid of lines, transformers and switches  Stakeholders should control routing and innovation  Why can’t I add my own routing protocol? Or mobility management protocol?  Why do you and I have to run the same routing protocol? 4D [Zhang], RCP [Rexford] and others

7 Local equilibrium  Inelegant closed design (like a bad API) leads to complexity, and fragility. Hard to innovate.  Not a conspiracy, just a local equilibrium Inelegance High Barrier to Entry Industry, IETF, … Add complexity Resistant to change

8 Why should we care? There are basic things the Internet can’t do well Mobility, access control/security, management, … Why not just add more features to fix it? An uneasy feeling that we are piling more and more complexity into the network, making it overloaded, fragile, less elegant and harder to innovate. We need a simple, reliable, constant substrate Oh my god – it really does look like another GENI talk…

9 Current substrate On a PC substrate, we can choose an OS. And now we have virtualization too. In the network world, your switch/router comes bundled and you have no choice. So… 1. Alternative OS on your switch/router? 2. Virtualization of your switch/router?

10 Outline  Routing doesn’t belong in routers  Substrates to enable innovation 1.Open OS’s on switches/routers 2.Virtualization of switches/routers  OpenFlow as a streamlined substrate

11 Innovation by ripping off the lid XORP (and Zebra; or just Linux) Open-source router code for Linux, FreeBSD, etc. Flexible, easy to use Software-based; limited performance Point solution NetFPGA Small (4x1GE) “router”; open-source hardware and software Flexible, low-cost, medium ease of use Hardware-based; line-rate performance Point solution, limited port count (4-8) OpenWRT Embedded Linux for wireless APs and routers

12 FPGA Memory 1GE PCI CPU Memory NetFPGA Board PC with NetFPGA Open source networking hardware Rack of 1U servers 96 x 1GE ports Hardware available from 3 rd party ~$500 (universities) Line-rate forwarding PCI board, or pre-built system (desktop or rack-mount) 500 boards, 15 countries; expect 2,000 this year Free reference designs, gateware and courseware

13 Outline  Routing doesn’t belong in routers  Substrates to enable innovation 1.Open OS’s on switches/routers 2.Virtualization of switches/routers  OpenFlow as a streamlined substrate

14 Innovation by changing the substrate IP Diverse link layers Diverse physical layers Diverse applications Diverse transport layers IP Diverse link layers Diverse physical layers Diverse applications Diverse transport layers Virtualization layer XYZ Proposed approach: OpenFlow … innovation from the “bottom up” This is the GENI approach: Innovation from the “top down” e.g. WUSTL SPP, VINI, …

15 OpenFlow Model IP Diverse physical layers Diverse transport layers Flow layer XYZ Diverse applications Ethernet Routing, Mobility, Naming/Addressing, Access Control, Management, Monitoring… Diverse link layers Allow lots of innovative research experiments Packet switching and circuit switching Collapse difference; ease use of optical circuits

16 OpenFlow Switching A way to innovate in the networks we use everyday. A “pragmatic” compromise Allow researchers to run experiments in their network… …without requiring vendors to expose internal workings. 1.Work with switch and AP vendors to add OpenFlow to their products 2.Deploy on university campuses 3.Stand back and watch students innovate… Basics An Ethernet switch (e.g. 128-ports of 1GE) Use flow-table already in every switch and chipset An open protocol to remotely add/remove flow entries

17 What we’d like…  Isolation: Regular production traffic untouched  Virtualized and programmable: Different flows processed in different ways  Equipment we can trust in our wiring closet  Open development environment for all researchers (e.g. Linux, Verilog, etc).  Flexible definitions of a flow  Individual application traffic  Aggregated flows  Alternatives to IP running side-by-side  …

18 Controller OpenFlow Switch Flow Table Flow Table Secure Channel Secure Channel PC OpenFlow Protocol SSL hw sw OpenFlow Switch specification OpenFlow Switching Add/delete flow entry Encapsulated packets Controller discovery

19 Flow Table Entry “Type 0” OpenFlow Switch Switch Port MAC src MAC dst Eth type VLAN ID IP Src IP Dst IP Prot TCP sport TCP dport RuleActionStats 1.Forward packet to port(s) 2.Encapsulate and forward to controller 3.Drop packet 4.Send to normal processing pipeline + mask Packet + byte counters

20 OpenFlow “Type 1” (future) More flexible header  Allow arbitrary matching of first few bytes Additional actions  Rewrite headers (e.g. NAT, or address obfuscation)  Map to queue/class  Encrypt Support multiple controllers  Load-balancing and robustness

21 OpenFlow Consortium Goal: Evangelize OpenFlow to vendors Free membership for all researchers Whitepaper, OpenFlow Switch Specification, Reference Designs Licensing: Free for research and commercial use

22 OpenFlow: Status Commercial Ethernet switches and routers  Working with six vendors to add to existing products  Expect OpenFlow “Type 0” to be available in Reference switches  Software: Linux and OpenWRT (for access points)  Hardware: NetFPGA (line-rate 1GE; available soon)  Working on low-cost 48-port 1GE switch based on Quanta switch (Broadcom chips) Reference controller  Simple test controller  NOX controller (available soon)

23 Deployment at Stanford Stanford Computer Science Department Gates Building ~1,000 network users 23 wiring closets Stanford Center for Integrated Systems (EE) Allen Building ~200 network users 6 wiring closets Working with HP Labs and Cisco on deployment

24 Deployment in Internet2 NetFPGA routers deployed 1GE links

25 OpenFlow Usage Models 1. Experiments at the flow level  Layer 2 or Layer 3 (same!)  User-defined routing protocols  Admission control  Network access control  Network management  Energy management  VOIP mobility and handoff …… 2. Experiments at the packet level  Slow: Controller handles packet processing  Fast: Redirect flows through programmable hardware  Modified routers, firewalls, NAT, congestion control… 3. Alternatives to IP Experiment-specific controllers Static or dynamic flow-entries

26 OpenFlow Switch Usage example: Production Traffic Unchanged Open API Commercial Switch or AP Linux PC OpenFlow Protocol SSL Flow Table Flow Table Secure Channel Secure Channel Hari sw hw Normal Software Normal datapath User Space “Hari” Hari: Use production network Policy Rule Controller Llinux kernel

27 OpenFlow Switch Usage example: New Protocols Isolated, But Alongside Open API Linux PC OpenFlow Protocol SSL Flow Table Flow Table Secure Channel Secure Channel Dina sw hw Normal Software Normal datapath User Space “Dina” Dina: Use Dina’s protocol Policy Rule Controller Llinux kernel Commercial Switch or AP

28 Example Experiment at the flow level Mobility Lots of interesting questions Management of flows Control of switches Access control of users and devices Tracking user location and motion

29 Controller PC NetFPGA Laboratory Innovation in the datapath Layerless plumbing OpenFlow-enabled Commercial Switch Flow Table Flow Table Secure Channel Secure Channel Normal Software Normal Datapath Line-rate packet processing 1.Encryption 2.IP++ 3.Packet inspection 4.Congestion control (XCP etc.) 5.…? Line-rate packet processing 1.Encryption 2.IP++ 3.Packet inspection 4.Congestion control (XCP etc.) 5.…?

30 Controller Innovation IPEthernet Alongside Normal routing, mgmt, inside box Static Controllers Experiment Specific Controllers 4DNOX Controller Innovation Layer Alongside Spanning tree, VLANs, mgmt, inside box TDM/WDM Circuits OpenFlow Substrate Layer Alongside circuit routing, mgmt, inside box

31 Controller Open Questions  Scalability of controller  Load-balanced and redundant controllers  Aggregation of flows

32 Conclusion  Networking has to work harder than most fields to enable innovation.  The good news is that industry is open to the idea.  And government agencies are on our side.  Let’s work together to do it… let’s deploy OpenFlow widely in universities.  If you are interested, please contact me