Unifying Packet & Circuit Networks with OpenFlow Saurav Das, Guru Parulkar, & Nick McKeown Stanford University Huawei, Feb 3 rd

Internet has many problems Plenty of evidence and documentation Internet’s “root cause problem” It is Closed for Innovations 2

Million of lines of source code 5400 RFCsBarrier to entry 500M gates 10Gbytes RAM BloatedPower Hungry We have lost our way Specialized Packet Forwarding Hardware Operating System Operating System App Routing, management, mobility management, access control, VPNs, …

Software Control Router Hardware Datapath Authentication, Security, Access Control HELLO MPLS NAT IPV6 anycast multicast Mobile IP L3 VPN L2 VPN VLAN OSPF-TE RSVP-TE HELLO Firewa ll Multi layer multi region iBGP, eBGP IPSec Many complex functions baked into the infrastructure OSPF, BGP, multicast, differentiated services, Traffic Engineering, NAT, firewalls, MPLS, redundant layers, … An industry with a “mainframe-mentality”

Operating System Reality App Specialized Packet Forwarding Hardware Operating System Operating System App Lack of competition means glacial innovation Closed architecture means blurry, closed interfaces

Deployment IdeaStandardize Wait 10 years Glacial process of innovation made worse by captive standards process Driven by vendors Consumers largely locked out Glacial innovation

Specialized Packet Forwarding Hardware Ap p Specialized Packet Forwarding Hardware Ap p Specialized Packet Forwarding Hardware Ap p Specialized Packet Forwarding Hardware Ap p Specialized Packet Forwarding Hardware Operating System Operating System Operating System Operating System Operating System Operating System Operating System Operating System Operating System Operating System Ap p Network Operating System App Change is happening in non-traditional markets

App Simple Packet Forwarding Hardware App Simple Packet Forwarding Hardware Network Operating System 1. Open interface to hardware 3. Well-defined open API 2. At least one good operating system Extensible, possibly open-source The “Software-defined Network”

The change has already started In a nutshell – Driven by cost and control – Started in data centers…. and may spread – Trend is towards an open-source, software-defined network – Growing interest for cellular and telecom networks

Example: New Data Center Cost 200,000 servers Fanout of 20  10,000 switches $5k commercial switch  $50M $1k custom-built switch  $10M Savings in 10 data centers = $400M Control 1.Optimize for features needed 2.Customize for services & apps 3.Quickly improve and innovate Large data center operators are moving towards defining their own network in software.

Windows (OS) Windows (OS) Windows (OS) Windows (OS) Linux Mac OS Mac OS x86 (Computer) x86 (Computer) Windows (OS) Windows (OS) App Linux Mac OS Mac OS Mac OS Mac OS Virtualization layer App Controller 1 App Controller 2 Controller 2 Virtualization or “Slicing” App OpenFlow Controller 1 NOX (Network OS) NOX (Network OS) Controller 2 Controller 2 Network OS Trend Computer IndustryNetwork Industry Simple common stable hardware substrate below+ programmability + strong isolation model + competition above = Result : faster innovation

Signaling Control Data Simple, Robust, Reliable Data Path Controller Decoupled Automated Control Open Interface Into Hardware

The Flow Abstraction Rule (exact & wildcard) ActionStatistics Rule (exact & wildcard) ActionStatistics Rule (exact & wildcard) ActionStatistics Rule (exact & wildcard) Default ActionStatistics Exploit the flow table in switches, routers, and chipsets Flow 1. Flow 2. Flow 3. Flow N. e.g. Port, VLAN ID, L2, L3, L4, … e.g. unicast, mcast, map-to-queue, drop Count packets & bytes Expiration time/count

14 Controller OpenFlow Switch Flow Table Flow Table Secure Channel Secure Channel OpenFlow Protocol SSL hw sw OpenFlow Switching Add/delete flow entry Encapsulated packets Controller discovery A Flow is any combination of above fields described in the Rule

Controller Flow Example OpenFlow Protocol RuleActionStatisticsRuleActionStatisticsRuleActionStatistics A Flow is the fundamental unit of manipulation within a switch Routing

OpenFlow is Backward Compatible Ethernet Switching * Switch Port MAC src MAC dst Eth type VLAN ID IP Src IP Dst IP Prot TCP sport TCP dport Action * 00:1f:.. *******port6 Application Firewall * Switch Port MAC src MAC dst Eth type VLAN ID IP Src IP Dst IP Prot TCP sport TCP dport Action ********22drop IP Routing * Switch Port MAC src MAC dst Eth type VLAN ID IP Src IP Dst IP Prot TCP sport TCP dport Action * * *** ***port6

OpenFlow allows layers to be combined VLAN + App * Switch Port MAC src MAC dst Eth type VLAN ID IP Src IP Dst IP Prot TCP sport TCP dport Action ***vlan1****80 port6, port7 Flow Switching port3 Switch Port MAC src MAC dst Eth type VLAN ID IP Src IP Dst IP Prot TCP sport TCP dport Action 00:1f vlan port600:2e.. port3 Switch Port MAC src MAC dst Eth type VLAN ID IP Src IP Dst IP Prot TCP sport TCP dport Action port 1000:2e.. Port + Ethernet + IP * * ***

A Clean Slate Approach 18 Goal: Put an Open platform in hands of researchers/students to test new ideas at scale Approach: 1. Define OpenFlow feature 2. Work with vendors to add OpenFlow to their switches 3. Deploy on college campus networks 4. Create experimental open-source software - researchers can build on each other’s work

OpenFlow Hardware Cisco Catalyst 6k NEC IP8800 HP Procurve 5400 Juniper MX-series WiMax (NEC) WiFi Quanta LB4G Ciena CoreDirector Arista 7100 series (Fall 2009) (Fall 2009)

OpenFlow Deployments Stanford Deployments – Wired: CS Gates building, EE CIS building, EE Packard building (soon) – WiFi: 100 OpenFlow APs across SoE – WiMAX: OpenFlow service in SoE Other deployments – Internet2 – JGN2plus, Japan – research groups have switches Research and Production Deployments on commercial hardware Juniper, HP, Cisco, NEC, (Quanta), …

UW Stanford Univ Wisconsin Indiana Univ Rutgers Princeton Clemson Georgia Tech Internet2 NLR Nationwide OpenFlow Trials Production deployments before end of 2010 Production deployments before end of 2010

D D C D D C D D C D D C IP/MPLS C D D C D D C D D D D D D D D D D CC D D D D GMPLS Motivation IP and Transport networks  are separate networks that are controlled and managed independently leading to duplication of functions and resources in multiple layers and high capex and opex  do not dynamically interact and thus do not benefit from diverse switching technologies  have very different architectures that makes integrated operation and convergence hard

Flow Network D D C D D C D D C D D C IP/MPLS C D D C D D C D D D D D D D D D D CC D D D D GMPLS UCP

pac.c Flow Network … that switch at different granularities: packet, time-slot, lambda & fiber Simple, Unified, Automated Control Plane Simple,Robust,Reliablenetworkof FlowSwitches Research Goal: Packet and Circuit Flows Commonly Controlled & Managed

25 OpenFlow & Circuit Switches Exploit the cross-connect table in circuit switches Packet Flows Switch Port MAC src MAC dst Eth type VLAN ID IP Src IP Dst IP Prot TCP sport TCP dport Action 25 Circuit Flows Out Port Out Lambda Starting Time-Slot Signal Type VCG 25 In Port In Lambda Starting Time-Slot Signal Type VCG The Flow Abstraction presents a unifying abstraction … blurring distinction between underlying packet and circuit and regarding both as flows in a flow-switched network

Unified Architecture OPENFLOW Protocol Packet Switch Circuit Switch Packet & Circuit Switch NETWORK OPERATING SYSTEM Underlying Data Plane Switching App Unified Control Plane Unifying Abstraction Networking Applications

Congestion Control QoS 27 OpenFlow UCP enables pkt-ckt interface Network Recovery Traffic Engineering Power Mgmt Security Discovery Routing

IN OUT GE ports TDM ports Packet Switch Fabric Packet Switch Fabric OpenFlow (software) OpenFlow (software) RAS RAS IP VLAN2, P1 VLAN2 VCG 3 OpenFlow (software) OpenFlow (software) VLAN VLAN2, P2 VLAN7VCG5 Packet Switch Fabric IP TCP 80 + VLAN7, P2 TDM Circuit Switch Fabric VCG5 VCG3 P1 VC4 1 P2 VC4 4 P1 VC4 10 VCG5 P3 STS192 1 OpenFlow Example

Congestion Control Example Application (1)..via Variable Bandwidth Packet Links

OpenFlow Demo at SC09

Traffic Engineering Example Application (2)

Traffic Engineering Example Application (2)..via Dynamic Automated Optical Bypass

Controller OpenFlow protocol AWG WSS (1×9) AWG Fujitsu WSS based OF circuit switch Ethernet Hosts NOX WSS (1×9) NetFPGA based OF packet switch

Openflow Circuit Switch 25 km SMF OpenFlow packet switch GE-Optical Mux/Demux

OpenFlow Protocol CCC FLOWVISOR OpenFlow Protocol CK P P P P Unified Virtualization

OpenFlow Protocol CCC FLOWVISOR OpenFlow Protocol CK P P P P ISP ‘A’ Client Controller Private Line Client Controller High-end Client Controller Under Transport n/w Service Provider control Isolated Client Network Slices Single Physical Infrastructure of Packet & Circuit Switches Unified Virtualization

Summary OpenFlow is a large clean-slate program with many motivations and goals convergence of packet & circuit networks is one such goal OpenFlow simplifies and unifies across layers and technologies packet and circuit infrastructures - electronics and photonics while unified API allow innovations in data and control planes independently in network control, management and virtualization Example demonstrations at circuit & packet intersection Variable Bandwidth Packet Links Dynamic Automated Optical Bypass

Backup

Issues with Current IP & Transport n/w Separate management systems and incompatible protocols - complexity of managing across several layers, interfaces & architectures, leading to duplication of resources and functions Lack of a unified architecture across packet and circuit – fully distributed with tightly linked control and data planes in packet networks, fully distributed, decentralized or fully centralized in transport networks, multiple vendor domains with proprietary interfaces prevent greater integration and increase complexity GMPLS the only attempt towards a UCP across packet & circuit (2000) Today – Packet vendors and ISPs are not interested Transport n/w SPs view it as a signaling tool available to the mgmt system for provisioning private lines (not related to the Internet) After 10 yrs of development, next-to-zero significant deployment GMPLS Issues 

Issues are when considered as a unified architecture and control plane control plane complexity escalates when unifying across packets and circuits because it makes basic assumption that the packet network remains same: IP/MPLS network – many years of legacy L2/3 baggage and that the transport network remain same -, multiple layers and multiple vendor domains use of fragile distributed routing and signaling protocols with many extensions, increasing switch cost & complexity, while decreasing robustness does not take into account the conservative nature of network operation - can IP networks really handle dynamic links? Do transport network service providers really want to give up control to an automated control plane? does not provide easy path to network virtualization Issues with GMPLS

Software Control Transport NE Hardware Datapath LMP HELLO 41 UNI TL-1 GMPLS PBB-TE Carrier Ethernet MPLS-TP ASO N ENNI int ra ENNI int er OSPF-TE RSVP-TE HELLO CORBA L1VPN, L2VPN PCE PWE3 Many complex functions baked into the infrastructure More coming ……

Control Plane Data Plane OF Protocol Control Plane Architectures

OpenFlow: Architecture Concepts Separate data from control – A standard protocol between data and control Define a “generalized flow” based data path – Very flexible and generalized flow abstraction – Delayer or open up layers1-7 Hierarchically centralized “open” controller with API – For control and management applications Virtualization of data and control planes Backward compatible – Though allows completely new header

OpenFlow: Architecture Implications Separate data from control – Independent innovations in data and control planes – Less dependence on a single vendor Define a “generalized flow” based data path – Simpler data path: cheaper, uniform, stable – Applicable across technologies and layers Hierarchically centralized “open” control with an API – Easier to make reliable and robust – Enables lots of innovations by different stakeholders

OpenFlow: Architecture Implications Virtualization – Enable innovations and experimentation – Deployment of new ideas: “production revision control” Backward compatible – Easy to support in existing switches/routers and networks – Easy to show the value proposition Software Defined Networking