Openflow-based Multipath Switching in Wide Area Networks Artur Barczyk/Caltech Michael Bredel, Harvey Newman (Caltech) Ronald van der Pol (SURFnet) TERENA Networking Conference 2013 June 5, 2013 Artur.Barczyk@cern.ch

Outline Introduction to the problem space Openflow overview Summary of multipath techniques The OLiMPS project Current capabilities and demonstration results Future Outlook June 5, 2013 Artur.Barczyk@cern.ch

Introduction Our background: High-performance Networking for HEP HEP computing environment LHC Open Networking Environment June 5, 2013 Artur.Barczyk@cern.ch

Why do we need multiple paths? Multiple paths are used to increase capacity increase resiliency Load balancing is used to increase efficiency in capacity utilization increase robustness reduce/avoid congestion Both have been studied extensively in the past June 5, 2013 Artur.Barczyk@cern.ch

Multipath: old-style approaches Multipath is not new, various techniques exist: Layer 3: ECMP – Equal Cost Multi-Path Layer 2: LAG - Ethernet Link Aggregation Groups Layer 1: VCAT – Virtual conCATenation Common drawback: None are particularly flexible preconfigured static hashing algorithms Often vendor specific implementation Work on hop-by-hop basis, difficult to optimize globally Powerful new approaches in the data center: Layer 2: TRILL - Transparent Interconnection of Lots of Links (IEFT) Layer 2: SPB - Shortest Path Bridging (IEEE) June 5, 2013 Artur.Barczyk@cern.ch

The Approach in OLiMPS OLiMPS: Openflow Link-layer MultiPath Switching with a centralized, out-of-band control, we can construct a robust multi-path system without modifications to the Layer 2 frame structure Big plus: using Openflow, no need for new HW or feature support (other than Openflow) Addresses the problem of topology limitations in large-scale layer 2 networks Remove the necessity of a tree structure in the topology achieved though the use of Spanning Tree Protocol Allow for per-flow multipath switching Increase the robustness and Increase efficiency and Simplify management of layer 2 network resources June 5, 2013 Artur.Barczyk@cern.ch

Floodlight/OLiMPS OpenFlow Controller OLiMPS controller is based on Floodlight Written in JAVA Supported by Big Switch Open Source Floodlight implements a set of OpenFlow applications Link Discovery Topology and spanning tree calculation Simple packet forwarding and learning switch June 5, 2013 Artur.Barczyk@cern.ch

Topology (I) Floodlight Link Discovery Uses both LLDP and BDDP packets to detect links A “direct” link will be detected if an LLDP is sent out one port and the same LLDP is received on another port. Otherwise, it will be a “broadcast” link Based on the link information, Floodlight creates its topology view June 5, 2013 Artur.Barczyk@cern.ch

Topology (II) Floodlight Topology Service Computes topologies based on link information it learns from the link discovery Uses the notion of ”OpenFlow islands” An island (or cluster) is defined as a group of directly connected OpenFlow switches under the same instance of Floodlight June 5, 2013 Artur.Barczyk@cern.ch

Topology (III) In OLiMPS: Make use of centralized topology view for Path calculations Forwarding and routing decisions Need to enable multiple links via non-OpenFlow islands VLANs and VLAN tunnels June 5, 2013 Artur.Barczyk@cern.ch

OLiMPS Floodlight extensions Current OLiMPS controller implementation contains Tunnels between OpenFlow Islands Broadcast filtering, loop prevention ProxyARP Multipath computation Load balancing Tools: CLI June 5, 2013 Artur.Barczyk@cern.ch

Algorithms: Multipath Computation Disjoint paths calculation: Dijkstra’s algorithm Graph theory: Suurballe’s algorithm for multiple paths June 5, 2013 Artur.Barczyk@cern.ch

Algorithms: Load Balancing (I) Multipath calculation gives us multiple paths, with different characteristics Load balancing provides the decision how to best use them First implementation: Round-Robin distribution of flows to available paths avoids uneven distribution intrinsic to hash-based algorithms although cannot prevent uneven utilization Next steps: need to take into account: diverse path characteristics diverse flow characteristics June 5, 2013 Artur.Barczyk@cern.ch

SC’12 Demo Successful SC’12 demo in collaboration with SURFnet, iCAIR and SARA OLiMPS controller used to establish multiple paths between Geneva, Amsterdam, Chicago, and Salt Lake City June 5, 2013 Artur.Barczyk@cern.ch

SC’12 Demo Successful SC’12 demo in collaboration with SURFnet, iCAIR and SARA OLiMPS controller used to establish multiple paths between Geneva, Amsterdam, Chicago, and Salt Lake City June 5, 2013 Artur.Barczyk@cern.ch

Algorithms: Load Balancing (II) More parameters to take into account: Nominal path bandwidth Link utilization along the path Distribute new flow to the least-used path Requested bandwidth when used with API Rebalance based on real-time feedback: Topology changes Congestion notification (maybe – needs detailed study) June 5, 2013 Artur.Barczyk@cern.ch

Topology (IV) Future extensions: Need to extend the topology view from a basic graph representation Nominal/design capacity Available bandwidth Delay June 5, 2013 Artur.Barczyk@cern.ch

Future work: OLiMPS and OSCARS OLiMPS/OSCARS interface User (or application) requests network setup from OLiMPS controller OLiMPS requests setup of multiple paths from OSCARS-IDC OLIMPS connects OpenFlow switches to OSCARS termination points, i.e. VLANs OLiMPS transparently maps the site traffic to the VLANs June 5, 2013 Artur.Barczyk@cern.ch

Lateral thinking: Inter-working with MPTCP MPTCP (Multipath TCP) is a different approach to multipath implemented on end-hosts, uses multiple available interfaces (physical or virtual) Without interface to the network nodes, MPTCP load balancing must rely on the statistical probability of achieving path diversity Approaches exist to make the flow path controlled by end-host/application: for example CFLB, see e.g. http://inl.info.ucl.ac.be/cflb Could easily be integrated in the OLiMPS controller June 5, 2013 Artur.Barczyk@cern.ch

Summary OLiMPS project: Study and implement a multipath fabric applicable in the WAN using Openflow Where we stand: Initial controller implementation with Multipath computation Load balancing Functional extensions to Floodlight: ProxyARP, CLI Successfully demonstrated at SC‘12 Where we are heading: Interface with OSCARS Improved schemes for load balancing Possibly an API for direct interaction with applications June 5, 2013 Artur.Barczyk@cern.ch

Questions? Artur.Barczyk@cern.ch Michael.Bredel@cern.ch June 5, 2013