1. Openflow-based Multipath Switching in Wide Area Networks Michael Bredel (Caltech) Artur Barczyk, Azher Mughal, Harvey Newman (Caltech) Ronald van der Pol (SURFnet) TERENA Networking Conference 2013 Michael.Bredel@cern.ch June 5, 2013

2. Outline Introduction to the problem space –Networking for the big bang theory –Multipath technologies The OLiMPS project –Brief intro to OpenFlow –Objectives and the OLiMPS OpenFlow Controller –Topology discovery, multipathing, and load balancing Current capabilities and testbed results Summary and Outlook Outline Michael.Bredel@cern.ch June 5, 2013

3. Our background: High-performance Networking for High Energy Physics –LHC Optical Private Network High Throughput Networking between LHC Tier1 sites on dedicated infrastructure –LHC Open Networking Environment Design for scalability to over 170 datacenters on 6 continents –Deal with large flows TBytes; hours-days Introduction Michael.Bredel@cern.ch June 5, 2013

4. Multiple paths are used to increase capacity increase resiliency Load balancing is used to increase efficiency in capacity utilization increase robustness reduce/avoid congestion Both have been studied extensively in the past Both can be used to reduce operational costs Why do we need multiple paths? Michael.Bredel@cern.ch June 5, 2013

5. Multipath is not new, various techniques exist: –Layer 3: ECMP – Equal Cost Multi-Path –Layer 2: LAG - Ethernet Link Aggregation Groups –Layer 1: VCAT – Virtual conCATenation Common drawback: None are particularly flexible –preconfigured –static hashing algorithms –limited to single links Often vendor specific implementation Work on hop-by-hop basis, difficult to optimize globally New approaches in the data center: –Layer 2: TRILL - Transparent Interconnection of Lots of Links (IEFT) –Layer 2: SPB - Shortest Path Bridging (IEEE) Multipath: old-style approaches Michael.Bredel@cern.ch June 5, 2013

6. OLiMPS: Openflow Link-layer MultiPath Switching with a centralized, out-of-band control, we can construct a robust multi-path system without modifications to the Layer 2 frame structure Big plus: using Openflow, no need for new HW or feature support (other than Openflow) Addresses the problem of topology limitations in large-scale layer 2 networks Remove the necessity of a tree structure in the topology achieved though the use of Spanning Tree Protocol Allow for per-flow multipath switching Non-equal paths Multiple wide area links The Approach in OLiMPS Michael.Bredel@cern.ch June 5, 2013

7. Software Defined Network and OpenFlow Decouples forwarding and control plane Introduction to OpenFlow (I) Michael.Bredel@cern.ch June 5, 2013 by Scott Shenker

8. Evolution of networks over the last decades The fowarding plane: –10 Mb/s (1983) –100 Mb/s (1995) –1000 Mb/s (2000) –10 Gb/s (2005) –100 Gb/s (2010) –400, 800 Gb/s (2013) –1 Tb/s (???) OK – network is kind of boring … Introduction to OpenFlow (II) Michael.Bredel@cern.ch June 5, 2013

9. Evolution of networks over the last decades The control plane: Introduction to OpenFlow (III) Michael.Bredel@cern.ch June 5, 2013 19902014 TelnetSSH by Rob Sherwood

10. OLiMPS controller is based on Floodlight Written in JAVA Supported by Big Switch Open Source (Apache License) Floodlight already implements a set of basic OpenFlow applications Link Discovery Topology and spanning tree calculation Simple packet forwarding and learning switch Floodlight/OLiMPS OpenFlow Controller Michael.Bredel@cern.ch June 5, 2013

11. OLiMPS provides a set of new features and functionalities VLAN Tunnels between OpenFlow Islands Broadcast filtering, loop prevention ProxyARP Flow cache service Statistics collector Multipath computation Load balancing Configuration and Management, e.g. CLI Floodlight/OLiMPS OpenFlow Controller Michael.Bredel@cern.ch June 5, 2013

12. Floodlight Link Discovery Uses both LLDP and BDDP packets to detect links –A “direct” link will be detected if an LLDP is sent out one port and the same LLDP is received on another port. –Otherwise, it will be a “broadcast” link Based on the link information, Floodlight creates its topology view Uses the notion of ”OpenFlow islands” Topology (I) Michael.Bredel@cern.ch June 5, 2013

13. OLiMPS Topology Service Uses the topology information for multipath calculation and forwarding Allows for multiple paths through non-OpenFlow equipment using VLAN tunnels Topology (II) Michael.Bredel@cern.ch June 5, 2013

14. Multipath calculation gives us multiple paths, with different characteristics Disjoint paths calculation: Dijkstra’s algorithm Graph theory: Suurballe’s algorithm for multiple paths Algorithms: Multipath Computation Michael.Bredel@cern.ch June 5, 2013

15. Load balancing provides the decision how to best use them Random path selection (ECMP-style) –A new flow is mapped to a randomly selected path Round-Robin distribution of flows to available paths –avoids uneven distribution intrinsic to hash-based algorithms –although cannot prevent uneven utilization Number-Of-Flows based path selection –selects the path that has the least number of flows already mapped to it Algorithms: Load Balancing (I) Michael.Bredel@cern.ch June 5, 2013

16. Preliminary results from basic experiments 5 link-disjoint paths Equal link characteristics, i.e. 10 Gbps (but different path characteristics, e.g. the number of hops) Equal flow characteristics, i.e. TCP and 5 GB files Algorithms: Load Balancing (I) Michael.Bredel@cern.ch June 5, 2013

17. Now the cool stuff Easily implement any load distribution algorithm you can think of and take any of the following parameters into account –Nominal path capacity –Link utilization (available bandwidth) along the path –Number of hops –Latency –Number of bits to transfer Rebalance and move flows based on real-time feedback: –Topology changes –Congestion notification (maybe – needs detailed study) Algorithms: Load Balancing (III) Michael.Bredel@cern.ch June 5, 2013

18. Future work: OLiMPS and OSCARS OLiMPS/OSCARS interface User (or application) requests network setup from OLiMPS controller OLiMPS requests setup of multiple paths from OSCARS- IDC OLIMPS connects OpenFlow switches to OSCARS termination points, i.e. VLANs OLiMPS transparently maps the site traffic to the VLANs Michael.Bredel@cern.ch June 5, 2013

19. MPTCP (Multipath TCP) is a different approach to multipath –implemented on end-hosts, uses multiple available interfaces (physical or virtual) Without interface to the network nodes, MPTCP load balancing must rely on the statistical probability of achieving path diversity Approaches exist to make the flow path controlled by end- host/application: for example CFLB, see e.g. http://inl.info.ucl.ac.be/cflb –Could easily be integrated in the OLiMPS controller Lateral thinking: Inter-working with MPTCP Michael.Bredel@cern.ch June 5, 2013

20. OLiMPS project: Study and implement a multipath fabric applicable in the WAN using Openflow What we can do (and demonstrate at our booth) –Multipath computation –Per-flow load balancing –Seamless (manual) flow movement Where we are heading: –Interface with OSCARS –Possibly an API for direct interaction with applications Summary Michael.Bredel@cern.ch June 5, 2013

21. QUESTIONS? Michael.Bredel@cern.ch Artur.Barczyk@cern.ch Michael.Bredel@cern.ch June 5, 2013