0. From GTS to CloudLab: on OpenFlow, NSI, and GTS-GENI or, a SAVI joy ride around Europe (Germany, France, Spain, Denmark) Clemson University KC Wang, Ryan Izard NorduNet Jerry Sobiesky TU-Kaiserslautern Paul Mueller, Dennis Schwerdel

1. FIRE-GENI Workshop September 2015
The SAVI Ride
Ryan had all the fun (and did all the work)
7/6-11 FGENI summer camp
7/13-17 Kaiserslautern
7/18-19 Stuttgart-Paris-Barcelona! (cheapest flight route )
7/20-31 I2CAT
* 10/19-23 Copenhagen NorduNet Conference 2016
Took tons of photos, and
Planned GENI Cinema trial across GENI-ToMaTo-GTS
Planned GENI-GTS-CloudLab federation over Clemson-GTS-Kaiserslautern
Developed OpenFlow Port Proxy on Floodlight for GTS, as part of planned Generalized Virtualization Framework (GVF)
Key technologies involved
GENI (& rack), GTS (& rack), ToMaTo (& rack), CloudLab
Network: GENI VLAN stitching, AL2S, NSI w/ EoMPLS, OpenFlow 1.0, 1.3
FIRE-GENI Workshop September 2015

2. OpenFlow Port Proxy Design for use in GTS
Ryan Izard

3. Background Network switches have physical port numbers
e.g. 1, 2, …, 47, 48
OpenFlow (OF) instances on a physical OF switch partition the switch into many logical OF switches
OF instance exposed to controller as unique datapath ID
Physical ports set/configured for each OF instance OF
Instance 1
DPID=X
Instance 2
DPID=Y
Instance 3
DPID=Z
Controller

4. Problem
How should the physical switch expose the port numbers to the controller for each OF instance?

5. The Simple Approach
Make each OF instance appear as the actual, physical partition of the physical switch
Port numbers exposed to controller as physical ports themselves
e.g. 17, 24, 30, 33 OF
Instance 1
DPID=X
Instance 2
DPID=Y
Instance 3
DPID=Z
Controller
17,24,30,33 17 24 30 33

6. The More Conscientious Approach
Make each OF instance appear as an independent switch
Port numbers exposed to controller as constant sequence of ports
e.g. 1, 2, 3, 4
Physical ports mapped to virtual ports
phys virt
17 <--> 1
24 <--> 2
30 <--> 3
33 <--> 4 OF
Instance 1
DPID=X
Instance 2
DPID=Y
Instance 3
DPID=Z
Controller 24 17 30 33
1, 2, 3, 4

7. Why Mapping the Ports is Better
Use Physical Port IDs
Use Constant Port IDs
Controller “sees” real network configuration (useful for admins?)
Inconsistent port numbers for switch users
Requires user apps handle/learn port numbers
User controller “sees” same switch representation for each experiment
No need for user to handle variable port numbers
More portable user experiments/code

8. How to Implement Port Mapping?
Have the switch vendor do it
Some switch vendors use constant, sequential port numbers for OF instances already (yay!)
…but others do not (aww)
Vendors not likely to willingly expose their OF SDK
Requires custom solution possibly per vendor, per model, and per firmware version
Not practical long-term

9. How to Implement Port Mapping?
Do it ourselves
Idea #1: Proxy talks to switch as a user controller; proxy talks to user controller as a switch
Perform mapping within
Hmm, sounds a lot like FlowVisor (minus port mapping)
FlowVisor only designed for OF 1.0; we need OF 1.3
Idea #2: Use a transparent proxy to perform the mapping on the live switch-to-controller connection
Need to intercept each control packet and swap the ports
Sounds like a good use-case for OpenFlow itself!

10. Proxy Design

11. Proxy Design: Based on OpenFlow
OF can manipulate other OF control packets
They’re just TCP packets that happen to contain OF data
Where can we perform the rewrite to map the ports?
Send to application
Send to the controller
In the switch itself with experimenter match/actions
Flexible rewrite location, but chose controller for simplicity
Not limited to mapping port numbers
Can map any OpenFlow type
DPID, switch attributes, port MAC addresses, etc.
Can map like-features between OpenFlow versions

12. Proxy Design: What Needs Mapping?
Flow mods aren’t the only OpenFlow messages that contain switch port numbers:
Flow mods
Group mods
Port configuration
Queue configuration
Flow removed
Statistics requests/replies
Multipart requests/replies
Features reply
Packet-in/out

13. Proxy Design: Floodlight Controller
Java-based OpenFlow controller
Implement proxy as a Java module
OpenFlow support
OpenFlowJ-Loxi library
Facilitates high-level coding
Relatively simple to implement an OF version-agnostic solution

14. Proxy Design: GTS Integration
One proxy per site/RCA (Remote Control Agent)
RCA communicates with proxy
JSON over secure REST API
Informs of new connections, the port mappings, and other mappings to perform (such as DPID if desired)
Differentiate user control connections by their VLAN IDs
Necessary b/c users can choose their own IP subnets

15. Useful for Migration Support
Need to maintain virtual ports when moving a user’s switch from one OF instance to another
RCA issues proxy “snapshot” command to capture user flowspace for a particular switch; proxy provides flow information
GTS determines new location for user’s switch
RCA at new location notifies proxy of new connection and of the flows to preinstall
Proxy inserts flows w/new physical port mapping on the switch prior to allowing user controller to handshake
User’s controller reconnects; flows are the logically equivalent (minus those with timeouts)
Note GTS/RCA is responsible for maintaining a consistent port mapping with respect to the topology

16. Migration Example Before Migration After Migration 24 17 30 33 39 34OF
Instance 1
DPID=X
Instance 2
DPID=Y
Instance 3
DPID=?
Controller 24 17 30 33
1, 2, 3, 4 OF
Instance 1
DPID=X
Instance 2
DPID=?
Instance 3
DPID=Y
Controller 39 34 47 48
1, 2, 3, 4
OF Instance 2 to
OF Instance 3

17. Progress and Future Work
Implemented in the Floodlight controller
~5,000 lines of code in module at present
Test framework with VMs will evaluate performance
VM for proxy (Floodlight + Open vSwitch)
VM for switch #1 (i.e. OF instance #1)
VM for switch #2 (i.e. OF instance #2)
VM for user controller

18. GEANT Testbeds Service (GTS) Update
Jerry Sobieski
Activity Leader, SA2 TEstbeds
Director, Int’l Research Initiatives, NORDUnet
FIRE GENI Workshop, Washington DC
Sep 17, 2015

19. GEANT Testbeds Service Status Sep 2015
NYC
GTS v2.0 in production in GEANT
7 locations now installed and operational:
LON, AMS, BRA, PRG*, LJU, MIL, HAM
NYC, PAR, <tbd> soon...
220+ VMs / 28 servers, 7 OpenFlow switches
30+ projects
HAM
AMS
tbd
PRG
LON
PAR
LJU
BRA
MIL

20. Key GTS demonstration (since Paris)
External Domains w/ ExoGENI Mar 2015 and w/ ONOS June 2015
Dataplane inter-connection across Atlantic
jFED GUI (iMinds) testbed construction:
Drag-n-Drop GTS network specification - June 2015 for EC review
Graphical real-time control of life-cycle from jFED GUI
ONOS (ON.lab, GARR, U of Rome, Createnet):
ONF/June 2015, SIGCOMM/Aug 2015
35+ GTS resources!
Stable! In continuous service since June (except for one planned maintenance window)

21. GTS Coming up: CY2015-Q4
New GTS pods: #8 NYC (MANLAN), #9 PAR (Paris), #10 TBD (suggestions?)
NSI peering:
Oct 2015: GEANT BoD (and thru BoD will reach other NSI global domains)
Nov/Dec: StarLight, GLIF AutoGOLE, CloudLab (Clemson)
Multi-domain One-Stop-Shopping – Currently in lab testing
jFED full lifecycle testbed control via API
eduGAIN integration (federated Authentication)
Had to refine API AA hooks first – done.
GTS Tech+Futures Workshop #1
Define the long term development roadmap for GTS
Oct Copenhagen

22. GTS Next Spring: CY2016-H1 Multi-domain deployments:
HEAnet – Dublin
DFN – Nuremburg (Erlangen), <tbd>
CESnet – locations tbd
NORDUnet – Copenhagen, Geneva, Washington DC, Chicago-SL (?)
Cloudlab (Clemson pilot) – Clemson (CUE), Starlight, <+1 tbd>
GTS features 2016-Q1 (eta)
10/100 Gbps (via GEANT core MX integration)
Bare Metal Server resources
Storage resources (plus major storage upgrade)
Checkpoint/Restart

23. Collabortive efforts Grad student Ryan Izard (Clemson) GENI-GTS (SAVI)
Ryan worked in Barcelona over the summer at I2Cat (yeah..“worked”)
Developed virtual port mapper for OFX resources
Present results to the GTS Tech+Futures Workshop (Oct ’15)
Next steps:
Abstract OFX Resource Control Agent (eta 2016-H1)
CloudLab GTS/NSI
Grad Student Richard Cziva (Univ Glasgow, Scottish Research Council)
Intern w NORDUnet Spring 2016
Topics: PV/AFU (Performance Verification and Automated Fault Uncovery) in virtualized cyber-infrastructure environments

24. Collabortive efforts Fed4FIRE (iMinds Ghent) CloudLab (Clemson Univ.)
jFED – Intuitive glass cockpit for testbed from cradle to grave
CloudLab (Clemson Univ.)
NSI v2.x deployment – Interdomain Virtual Circuit provisioning standard
Lots of aspects of NSI to leverage
NSI v3 ? - Many new features yet to be integrated and standardized
GVS (Generalized Virtualization Services) Pilot
Nodes at Clemson and StarLight.. Elsewhere?
ExoGENI (RENCI RTP)
Initial efforts focused on data plane connectivity (thru AL2S and MANLAN)
Hope to pursue the NSI based provisioning after SC15 and into 2016-Q1
StarLight (iCAIR Chicago)
Will be placing a GTS node in SL for SC15 demos...plan to leave it.

25. Questions?