1. OpenFlow for Campuses A Tutorial at GEC10 March 15, 2011
Srini Seetharaman, DT R&D Lab, USA
Masa Kobayashi, NEC Labs/Stanford

2. Introductions

3. Agenda Time Description 13:00-14:00
Introduction: How OpenFlow works, Potential, Limitations, Current vendors, SDN/OPEN
14:00-14:30
Show tools, demo mininet and NOX intro for researcher
14:30-15:00
Configuring NEC, HP and Pronto switches
Break
15:30-16:15
Controller demonstrations (SNAC, BigSwitch’s controller and NEC’s PFC)
16:15-16:45
Slice using FlowVisor, Expedient, and opt-in manager
16:45-17:00
Q&A, Discussions, Community building, Conclusions

4. Goals of this Tutorial By the end, everyone should know:
what OpenFlow is
how it’s used and how you can use it
where it’s going
how OpenFlow fits in the Software-Defined Networking (SDN) spectrum
how to slice your network
Present a useful mix of lecture-based content and actual operational info
Have fun
What: protocol,

5. Why OpenFlow?

6. Specialized Packet Forwarding Hardware
The Ossified Network
Routing, management, mobility management, access control, VPNs, …
Feature
Feature
Million of lines of source code
5400 RFCs
Barrier to entry
Operating
System
Specialized Packet Forwarding Hardware
Billions of gates
Bloated
Power Hungry
Many complex functions baked into the infrastructure
OSPF, BGP, multicast, differentiated services, Traffic Engineering, NAT, firewalls, MPLS, redundant layers, …
An industry with a “mainframe-mentality”, reluctant to change 6 6

7. Research Stagnation Lots of deployed innovation in other areas
OS: filesystems, schedulers, virtualization
DS: DHTs, CDNs, MapReduce
Compilers: JITs, vectorization
Networks are largely the same as years ago
Ethernet, IP, WiFi
Rate of change of the network seems slower in comparison
Need better tools and abstractions to demonstrate and deploy
In these other areas, you can real systems on top of high-quality, open platforms, and you don’t need to reinvent the wheel

8. Closed Systems (Vendor Hardware)
Stuck with interfaces (CLI, SNMP, etc)
Hard to meaningfully collaborate
Vendors starting to open up, but not usefully
Need a fully open system – a Linux equivalent

9. none have all the desired attributes!
Open Systems
Performance Fidelity
Scale
Real User Traffic?
Complexity
Open
Simulation
medium no
yes
Emulation
low
Software Switches
poor
NetFPGA
high
Network Processors
Vendor Switches
gap in the tool space
none have all the desired attributes!

10. See Ethane SIGCOMM 2007 paper for details
Ethane, a precursor to OpenFlow Centralized, reactive, per-flow control
Controller
Flow Switch
Flow Switch
Flow Switch
Host B
Host A
Flow Switch
See Ethane SIGCOMM 2007 paper for details

11. OpenFlow: a pragmatic compromise
+ Speed, scale, fidelity of vendor hardware
+ Flexibility and control of software and simulation
Vendors don’t need to expose implementation
Leverages hardware inside most switches today (ACL tables)

12. How does OpenFlow work? 12

13. Ethernet Switch

14. Control Path (Software)
Data Path (Hardware)

15. OpenFlow Controller Control Path OpenFlow Data Path (Hardware)
OpenFlow Protocol (SSL/TCP)
Control Path
OpenFlow
Data Path (Hardware)

16. OpenFlow Client Controller PC OpenFlow Example Software Layer MAC src
Flow Table
MAC
src
dst IP
Src
Dst
TCP
sport
dport
Action
Hardware
Layer *
port 1
port 1
port 2
port 3
port 4

17. OpenFlow Basics Flow Table Entries
Rule
Action
Stats
Packet + byte counters
Forward packet to zero or more ports
Encapsulate and forward to controller
Send to normal processing pipeline
Modify Fields
Any extensions you add!
Now I’ll describe the API that tries to meet these goals.
Switch
Port
VLAN ID
VLAN
pcp
MAC
src
MAC
dst
Eth
type IP
Src IP
Dst IP
ToS IP
Prot L4
sport L4
dport
+ mask what fields to match

18. Examples Switching Flow Switching Firewall Switch Port MAC src dst Eth
type
VLAN ID IP
Src
Dst
Prot
TCP
sport
dport
Action * *
00:1f:.. * * * * * * *
port6
Flow Switching
Switch
Port
MAC
src
dst
Eth
type
VLAN ID IP
Src
Dst
Prot
TCP
sport
dport
Action
port3
00:20..
00:1f..
0800
vlan1 4
17264 80
port6
Firewall
Switch
Port
MAC
src
dst
Eth
type
VLAN ID IP
Src
Dst
Prot
TCP
sport
dport
Action * * * * * * * * * 22
drop

19. Examples Routing VLAN Switching Switch Port MAC src dst Eth type VLANID IP
Src
Dst
Prot
TCP
sport
dport
Action * * * * * * * * *
port6
VLAN Switching
Switch
Port
MAC
src
dst
Eth
type
VLAN ID IP
Src
Dst
Prot
TCP
sport
dport
Action
port6,
port7,
port9 * *
00:1f.. *
vlan1 * * * * *

20. Centralized vs Distributed Control Both models are possible with OpenFlow
Centralized Control
Distributed Control
Controller
Controller
OpenFlow
Switch
OpenFlow
Switch
Controller
OpenFlow
Switch
OpenFlow
Switch
Controller
OpenFlow
Switch
OpenFlow
Switch

21. Flow Routing vs. Aggregation Both models are possible with OpenFlow
Flow-Based
Every flow is individually set up by controller
Exact-match flow entries
Flow table contains one entry per flow
Good for fine grain control, e.g. campus networks
Aggregated
One flow entry covers large groups of flows
Wildcard flow entries
Flow table contains one entry per category of flows
Good for large number of flows, e.g. backbone

22. Reactive vs. Proactive (pre-populated) Both models are possible with OpenFlow
First packet of flow triggers controller to insert flow entries
Efficient use of flow table
Every flow incurs small additional flow setup time
If control connection lost, switch has limited utility
Proactive
Controller pre-populates flow table in switch
Zero additional flow setup time
Loss of control connection does not disrupt traffic
Essentially requires aggregated (wildcard) rules

23. Usage examples Alice’s code: Simple learning switch Per Flow switching
Network access control/firewall
Static “VLANs”
Her own new routing protocol: unicast, multicast, multipath
Home network manager
Packet processor (in controller)
IPvAlice
VM migration
Server Load balancing
Mobility manager
Power management
Network monitoring and visualization
Network debugging
Network slicing
What is possible in the controller? Anything that needs intelligent routing of a flow
At Stanford, we have even shown how OpenFlow may be used for:
VM migration
Power management
Load balancing
Network monitoring and debugging
Easier network visualization
… and much more you can create! 23 23

24. Quiz Time
How do I provide control connectivity? Is it really clean slate?
Why aren’t users complaining about time to setup flows over OpenFlow? (Hint: What is the predominant traffic today?)
Considering switch CPU is the major limit, how can one take down an OpenFlow network?
How to perform topology discovery over OpenFlow-enabled switches?
What happens when you have a non-OpenFlow switch inbetween?
What if there are two islands connected to same controller?
How scalable is OpenFlow? How does one scale deployments?

25. What can you not do with OpenFlow ver1.0
Non-flow-based (per-packet) networking
ex. Per-packet next-hop selection (in wireless mesh)
yes, this is a fundamental limitation
BUT OpenFlow can provide the plumbing to connect these systems
Use all tables on switch chips
yes, a major limitation (cross-product issue)
BUT an upcoming OF version will expose these

26. What can you not do with OpenFlow ver1.0
New forwarding primitives
BUT provides a nice way to integrate them through extensions
New packet formats/field definitions
BUT a generalized OpenFlow (2.0) is on the horizon
Optical Circuits
BUT efforts underway to apply OpenFlow model to circuits
Low-setup-time individual flows
BUT can push down flows proactively to avoid delays

27. Where it’s going OF v1.1: Extensions for WAN, spring 2011 OF v2+
multiple tables: leverage additional tables
tags and tunnels
multipath forwarding
OF v2+
generalized matching and actions: an “instruction set” for networking

28. OpenFlow Implementations (Switch and Controller)28

29. OpenFlow building blocks
Monitoring/ debugging tools
oftrace
oflops
openseer
Stanford Provided
ENVI (GUI)
LAVI
n-Casting
Expedient
Applications
NOX
Beacon
Helios
Maestro
SNAC
Controller
Slicing
Software
FlowVisor
Console
FlowVisor
There are components at different levels that work together in making it work
The commercial switch details will follow in next slide
There are a plethora of applications possible. I only list those available at Stanford
Commercial Switches
Stanford Provided
Software
Ref. Switch
NetFPGA
Broadcom
Ref. Switch
HP, NEC, Pronto, Juniper.. and many more
OpenFlow
Switches
OpenWRT
PCEngine WiFi AP
OpenVSwitch 29

30. Current SDN hardware More coming soon... Juniper MX-series WiMax (NEC)
NEC IP8800
UNIVERGE PF5240
WiMax (NEC)
HP Procurve 5400
Netgear 7324
PC Engines
Pronto 3240/3290
Ciena Coredirector
More coming soon... 30

31. Commercial Switch Vendors
Model
Virtualize
Notes
HP Procurve 5400zl or 6600
1 OF instance per VLAN
LACP, VLAN and STP processing before OpenFlow
Wildcard rules or non-IP pkts processed in s/w
Header rewriting in s/w
CPU protects mgmt during loop
NEC
IP8800 Series and
UNIVERGE PF5240
OpenFlow takes precedence
Most actions processed in hardware
MAC header rewriting in h/w
More than 100K flows (PF5240)
Pronto 3240 or 3290 with Pica8 or Indigo firmware
1 OF instance per switch
No legacy protocols (like VLAN and STP)
All support ver 1.0
All have approx 1500 flow table entry limit 31

32. Controller Vendors Vendor Notes Nicira’s NOX Nicira’s ONIX SNAC Vendor
Open-source GPL
C++ and Python
Researcher friendly
Nicira’s ONIX
Closed-source
Datacenter networks
SNAC
Code based on NOX0.4
Enterprise network
C++, Python and Javascript
Currently used by campuses
Vendor
Notes
Stanford’s Beacon
Open-source
Researcher friendly
Java-based
BigSwitch controller
Closed source
Based on Beacon
Enterprise network
Maestro (from Rice Univ)
Based on Java
NEC’s Helios
Written in C and Ruby
NEC UNIVERGE PFC
Based on Helios 32

33. Providers and business-unit
Growing Community
Vendors and start-ups
Providers and business-unit
More...
More...
Note: Level of interest varies 33

34. Software-Defined Networking (SDN)34

35. Current Internet Closed to Innovations in the Infrastructure Closed
App
Operating
System
App
Specialized Packet Forwarding Hardware
Operating
System
App
Specialized Packet Forwarding Hardware
Operating
System
The next 3 slides are a set of animation to show how we enable innovation:
- Infrastructure is closed to innovation and only driven by vendors. Consumers have little say
- Business model makes it hard for new features to be added
App
Specialized Packet Forwarding Hardware
Operating
System
Specialized Packet Forwarding Hardware
App
Operating
System
Specialized Packet Forwarding Hardware 35

36. “Software Defined Networking” approach to open it
Network Operating System
App
Operating
System
App
Specialized Packet Forwarding Hardware
Operating
System
App
Specialized Packet Forwarding Hardware
Operating
System
How do we redefine the architecture to open up networking infrastructure and the industry!
By bring to the networking industry what we did to the computing world
App
Specialized Packet Forwarding Hardware
Operating
System
Specialized Packet Forwarding Hardware
App
Operating
System
Specialized Packet Forwarding Hardware 36

37. The “Software-defined Network”
2. At least one good operating system
Extensible, possibly open-source
3. Well-defined open API
App
App
App
Network Operating System
1. Open interface to hardware
Simple Packet Forwarding Hardware
Simple Packet Forwarding Hardware
Switches, routers and other middleboxes are dumbed down
The key is to have a standardized control interface that speaks directly to hardware
Simple Packet Forwarding Hardware
Simple Packet Forwarding Hardware
Simple Packet Forwarding Hardware 37

38. Mininet Setup h2 h3 h4 Controller port6633 c0 OpenFlow Tutorial
loopback
( :6633)
OpenFlow Tutorial
3hosts-1switch
topology s1
OpenFlow Switch
loopback
( :6634)
dpctl
(user space process)
s1-eth0
s1-eth1
s1-eth2
h1-eth0
h3-eth0
h4-eth0 h2 h3 h4
virtual hosts

39. Switch Configuration Setup
Of-demo-5 [eth1]
00:1b:21:5d:4c:3d
Of-demo-6 [eth1]
00:1b:21:5d:4e:c5
GUI
A13 13
A14
Disabled Backup 14
hpsw2 15
necsw5
A15
Controller 13 14
Pronto 3290

40. Virtualizing OpenFlow40

41. Virtualization or “Slicing”
Trend
Controller 1
App
Controller 2
Virtualization or “Slicing”
OpenFlow
NOX
(Network OS)
Network OS
App
App
App
Windows
(OS)
Linux
Mac OS
Windows
(OS)
Linux
Mac OS
Windows
(OS)
Linux
Mac OS
Virtualization layer
x86
(Computer)
Hidden slide (just for backup reasons)
Shows how far along we can go in opening up the network
Computer Industry
Network Industry 41

42. Virtualization or “Slicing” Layer
Isolated “slices”
Many operating systems, or
Many versions
App
Network Operating System 1
Network Operating System 2
Network Operating System 3
Network Operating System 4
Open interface to hardware
Virtualization or “Slicing” Layer
Open interface to hardware
Simple Packet Forwarding Hardware
Simple Packet Forwarding Hardware
Simple Packet Forwarding Hardware
Simple Packet Forwarding Hardware
Simple Packet Forwarding Hardware

43. Switch Based Virtualization Exists for NEC, HP switches but not flexible enough
Controller
Research VLAN 2
Flow Table
Controller
Research VLAN 1
Flow Table
Production VLANs
Normal L2/L3 Processing
Experiments running on PRODUCTION infrastructure
Key to get scale, key to get traffic on the network
(e.g. can’t just do a reset...)

44. FlowVisor-based Virtualization
Heidi’s
Controller
Craig’s
Controller
Aaron’s
Controller
Topology discovery is per slice
OpenFlow
Protocol
OpenFlow FlowVisor & Policy Control
OpenFlow
Switch
OpenFlow
Protocol
OpenFlow
Switch
OpenFlow
Switch 44

45. FlowVisor-based Virtualization
http
Load-balancer
Multicast
Broadcast
Separation not only
by VLANs, but any
L1-L4 pattern
OpenFlow
Protocol
dl_dst=FFFFFFFFFFFF
tp_src=80, or tp_dst=80
OpenFlow
FlowVisor & Policy Control
OpenFlow
Switch
OpenFlow
Protocol
OpenFlow
Switch
OpenFlow
Switch

46. FlowSpace: Maps Packets to Slices

47. FlowVisor Message Handling
OpenFlow
Firmware
Data Path
Alice
Controller
Bob
Cathy
FlowVisor
Rule
Policy Check:
Is this rule allowed?
Policy Check:
Who controls this packet?
Full Line Rate
Forwarding
Exception
Packet
Packet

48. Use Case: New CDN - Turbo Coral ++
Basic Idea: Build a CDN where you control the entire network
All traffic to or from Coral IP space controlled by Experimenter
All other traffic controlled by default routing
Topology is entire network
End hosts are automatically added (no opt-in)
Switch
Port
MAC
src
MAC
dst
Eth
type
VLAN ID IP
Src IP
Dst IP
Prot
TCP
sport
TCP
dport *
84.65.* *

49. Use Case: Aaron’s IP A new layer 3 protocol Replaces IP
Defined by a new Ether Type
Switch
Port
MAC
src
MAC
dst
Eth
type
VLAN ID IP
Src IP
Dst IP
Prot
TCP
sport
TCP
dport *
AaIP *
!AaIP

50. OpenFlow/GENI Deployments50

51. OpenFlow Deployment at Stanford
Switches (23)
APs (50)
WiMax (1)

52. Live Stanford Deployment Statistics

53. How did we get there? Staged Deployment of OpenFlow
Add experimental VLAN
Enable OpenFlow for Exptl VLAN
Configure Controller for new network
Add new Production subnetwork
Gradually add/move users to new subnet
Enable OpenFlow for new subnet
Slice the network
Verify correctness and performance
Verify reachability

54. GENI OpenFlow deployment (2010)
10 institutions and 2 National Research Backbones
Kansas State

55. GENI Network Evolution
National Lambda Rail

56. GENI Integration FlowVisor Expedient Opt-in Manager Slicing control
Experimenter’s portal for slice management
Opt-in Manager
Network admins’ portal to approve/ deny expt requests for traffic
Expedient3
GENI API
API X
Expedient1
Expedient2
API X
API X
Opt-in Mgr1
Opt-in Mgr2
FlowVisor API
FlowVisor API
FlowVisor1
FlowVisor2
OpenFlow
OpenFlow
Substrate 1
Substrate 2

57. Mistakes we made OpenFlow over Q-in-Q
OpenFlow routing is unaware and sends traffic with same MAC address in both direction, causing perpetual learning and CPU inflation
Moving uplinks for 1 switch, while being pointed to the same controller (causing two islands)
Causes controller learning to oscillate between the 2 uplinks
Bad interaction with legacy protocols
LLDP and STP are treated differently with different switches
Loop in OpenFlow network being exposed to non-OF side
Miscommunication between the aggregate operator and the experimenter during testing phase
Loop across backbones
Same campus connected over NLR and Internet2

58. Next steps for GENI Remove duct-tape Be better prepared Grow topology
Fix any issues that we learned about during demos
Be better prepared
Higher stability and better isolation
Test bandwidth slicing
Grow topology
Add more switches and hosts
Wean off
Each campus takes charge of control framework
33 bugs to be fixed in FlowVisor

59. Tutorial Setup so far h2 h3 h4 Controller port6633 c0
loopback
( :6633)
OpenFlow Tutorial
3hosts-1switch
topology s1
OpenFlow Switch
loopback
( :6634)
dpctl
(user space process)
s1-eth0
s1-eth1
s1-eth2
h1-eth0
h3-eth0
h4-eth0 h2 h3 h4
virtual hosts 59

60. Virtualized Network Setup
Controller
port 7000
loopback
( :7000) c0
FlowVisor
port 6633
OpenFlow Tutorial
3hosts-1switch
topology
loopback
( :6634) s1
OpenFlow Switch
loopback
( :6634)
dpctl
(user space process)
s1-eth0
s1-eth1
s1-eth2
h1-eth0
h3-eth0
h4-eth0 h2 h3 h4
virtual hosts 60

61. FlowVisor Usage man ./doc/flowvisor.8
<flowvisor dir>/script/fvctl.sh
listDevices
list of all OpenFlow switches’ datapath ID
getLinks
list of all links (port # and datapath ID of both end switches)
createSlice
creating slice (specifying controller’s URL and slice name)
listSlices
addFlowSpace
add flow space to slices
listFlowSpace
show current flow space 61

62. Concluding Remarks 62

63. Highlights of Deployments
Stanford deployment
McKeown group for 1.5 years: production and experiments
To scale later this year to entire building (~500 users)
Nation-wide trials and deployments
7 other universities and BBN deploying now
GEC9 in Nov, 2010 showcased nation-wide OF
Internet 2 and NLR starting to serve as the GENI Backbone
Global trials
Over 60 organizations experimenting
2011 likely to be a big year for OpenFlow

64. Current Trials
68 trials/deployments spanning 13 countries

65. Get involved! Ask and answer questions on mailing lists:
openflow-discuss
openflow-spec
Share and update wiki content
Submit bug-reports and/or patches to OF reference implementation
Release open-source applications
Write a controller!

66. Are you innovating in your network?

67. SDN Team at Stanford