Programming Abstractions & Languages for SDN: Frenetic & Pyretic

Network Programming is Hard Programming network equipment is hard –Complex software by equipment vendors –Complex configuration by network administrators Expensive and error prone –Network outages and security vulnerabilities –Slow introduction of new features SDN gives us a chance to get this right! –Rethink abstractions for network programming 2

Programming Software Defined Networks OpenFlow already helps a lot –Network-wide view at controller –Direct control over data plane The APIs do not make it easy –Limited controller visibility –No support for composition –Asynchronous events 3 Controller Switches

Example Network 4 Internet Servers B A SDN Switch w/ labeled ports

Counters for each rule - #bytes, #packets A Simple OpenFlow Program 5 Pattern Action Priority Route: IP/fwd B A :dstip=A -> fwd(2) 1:* -> fwd(1) 2:dstip=B -> fwd(3) OpenFlow Program dstip=A dstip=B dstip!=A dstip!=B

6 1:dstmac=A -> fwd(2) 1:dstmac=B -> fwd(3) 2:* -> fwd(1) Pattern Switch: MAC/fwd B A One API, Many Uses Priority Ordered Action

7 Load Balancer: IP/mod B A Pattern Action srcip=0*,dstip=P -> mod(dstip=A) srcip=1*,dstip=P -> mod(dstip=B) One API, Many Uses

Composition: Simple Repeater def switch_join(switch): # Repeat Port 1 to Port 2 p1 = {in_port:1} a1 = [forward(2)] install(switch, p1, DEFAULT, a1) # Repeat Port 2 to Port 1 p2 = {in_port:2} a2 = [forward(1)] install(switch, p2, DEFAULT, a2) def switch_join(switch): # Repeat Port 1 to Port 2 p1 = {in_port:1} a1 = [forward(2)] install(switch, p1, DEFAULT, a1) # Repeat Port 2 to Port 1 p2 = {in_port:2} a2 = [forward(1)] install(switch, p2, DEFAULT, a2) Simple Repeater 12 Controller When a switch joins the network, install two forwarding rules.

Composition: Web Traffic Monitor 9 def switch_join(switch)): # Web traffic from Internet p = {inport:2,tp_src:80} install(switch, p, DEFAULT, []) query_stats(switch, p) def stats_in(switch, p, bytes, …) print bytes sleep(30) query_stats(switch, p) def switch_join(switch)): # Web traffic from Internet p = {inport:2,tp_src:80} install(switch, p, DEFAULT, []) query_stats(switch, p) def stats_in(switch, p, bytes, …) print bytes sleep(30) query_stats(switch, p) Monitor “port 80” traffic 12 Web traffic When a switch joins the network, install one monitoring rule.

Composition: Repeater + Monitor def switch_join(switch): pat1 = {inport:1} pat2 = {inport:2} pat2web = {in_port:2, tp_src:80} install(switch, pat1, DEFAULT, None, [forward(2)]) install(switch, pat2web, HIGH, None, [forward(1)]) install(switch, pat2, DEFAULT, None, [forward(1)]) query_stats(switch, pat2web) def stats_in(switch, xid, pattern, packets, bytes): print bytes sleep(30) query_stats(switch, pattern) def switch_join(switch): pat1 = {inport:1} pat2 = {inport:2} pat2web = {in_port:2, tp_src:80} install(switch, pat1, DEFAULT, None, [forward(2)]) install(switch, pat2web, HIGH, None, [forward(1)]) install(switch, pat2, DEFAULT, None, [forward(1)]) query_stats(switch, pat2web) def stats_in(switch, xid, pattern, packets, bytes): print bytes sleep(30) query_stats(switch, pattern) Repeater + Monitor Must think about both tasks at the same time.

Controller Platform Switch API (OpenFlow) Monolithic Controller Switches App Runtime OpenFlow Programming Stack Control Flow, Data Structures, etc. *First Order Approximation

Limited Controller Visibility Example: MAC-learning switch –Learn about new source MAC addresses –Forward to known destination MAC addresses Controller program is more complex than it seems –Cannot install destination-based forwarding rules –… without keeping controller from learning new sources Solution: rules on 12 Must think about reading and writing at the same time sends to 2  learn 1, install 3 sends to 1  never learn 3 1 sends to 3  always floods

Asynchrony: Switch-Controller Delays Common OpenFlow programming idiom –First packet of a flow goes to the controller –Controller installs rules to handle remaining packets What if more packets arrive before rules installed? –Multiple packets of a flow reach the controller What if rules along a path installed out of order? –Packets reach intermediate switch before rules do 13 Must think about all possible event orderings. Controller packets

Wouldn’t It Be Nice if You Could… Separate reading from writing –Reading: specify queries on network state –Writing: specify forwarding policies Compose multiple tasks –Write each task once, and combine with others Prevent race conditions –Automatically apply forwarding policy to extra packets 14 This is what Frenetic does!

Programming SDN w/ OpenFlow 15 Images by Billy Perkins The Good –Network-wide visibility –Direct control over the switches –Simple data-plane abstraction The Bad –Low-level programming interface –Functionality tied to hardware –Explicit resource control The Ugly –Non-modular, non-compositional –Challenging distributed programming

Controller Platform Switch API (OpenFlow) Monolithic Controller Switches App Runtime Today: OpenDaylight, POX, etc. Data and Control Flow ~ Registers, adders, etc. ~ Assembly language

Programming Software Defined Networks OpenFlow already helps a lot –Network-wide view at controller –Direct control over data plane The APIs do not make it easy –Limited controller visibility –No support for composition –Asynchronous events Frenetic simplifies the programmer’s life –A language that raises the level of abstraction –A run-time system that handles the gory details 17 Controller Switches

Frenetic Language Reads: query network state –Queries can see any packets –Queries do not affect forwarding –Language designed to keep packets in data plane Writes: specify a forwarding policy –Policy separate from mechanism for installing rules –Streams of packets, topology changes, statistics, etc. –Library to transform, split, merge, and filter streams Current implementation –A collection of Python libraries on top of NOX 16

Example: Repeater + Monitor 19 # Static repeating between ports 1 and 2 def repeater(): rules=[Rule(inport:1, [forward(2)]), Rule(inport:2, [forward(1)])] register(rules) # Static repeating between ports 1 and 2 def repeater(): rules=[Rule(inport:1, [forward(2)]), Rule(inport:2, [forward(1)])] register(rules) # Monitoring Web traffic def web_monitor(): q = (Select(bytes) * Where(inport:2 & tp_src:80) * Every(30)) q >> Print() # Monitoring Web traffic def web_monitor(): q = (Select(bytes) * Where(inport:2 & tp_src:80) * Every(30)) q >> Print() # Composition of two separate modules def main(): repeater() web_monitor() # Composition of two separate modules def main(): repeater() web_monitor() Repeater Monitor Repeater + Monitor

Frenetic System Overview High-level language –Query language –Composition of forwarding policies Run-time system –Interprets queries and policies –Installs rules and tracks statistics –Handles asynchronous events 20

Frenetic Run-Time System Rule granularity –Microflow: exact header match –Wildcard: allow “don’t care” fields Rule installation –Reactive: first packet goes to controller –Proactive: rules pushed to the switches Frenetic run-time system –Version 1.0: reactive microflow rules [ICFP’11] –Version 2.0: proactive wildcard rules [POPL’12] Get it right once, and free the programmer! 21

Evaluation Example applications –Routing: spanning tree, shortest path –Discovery: DHCP and ARP servers –Load balancing: Web requests, memcached queries –Security: network scan detector, DDoS defenses Performance metrics –Language: lines of code in applications Much shorter programs, especially when composing –Run-time system: overhead on the controller Frenetic 1.0: competitive with programs running on NOX Frenetic 2.0: fewer packets to the controller 22

Conclusions Frenetic foundation –Separating reading from writing –Explicit query subscription and policy registration –Operators that transform heterogeneous streams Makes programming easier –Higher-level patterns –Policy decoupled from mechanism –Composition of modules –Prevention of race conditions And makes new abstractions easier to build! 23

Modular SDN Programming w/ Pyretic

Pyretic Controller Platform Switch API Programmer API (OpenFlow) LB Route Monitor FW Switches Apps Runtime (Pyretic) Pyretic: Language & Platform

{Frene,Pyre}tic? Frenetic is a SDN umbrella project focused on –Language design –Compiler implementation Princeton/Cornell Team Produced: –Great (northbound) abstraction for programmers –Techniques for implementing these abstractions Two current members of the family –Pyretic: embedded & implemented in Python –Frenetic Ocaml: embedded & implemented in OCaml 26

Pyretic or Frenetic Ocaml? 27 PyreticFrenetic Ocaml Target Community Systems and Networking Programming Languages Primary Dev. Location PrincetonCornell Host LanguagePythonOCaml

Pyretic Philosophy Provide high-level programming abstractions Based on state-of-the-art PL techniques Implement w/ state-of-the-art systems tech Package for the networking community Focus on real world utility and cutting edge features 28

Open Source and Programmer-Friendly Embedded and Implemented in Python –Rich support for dynamic policies –Familiar constructs and environment –Python library / module system Default 3-clause BSD license Active & growing developer community –GitHub repository & wiki –Video tutorials and documentation

Basic Programmer Wishlist Encode policy concisely Write portable code Specify traffic of interest Compose code from independent modules Query network state Modify policy dynamically 30

31 Pyretic = * Our goal

Pyretic Provides 32 FeatureExample Concise policy encoding flood() Boolean predicates ~(match(dstip= ) | match(srcip= ) ) Composition operators (load_balance + monitor) >> route Virtual header fields match(switch) modify(class=‘staff’) Query Policies count_packets()

Pyretic: Key Abstractions 33 Abstract Packet Model –packet flows as a dictionary mapping field names to values –(nested) virtual header fields –located packet: set  multi-sets Network Object Model: virtual switches & ports –“big switch”: one-to-many –“virtual network”: many-to-one

Pyretic: Policy Language 34 Composition: parallel and sequential composition operators: “|” and “>>” High-level Policy Functions –Static Policies: primitive actions Predicates query –Dynamic Application def main(): P.when(…)

Policy in OpenFlow Defining “policy” is complicated –All rules in all switches –Packet-in handlers –Polling of counters Programming “policy” is error-prone –Duplication between rules and handlers –Frequent changes in policy (e.g., flowmods) –Policy changes affect packets in flight 35

From Rules to a Policy Function Located packet –A packet and its location (switch and port) Policy function –From located packet to set of located packets Examples –Original packet: identity –Drop the packet: drop –Modified header: modify(f=v) –New location: fwd(a) 36

Why a set? Consider flood To flood every packet: from pyretic.lib.corelib import * def main(): return flood() 37

No worrying about: Writing a separate handler for controller Constructing and sending OpenFlow rules Keeping track of each switch Whether all switches supports the OpenFlow “flood” action (portability!) 38

Programmer Wishlist Encode policy concisely Specify traffic of interest Write portable code Compose code from independent modules Query network state Modify policy dynamically 39

From Bit Patterns to Predicates OpenFlow: bit patterns –No direct way to specify dstip!= –Requires two prioritized bitmatches Higher priority: dstip= Lower priority: * Pyretic: boolean predicates –Combined with &, |, and ~ –E.g., ~match(dstip= ) 40

Example: Access Control Block host def access_control(): return ~(match(srcip=‘ ’) | match(dstip=‘ ’) ) 41

Programmer Wishlist Encode policy concisely Write portable code Specify traffic of interest Compose code from independent modules Query network state Modify policy dynamically 42

OpenFlow Packets Location specific code needs both –The packet –The OF packet_in message* Tagging packets requires choosing –Header field (e.g., VLAN, MPLS, TOS bits) –Set of values Neither concise nor portable 43 *

Pyretic Virtual Header Fields Unified abstraction –Real headers: dstip, srcport, … –Packet location: switch and port –User-defined: e.g., traffic_class Simple operations, concise syntax –Match: match(f=v) –Modify: modify(f=v) Example – match(switch=A) & match(dstip=‘ ’ ) 44

Runtime Handles Implementation Compiler chooses –What set of header bits to use –What each value means Conceptual – no tying to particular mech. Portable – different hw, other modules Efficient –Don’t need same vlan value network-wide –Sometimes tags will ‘factor out’ 45

Programmer Wishlist Encode policy concisely Write portable code Specify traffic of interest Compose code from independent modules Query network state Modify policy dynamically 46

Recall OpenFlow 47

srcip=0*,dstip=P -> mod(dstip=A) srcip=1*,dstip=P -> mod(dstip=B) dstip=A -> fwd(2) dstip=B -> fwd(3) * -> fwd(1) Balance then Route (in Sequence) 48 Combined Rules? (only one match) srcip=0*,dstip=P -> mod(dstip=A) srcip=1*,dstip=P -> mod(dstip=B) dstip=A -> fwd(2) dstip=B -> fwd(3) * -> fwd(1) Balances w/o Forwarding! srcip=0*,dstip=P -> mod(dstip=A) srcip=1*,dstip=P -> mod(dstip=B) dstip=A -> fwd(2) dstip=B -> fwd(3) * -> fwd(1) Forwards w/o Balancing!

dstip =  fwd(2) dstip =  fwd(3) *  fwd(1) srcip =  count dstip =  fwd(2) dstip =  fwd(3) *  fwd(1) Forwards but doesn’t count Route and Monitor (in Parallel) 49 IP = IP = Monitor Route dstip =  fwd(2) dstip =  fwd(3) *  fwd(1) srcip =  count Counts but doesn’t forward srcip =  count Combined rules installed on switch?

srcip =  count dstip =  fwd(2) dstip =  fwd(3) *  fwd(1) Requires a Cross Product [ICFP’11, POPL’12] 50 IP = IP = Monitor Route srcip = , dstip =  fwd(2), count srcip = , dstip =  fwd(3), count srcip =  fwd(1), count dstip =  fwd(2) dstip =  fwd(3) *  fwd(1)

Parallel ‘+’:Do both C1 and C2 simultaneously Sequential ‘>>’:First do C1 and then do C2 51 Policy Functions Enable Compositional Operators

Example: Sequential Composition To first apply access control and then flood access_control() >> flood() Two totally independent policies Combined w/o any change to either 52

E.g., the ‘if_’ policy def if_(F,P1,P2): return (F >> P1) + (~F >> P2) 53 Easily Define New Policies

Or flood xfwd(a) = ~match(inport=a) >> fwd(a) flood = parallel([match(switch=sw) >> parallel(map(xfwd,ports)) for sw,ports in MST]) Portable: compiler chooses whether to implement using OpenFlow ‘flood’ or ‘forward’ 54

Programmer Wishlist Encode policy concisely Write portable code Specify traffic of interest Compose code from independent modules Query network state Modify policy dynamically 55

Querying In OpenFlow Pre-install rules at the needed granularity (so byte/packet counters are available) Issue queries to poll these counters Parse the responses when they arrive Combine counter values from multiple rules Keep track of any packets sent to controller 56

Queries in Pyretic Just a special type of policy Forwarding to a “bucket” – q = packets(limit=1,group_by=['srcip']) Used like any other (>>, +) w/ Callback function registration – q.register_callback(printer) 57

Example: Monitor q = count_bytes(interval=1) q.register_callback(printer) monitor = match(srcip=' ') >> q 58

Query Policy Library 59 SyntaxSide Effect packets( limit=n, group_by=[f1,f2,…]) callback on every packet received for up to n packets identical on fields f1,f2,... count_packets( interval=t, group_by=[f1,f2,…]) count every packet received callback every t seconds providing count for each group count_bytes( interval=t, group_by=[f1,f2,…]) count bytes received callback every t seconds providing count for each group

Recall OpenFlow Toy Balance Route Monitor 60

In Pyretic balance = (match(srcip=0*,dstip=P) >> modify(dstip=A)) + (match(srcip=1*,dstip=P) >> modify(dstip=B)) + (~match( dstip=P) >> identity) route = (match(dstip=A) >> fwd(2)) + (match(dstip=B) >> fwd(3)) + (~(match(dstip=A) | match(dstip=B)) >> fwd(1)) b = count_bytes(interval=1) b.register_callback(print) monitor = match(srcip=X) >> b mlb = (balance >> route) + monitor 61 1* 0* B A 1 2 3

Compared to install_flowmod(5,srcip=X & dstip=P,[mod(dstip=A),fwd(2)]) install_flowmod(4,srcip=0* & dstip=P,[mod(dstip=A),fwd(2)]) install_flowmod(4,srcip=1* & dstip=P,[mod(dstip=B),fwd(3)]) install_flowmod(4,srcip=X & dstip=A,[ fwd(2)]) install_flowmod(4,srcip=X & dstip=B,[ fwd(3)]) install_flowmod(3, dstip=A,[ fwd(2)]) install_flowmod(3,dstip=B,[fwd(3)]) install_flowmod(2,srcip=X,[fwd(1)]) install_flowmod(1,*,[fwd(3)]) 62

Programmer Wishlist Encode policy concisely Write portable code Specify traffic of interest Compose code from independent modules Query network state Modify policy dynamically 63

Functions Support Dynamism Dynamic policies are also functions of time Value at current moment in self.policy Reassigning updates dynamic policy Can be driven by queries 64

Update current val Otherwise, policy unchanged Example: MAC-Learning class learn(DynamicPolicy): def init(self): q = packets(limit=1,[’srcmac’,’switch’]) q.register_callback(update) self.policy = flood() + q def update(self,pkt): self.policy = if_(match(dstmac=pkt[’srcmac’], switch=pkt[’switch’]), fwd(pkt[’inport’]), self.policy) 65 First packet with unique srcmac, switch Defined momentarily to flood If newly learned MAC New Type of Dynamic Policy and query Initialize current value of time series Forward directly to learned port

Dynamic Policies Can also be driven by external events! 66

Example: UI Firewall def __init__(self): print "initializing firewall" self.firewall = {}... self.ui = threading.Thread( target=self.ui_loop) self.ui.daemon = True self.ui.start() 67 def update_policy (self): self.policy = ~union([(match(srcmac=mac1) & match(dstmac=mac2)) | (match(dstmac=mac1) & match(srcmac=mac2)) for (mac1,mac2) in self.firewall.keys()]) def AddRule (self, mac1, mac2): if (mac2,mac1) in self.firewall: return self.firewall[(mac1,mac2)]=True self.update_policy()

Programmer Wishlist Encode policy concisely Write portable code Specify traffic of interest Compose code from independent modules Query network state Modify policy dynamically 68

Pyretic Platform 69

Switches OF Controller Platform OF Controller Client Pyretic Backend Pyretic Runtime Main App 1 App 2 OpenFlow Messages Serialized Messages (TCP socket) Controller Platform Architecture Process 1 Process 2

Modes of Operation Interpreted (on controller) –Fallback when rules haven’t been installed –Good for debugging Reactive –Rules installed in response to packets seen –Fallback when proactive isn’t feasible –Various flavors and optimizations Proactive –Analyze policy and push rules –Computation can be an issue –Currently nuclear and simple incremental 71

HA & Scale Out Interpreter, QoS, ACL, Topo Runtime Architecture (in Progress) Flow Table Cache Management Consistent Update Incrementalization Classifier Generation Query Mgmt

Also in Progress New features –Policy access controls – QoS operators –Enhanced querying library Applications –Incorporation of RADIUS & DHCP services –Wide-area traffic-management solutions for ISPs at SDN-enabled Internet Exchange Points. 73

Test It Out Yourself 74

HW Middleware QA Ops 75 WAN, Enterprise, DC, Home Wireless & Optical Systems Challenges –Scalability –Fault-tolerance –Consistency –Upgradability –Performance Security Virtualization –of network –of services –of address space Hard and soft-switches –Network processors –Reconfigurable pipelines –Hardware accelerated features –Programmable x86 dataplane APIs and Languages Traffic Monitoring/Sampling/QoS Testing, Debugging, Verification Experimentation & Testbeds Integration & Orchestration –End hosts –Legacy switches –Middleboxes –Network Services

Where Pyretic Fits Hard and soft-switches –Network processors –Reconfigurable pipelines –Hardware accelerated features –Programmable x86 dataplane APIs and Languages Traffic Monitoring/Sampling/QoS Testing, Debugging, Verification Experimentation & Testbeds Integration & Orchestration –End hosts –Legacy switches –Middleboxes –Network Services 76 WAN, Enterprise, DC, Home Wireless & Optical Systems Challenges –Scalability –Fault-tolerance –Consistency –Upgradability –Performance Security Virtualization –of network –of services –of address space