1. Slide 1 Simple, Flexible Programming of Data Movement Paths using Algorithmic Policies PIs: Y. Richard Yang, Robert Bjornson, Andrew Sherman Architect: Andreas Voellmy

2. Slide 2 Context: Types of Devices in a Network Cyberinfrastructure Source: [Sherry, et. al. SIGCOM’12] Small: 1k hosts; Medium: 1-10k; Large: 10-100k; Very large: >= 100k.

3. Slide 3 Goals of Select Data Movement Paths in Complex Networks Satisfy CI app need Maximize resource util. Satisfy CI policies Adapt to state changes

4. Slide 4 Our Approach A programming framework to allow highly flexible control of data movement paths. Key goal: general, but extremely simple programming.

5. Slide 5 Project Context: SDN

6. Slide 6 Focus of Current SDN Systems: Data Store Network View NE Datapath Service/ Policy NE Datapath logically centralized data store

7. Slide 7 Example: Architecture of OpenDaylight https://wiki.opendaylight.org/view/OpenDaylight_Controller:Architectural_Framework

8. Slide 8 What is Missing? Network View NE Datapath Service/ Policy NE Datapath logically centralized data store Program

9. Slide 9 Why is “Bridging” Hard? Low-level, limited, flow table computation models Evolving, increasingly complex forwarding models (e.g., OF1.0, OF1.3, P4)

10. Slide 10 Example: A Simple ScienceNet Controller badSrc = 100 // policy topo = {…} // netview def onPacketIn(p): // policy+netvw->DP if badSrc == p.ip_src: return drop else: return mySNAlg(topo, p.ip_dst) def mySNAlg(topo, dst-ip): if dtn.ip == dst-ip: return alg1(topo, dst-ip) else: return alg2(topo, dst-ip) D (dtn) A B

11. Slide 11 Complexity: Controller Program => Flow Table badSrc = 100 // policy topo = {…} // netview def onPacketIn(p): // policy+netvw->DataPath if badSrc == p.ip_src: return drop installRule({‘match':{’ip_src’:badSrc}, ‘action’:[]}) else: return myAlg(topo, p.ip_dst) installRule({‘match’: {‘ip_dst’:p.ip_dst, ’ip_src’!=badSrc}, ‘action’:[myAlg(topo, p.ip_dst)]}) DataPath action does not support logical negation!

12. Slide 12 Complexity: Controller Program => Flow Table badSrc = 100 // policy topo = {…} // netview def onPacketIn(p): // policy+netvw->DataPath if badSrc == p.ip_src: return drop installRule({‘priority’:1, ‘match':{’ip_src’:badSrc}, ‘action’:[]}) else: return myAlg(topo, p.ip_dst) installRule({‘priority’:1, ‘match':{’ip_src’:badSrc}, ‘action’:?}) installRule({‘priority’:0, ‘match’: {‘ip_dst’:p.ip_dst, ‘action’:[myAlg(topo, p.ip_dst)]})

13. Slide 13 Complexity: Controller Program => Flow Table badSrc = 100 // policy topo = {…} // netview def onPacketIn(p): // policy+netvw->DataPath if badSrc == p.ip_src: return drop installRule({‘priority’:1, ‘match':{’ip_src’:badSrc}, ‘action’:[]}) else: return myAlg(topo, p.ip_dst) installRule({‘priority’:1, ‘match':{’ip_src’:badSrc}, ‘action’:[]}) installRule({‘priority’:0, ‘match’: {‘ip_dst’:p.ip_dst, ‘action’:[myAlg(topo, p.ip_dst)]})

14. Slide 14 End Result: Flow Table Table has many wildcard entries => inefficient. D A B

15. Slide 15 More Efficient Flow Tables using OF 1.3 Table 1 Table 2 D A B

16. Slide 16 Problem: Handling Platform Diversity badSrc = 100 // policy topo = {…} // netview def onPacketIn(p): // policy+netvw->DP if badSrc == p.ip_src: return drop … else: return myAlg(topo, p.ip_dst) if (switch_supports pipeline) installRule for two tables else: installRule for one table D A B

17. Slide 17 Our Project: Datapath Oblivious Programming Switches Programmer level Under the hood Network OS (NOS) Step 1. Make Decisions Step 2. Generate DataPath Make Decisions Switches Network OS (NOS) Step 2. Generate DataPath current our project

18. Slide 18 Our Programming Abstraction Control application expressed in a general purpose language that describes how a packet should be moved across a network, not how flow tables are configured –Written in a familiar language such as Java, Python, or Haskell. Conceptually invoked on every packet entering the network; may also access network environment state

19. Slide 19 Example Program in Java Route onPacketIn(Packet p) { if (p.tcpDstIs(22)) return null(); else { Switch srcSw = macTable.get(p.ethSrc()); Switch dstSw = macTable.get(p.ethDst()); return myRoutingAlg(srcSw, dstSw); } Does not specify anything on data path!

20. Slide 20 Key Challenge Route onPacketIn(Packet p) { 1. Switch srcSw = macTable.get(p.ethSrc()); 2. Switch dstSw = macTable.get(p.ethDst()); 3. return myRoutingAlg(srcSw, dstSw); } Challenge: How to convert a general program into device data (flow tables)?

21. Slide 21 Key Insight: Flow Tables are Memorization Tables Route onPacketIn(Packet p) { 1. Switch srcSw = macTable.get(p.ethSrc()); 2. Switch dstSw = macTable.get(p.ethDst()); 3. return myRoutingAlg(srcSw, dstSw); } Line 1 ethSrc srcSw Line 2 ethDst dstSw Line 3 route

22. Slide 22 Key Insight: Flow Tables are Memorization Tables Route onPacketIn(Packet p) { 1. Switch srcSw = macTable.get(p.ethSrc()); 2. Switch dstSw = macTable.get(p.ethDst()); 3. return myRoutingAlg(srcSw, dstSw); } Line 1 ethSrc srcSw Line 2 ethDst dstSw Line 3 route Table 1 Table 2 Table 3

23. Slide 23 Main Components and Workflow IR Compile Analyze & Design Code Gen static compile time Explorer Graph Explorer dynamic run time Packets Notifications Rules Table Graph onPacket DB Openflow switch P4 switch P4 Pipeline Northbound FlowRule Compile

24. Slide 24 Status Two implementations 24 Maple A reactive (dynamic-tracing) blackbox approach that demonstrated the basic feasibility of automating programming Code will become openly available in OpenDaylight as part of CCIIE Test deployment at Yale ScienceNet Magellan (aka Maple 2.0) A general view that a centralized algorithmic policy (AP) is a stream processing function A whitebox approach where distributed network flow tables are memorization tables implementing algorithmic policies In active design and implementation phase

25. Slide 25 Thank you! Welcome collaboration. Please contact me: yry@cs.yale.edu