Sponsored by the National Science Foundation 1 ENTGENI and OFSTAN Spiral 2 Year-end Project Review Stanford University PI: Nick McKeown, Guru Parulkar Staff: Guido Appenzeller, Masayoshi Kobayashi, Johan van Reijendam, Srini Seetharaman, Rob Sherwood, Dan Talayco, Paul Weissmann, Tatsuya Yabe Students: David Erickson, Glen Gibb, Jad Naous, KK Yap, Yiannis Yiakoumis Aug

Sponsored by the National Science Foundation 2 Goal Build and deploy OpenFlow networking substrate for GENI Sliceable and programmable Support research and production use on same substrate Integrated with GENI control framework 25-Aug-2010

Sponsored by the National Science Foundation 3 Scope Revise and maintain OpenFlow spec & reference implementations Design, build, and maintain software components –FlowVisor, Expedient, SNAC, NOX and a set of tools Deploy OpenFlow in Stanford campus –McKeown group production network and open wireless (ofwifi) Support other campus deployments –Production and research use as part of GENI –Build larger OpenFlow community Integrate with GENI –With control framework and computing substrate Create compelling demonstrations –With other campuses 25-Aug-2010

Sponsored by the National Science Foundation 4 Accomplishments Released OpenFlow 1.0 and working on OpenFlow 1.1 –GEC9 to use OpenFlow 1.0 that is starting to be relatively stable –Supports slicing for isolation among experiments New release for all major software components: FV, NOX, Expedient –Support OF1.0 –Expedient as OF aggregate manager supports GENI API and Opt-in –Expedient as a “plug and play” clearinghouse – demonstrated at GEC8 Stanford deployment –Expanded to include more production and experiment users –Expanded to use OF switches from more vendors Campus deployments –All of them have OF deployments and getting ready for GEC9 –Used for both limited production and research use 25-Aug-2010

Sponsored by the National Science Foundation 5 Accomplishments (contd.) GENI Integration –A preliminary integration demo at GEC8: featured various components on computing and networking substrates at GPO and Stanford –Some progress on an end to end OpenFlow network: control plane and data plane interconnection being worked on Building a larger community –Talks, CIO summit, documentation, HOWTO guides, tutorials, mailing lists –Preliminary survey: OF deployments in 68 institutions spanning 13 countries Built strong collaboration between Stanford and GPO –With both Heidi’s team and Tom’s team –Opportunity to build on it further for GEC9 and long term GENI 25-Aug-2010

Sponsored by the National Science Foundation 6 OpenFlow Spec and Reference Implementations 25-Aug-2010

Sponsored by the National Science Foundation 7 OpenFlow Spec and Switch Reference Implementations OpenFlow ver 1.0 reference implementation released Dec New features: –Slicing: OpenFlow now supports multiple queues per output port. –Flow cookies: Flows have been extended to include an opaque flow identifier –User-specifiable datapath description: The OFPST_DESC (switch description) reply now includes a string specified by the switch owner to describe the switch. –Match on IP fields in ARP packets: Allows match on IP fields inside ARP packets. –Match on IP ToS/DSCP bits and VLAN PCP bits –Querying port stats for individual ports: Port stat request messages include a port_no field to allow stats for individual ports to be queried. –Improved flow duration resolution in stats/expiry messages: Flow durations in stats and expiry messages are now expressed with nanosecond resolution. OpenFlow 1.1 under discussion based on popular use cases. Likely release in Q New features: –Multipath ProposalMultipath Proposal –Rate Limiter ProposalRate Limiter Proposal –Tags and tunnelsTags and tunnels –Multiple tables for matchingMultiple tables for matching 25-Aug-2010

Sponsored by the National Science Foundation 8 Indigo Hardware Accelerated OpenFlow Release for Broadcom Chipsets Indigo now supported by Big Switch Networks HW Accelerated OpenFlow Provides an affordable, open, high-performance OpenFlow Switch for research and development Milestones Completed Indigo 1.0 Beta Released (6/28) 48 x GE + 4 x 10 GE at line rate 1.0 OpenFlow support Live in Stanford's OpenFlow deployment, used in DNRC test cluster Roadmap Beta release for Pronto 3290 with improved flow table size: 8/30 General 1.0 release expected soon after

Sponsored by the National Science Foundation 9 Other Reference Implementations Software –User-space datapath switch NetFPGA –Publicly available hardware FPGA bitfile for datapath –Has support for tunneling Mininet –Supports creating a complete experimental network on a laptop or other PC. Easily scalable virtual networking platform – 25-Aug-2010

Sponsored by the National Science Foundation 10 OpenFlow Software Components 25-Aug-2010

Sponsored by the National Science Foundation 11 FlowVisor Complete rewrite of FlowVisor code –Internally released as flowvisor-0.6-beta1 New features –Unified FlowSpace instead of per-slice –Removed error-prone Slice “ID”s –Rolls and permissions New API for remote management –Create slice, reserve flowspace, debugging information –Used by Opt-In Manager, Expedient –fvctl command line API client Deployment –Stanford's Demo network –Extensive use in GEC8 (recursively!) Ongoing activities Improve stability/isolation –Direct switch CPU monitoring –Rate limiting to/fromswitch –Improve timeouts on inactive connections Documentation –Man pages –Configuration file format –Technical description

Sponsored by the National Science Foundation 12 NOX 25-Aug-2010 NOX updated to be OpenFlow-1.0 compatible Changes in NOX (destiny branch) –Move from XML to JSON (embedded parser, remove Xerces) –Inclusion of OpenFlow message packing library –Support for SNMP trap –Added doxygen documentation for both C/C++ and Python –Rewrite LAVI with new JSON interface (also used for NOX's console) Ongoing activities Component distribution model Ripcord support in NOX Improved Python integration Joint work with ICSI/Berkeley

Sponsored by the National Science Foundation 13 SNAC OpenFlow Controller with a Graphical UI SNAC now supported by Big Switch Networks Contract expected in next two weeks Development work in progress Milestones Completed Released complete Source Code of SNAC under the GNU Public License Merged Stanford SNAC modifications for 1.0 onto NOX destiny Roadmap SNAC 1.0 release with OpenFlow 1.0 Support (In Progress) Ready made packages for multiple operating systems Additional Features

Sponsored by the National Science Foundation 14 SNAC Launch of Snacsource.org

Sponsored by the National Science Foundation 15 Expedient as Aggregate Manager Developed a Django-based modular web platform for managing users, and creating slices through pluggable user interfaces –Called Expedient –Fine-grained permissions and per-project roles for access control –OpenFlow plugin talks to Opt-in Manager that enforces local aggregate policies Opt-In Manager complements Expedient in dynamic slice creation within an aggregate –Manages local user opt-in to slices –Enforces local policies –Communicates with the Flowvisor to create slices and add flowspace 25-Aug-2010 ToDos for GEC9: Allow users to create slices through Expedient's GENI- API interface Deploy Expedient as AM and Opt-In Manager at each campus. Allow remote opt-in so researchers do not need to separately go to opt-in manager to add flowspace.

Sponsored by the National Science Foundation 16 Expedient as Clearing House Expedient could also be used to manage multiple aggregates using the pluggable interface. Expedient can coordinate with AMs of different aggregates and serve as a Clearinghouse –Deploy central Expedient to access Web interface and reserve slices on all campuses for both OpenFlow and PlanetLab. 25-Aug-2010 Future Plans: Add Emulab and EC2 as aggregate plugins Open Expedient for community development Associate all campus Aggregate managers to a centralized Expedient

Sponsored by the National Science Foundation 17 Other Tools ENVI/LAVI: –Network and flow visualization Debugging/monitoring tools –Data plane: ofpeck –Control plane: oftrace

Sponsored by the National Science Foundation 18 Software Releases 25-Aug-2010 OpenFlow ref. implementation git://openflowswitch.org/openflow.git NOX controller w/ LAVI support git://noxrepo.org/nox (Branch destiny)0.8 ENVI git://github.com/dound/envi.git - SNAC FlowVisor git://openflowswitch.org/flowvisor.git0.6 Expedient, Opt-in mgr git://openflow.org/expedient (VM at wk/index.php/VM_instructions) TBD HP, NEC, Toroki, and Pronto firmware Vendor distribution mode HP: 2_02h NEC: 278 Indigo_beta

Sponsored by the National Science Foundation 19 Software Release Plan Different snapshots in the revision of each software –Tags made if there are substantial changes made. Helps associate bugs software revisions SNAC NOX FlowVisor Vendor firmware Aggregate mgr/ Opt-in mgr Stable Sept2010 Bleeding edge Stable Aug2010

Sponsored by the National Science Foundation 20 OpenFlow Deployments 25-Aug-2010

Sponsored by the National Science Foundation 21 Stanford Deployment 70 weeks of OpenFlow in production setting Four networks using 15 switches (HP, NEC, Pronto) and 35 wifi APs –Production wired VLAN74 –Experimental wireless VLAN374 –Demo network VLAN 374, interconnected with NLR –Isolated test network VLAN 1001 Wired network –18 users across 7 rooms in Gates 3A wing Wireless network –77 unique users opted-in to open wireless net: ofwifi

Sponsored by the National Science Foundation 22 Campus Deployments Status Georgia Tech: Existing deployment in KACB and CCB buildings. Expanding deployment to TSRB and dorms Clemson: Wireless mesh deployment and production deployment including lab users. Plans to expand to 3 other buildings and outdoor parking lot. Princeton: Production deployment in Computer Science building. Adding production users from Jen Rexford and Mike Freedman’s group. Wisconsin: Existing wireless deployment. Adding more production wired users from research group. Washington: Existing wireless users and servers in OpenFlow network. Expanding to opt-in users from Tom Anderson’s group Rutgers: Deployment spanning WinLab and LCSR, along with a test deployment in OrbitLab. Interconnected to Internet2 through Magpie. Indiana: Production users from Indiana University on HP Procurve switches. Expanding to include more campuses in University system 25-Aug-2010 Highlights: All campuses have OpenFlow networks deployed and used for both production and research Demonstrations at GEC7 and GEC8 All on-board for GEC9 Good progress and great attitude – pleasure to work with

Sponsored by the National Science Foundation 23 Support Overview SNAC: Big Switch Networks NOX: Stanford and Berkeley FlowVisor: Stanford General OpenFlow Support: Stanford Switches Expedient/Opt-In: Stanford Indigo: Big Switch Networks HP: HP Support NEC: NEC Support... Local Campus IT Level 1Level 2Level 3 Support has been through bi-weekly calls, tutorials, online guides, mailing lists, bug tracking, and onsite visits

Sponsored by the National Science Foundation 24 Connectivity Update 25-Aug-2010 GPO and Stanford collaborating in providing end-to-end Layer-2 connectivity to experimenters Inter-connectivity of campuses proceeding on target –NLR connectivity status 3700 Univ Wisconsin Madison (complete) 3702 Indiana Univ (in progress) 3704 BBN (complete) 3705 Shared monitoring and testing (complete) 3707 Clemson (complete) 3709 Georgia Tech (complete) 3715 NLR-OpenFlow-Core (South) (in progress) 3716 NLR-OpenFlow-Core (North) (in progress) –Washington, Rutgers and Princeton connecting through NetFPGA OpenFlow switches in Internet2

Sponsored by the National Science Foundation 25 OpenFlow deployments worldwide –68 deployments over 13 countries Several support mailing lists with active participation from open- source community –nox-dev, openflow-discuss, snac-users, egeni-trials OpenFlow talks –JointTech/Internet2, Quilt OpenFlow-based regional proposals Community Building 25-Aug-2010

Sponsored by the National Science Foundation 26 GENI Integration Update Demonstrated successful control-plane integration of OpenFlow substrate with computing substrates using common GENI API –The OpenFlow aggregate manager was upgraded to support the GENI API. We dynamically created slices using it 25-Aug-2010

Sponsored by the National Science Foundation 27 Demonstrations 25-Aug-2010

Sponsored by the National Science Foundation 28 GEC7 Demo Highlights 6 campuses demonstrated their OpenFlow test network Clemson Laptop handoff experiments across 2 APs and 1 Toroki switch Clemson Laptop handoff experiments across 2 APs and 1 Toroki switch Washington Distributed web- hosting across Stanford and UWash Wisconsin OpenSAFE network monitoring Indiana 2 PlanetLab nodes connected to 3 HP OpenFlow switches controlled by SNAC to restrict traffic Rutgers OpenFlow usage in the ORBIT Lab sandbox Georgia Tech NOX-based captive portal for admission control over 5 OpenFlow switches spread across campus

Sponsored by the National Science Foundation 29 GEC8 Demo Highlights 25-Aug-2010 Stanford+BBN: Expedient / Integration –A pluggable non-intrusive exploratory control framework for GENI called Expedient –Creating an end-to-end slice through Expedient and GENI API –Slice spanned PlanetLab and OpenFlow resources at Stanford and BBN Clemson+GaTech: Access control –Management system including network access control for wired and wireless users Deployment at Indiana Univ –Tools used to monitor and manage OpenFlow Networks deployed across Indiana Univ and backbone networks Stanford: Packet and Circuit Convergence –Demonstrated network convergence, where OpenFlow control both packet and circuit switching technologies in a common way Stanford + Washington + Indiana + Princeton + BBN: Aster*x load-balancer –Load-balancing that takes into account both the server and network load –Network and computation slice spanning multiple campuses across the country Wisconsin: Mobile Application Migration in Enterprise Metworks –Demonstrated the application migration component of an in-progress offloading system prototype Wisconsin: Packet Level Redundancy Elimination (RE) on Network Elements –Reduce packet redundancy and improve network efficiency, without expensive upgrading of network links to higher bandwidths Rutgers/BBN/Stanford: WiMax –Demo of an application including clients at multiple WiMAX sites, plus a WiFI connected client at demo site.

Sponsored by the National Science Foundation 30 Milestone & QSR Status: ENTGENI IDMilestoneStatusOn Time? On Wiki? GPO signoff? S2.aRelease OpenFlow 1.0.Delivered on Dec 31, 2009Yes S2.bSupport Campus Installation of OpenFlow 1.0Delivered. GaTech/Clemson transitioned in Aug 2010, all others transitioned by GEC8 Yes S2.cUpgrade and maintain FlowVisor, Aggregate Manager and deployment tools. Delivered. New FlowVisor is running on all campuses. Expedient is publicly available and undergoing deployment Yes S2.dIntegrate OpenFlow, PlanetLab Control Framework (SFI), GENI API, and NOX on Stanford campus (primary responsibility is with Princeton and GPO. Delivered. This was demonstrated in GEC8. Yes S2.eIntegration testing with PlanetLab, I2, NLR, and GPO for end-to-end OpenFlow dataplane and Openflow/AM/SFI control plane. Delivered. This was demonstrated in GEC8 Yes S2.fCoordinate campus support interface with Nicira. Actively supporting or coordinating these activities (These are always ongoing and do not have a finite end) Yes S2.gCoordinate development support for improved NOX packaging, documentation and usability with developers at ICSI and Nicira. S2.hSupport I2 and NLR OpenFlow system design, deployment and debugging. 25-Aug-2010

Sponsored by the National Science Foundation 31 Milestone & QSR Status: OFSTAN IDMilestoneStatusOn Time? On Wiki? GPO signoff? S2.aContinue to build E-GENI at StanfordOur deployment transitioned to 1.0 and has expanded (in the past 1 year) to 25 wired users and 25 wireless users. Yes S2.bSupport use of E-GENI at Stanford by other GENI researchers -- very limited number of pre-approved users -- most likely users participating in the demo. Stanford E-GENI networks (experimental and demo network) were opened up for several researchers to conduct demos Yes S2.cProvide training and documentation for E-GENI Extensive documentation to deploy OpenFlow (foswiki), develop OpenFlow experiments (NOX doxygen, and OpenFlow tutorial in GEC8) Yes 25-Aug-2010

Sponsored by the National Science Foundation 32 Milestone & QSR Status: OFNOX IDMilestoneStatusOn Time? On Wiki? GPO signoff? S2.aPublish NOX feature listDelivered.No S2.bRelease NOX supporting OpenFlow 1.0Delivered. Openflow-1.0 branchYes S2.cInitial rough NOX API documentationDelivered. Using doxygenYes S2.dRelease software that supports OpenFlow 1.0 configuration protocol, and NOX API V2 Delivered. Openflow-1.0 branchYes S2.eSupport campus installations of GENI software with AM API Ongoing (no finite end)Yes S2.fPort existing applications to latest OpenFlow identifiers Delivered. Released transition notes and work with individual teams to port their apps Yes S2.gAdd support for multiple Openflow protocolsNot planned for- S2.hIncrease portabilityDelivered. Published guide for porting to CentOS and reduced dependency Yes S2.iRelease 1 st ver SNAC/NOX console softwareDelivered. Also in NOX-destiny.Yes S2.jPlan and engineer GEC 9 demoOngoingYes S2.kSupport campuse upgrades to OF 1.0 and GENI API Delivered. All campuses transitionedYes S2.lImprove system and regression test supportOngoingYes S2.mPublish feature list for 2011 releasesOngoingYes 25-Aug-2010

Sponsored by the National Science Foundation 33 Program Changes Nicira deciding to drop off –Implications on NOX and SNAC –Realization that NOX belongs in the university Unavoidable slow down. End result likely to be better. Toroki not working out as a provider of inexpensive OF switches BigSwitch stepping up –To continue to develop and support SNAC –To continue to develop and support Indigo reference implementation More central to its business and so we can expect good results ICSI/Berkeley and Stanford are stepping up –To continue to develop and support NOX –To Build an open source community around it Good for the research community Expedient –A significant new activity –Decided not to do just a simple aggregate manager 25-Aug-2010

Sponsored by the National Science Foundation 34 Issues GENI Integration: Not as mature as we would like –GPO and Stanford collaboration for GEC8 provided good start –Proposed plan between GPO and Stanford encouraging Software release and support –Need more fine tuning and consensus building Internet2 moving slowly –Can be an issue Inter-campus connectivity harder than we thought –Requiring coordination by many parties and lot of logistics –Have to keep pushing and get it all working soon 25-Aug-2010

Sponsored by the National Science Foundation 35 Plans: Main Focus on GEC9 25-Aug-2010

Sponsored by the National Science Foundation 36 GENI Integration Plans Expedient and Opt-in Manager testing and deployment –By Aug 30, perform necessary internal testing of OpenFlow control stack and tag working version –Track bugs and generate regression tests for bugs discovered Proposed plan between BBN and Stanford –BBN and Stanford will in parallel do following tests: install FV/Expedient/OptIn Manager use this stack to reserve an Openflow slice ensure candidate demos run –By Sept 15, generate RPMs/DEBs that have been tested and produce necessary documentation for campuses to rollout 25-Aug-2010

Sponsored by the National Science Foundation 37 GEC9 Demo Support Main activities –Allow each campus to operate their own Aggregate manager and Opt-in manager –Verify control plane integration –Verify data plane integration Connect demo networks in each campus to NLR/Internet2 –Perform dry-runs and testing for GEC9 –Polish demos Tech support for other campuses –Help transition software to latest supported version 25-Aug-2010

Sponsored by the National Science Foundation 38 Post-GEC9 Plans Scale up the production network to include more users The GPO is starting to formulate goals for Spiral 3. What are your thoughts regarding potential Spiral 3 work? –Streamline campus support –Scale OF deployments on campuses for research and production use –Get more campuses ready for OF deployments –More involvement of the core networks and regional networks –Increased scale of experiments –Ensure safe resource allocation across experiments –Provide OF network and GENI racks integration 25-Aug-2010

Sponsored by the National Science Foundation 39 Final Takeaways We are on track to build and deploy OpenFlow networking substrate for GENI –Different components and deployments coming together –Very good collaboration with campuses and GPO We are excited about and focused on GEC9 Two issues –OpenFlow and GENI control framework integration –Inter-campus connectivity Both seem to be under control with a plan – we need to execute Exciting long term prospects –For scaling OpenFlow networks on campuses –For OpenFlow and software defined networking in general 25-Aug-2010