1. Hybrid MLN DOE Office of Science DRAGON Multi-Layer, Multi-Domain Control Plane Hybrid Networks Architecture Current Status and Future Issues Andy Lake, John Vollbrecht Internet2 Nasir Ghani University of New Mexico Nagi Rao Oak Ridge National Laboratory ESCC/Internet2 Joint Techs Winter Meeting January 22, 2008 University of Hawaii Honolulu, Hawaii Tom Lehman, Xi Yang Information Sciences Institute East University of Southern California Chin Guok Network Engineering Services Group ESnet

2. Multi-Domain, Multi-Layer Hybrid Networks Hybrid networks are intended to provide a flexible mix of IP routed service and dedicated capacity “circuits” The “Multi-Layer” is meant to identify several items regarding how hybrid networks may be built. In this context it includes the following: Multi-Technology - MPLS, Ethernet, Ethernet PBB-TE, SONET, NG-SONET, T-MPLS, WDM Multi-Level - domains or network regions may operate in different routing areas/regions, and maybe be presented in an abstracted manner across area/region boundaries Multi-Domain indicates that we want to allow hybrid network service instantiation across multiple domains And of course all this implies that this will be a Multi- Vendor environment.

3. Multi-Layer, Multi-Domain Control Plane Hybrid Networks Control Plane Where are today? What is interesting to think about for the future?

4. Hybrid Networks Control Plane - Today Currently an early deployment of a Hybrid Network Control plane on ESnet SDN and Internet2 DCN and some regional's such as DRAGON, NYSERNet, others This evolved out of collaborations between several projects including ESNet OSCARS, I2 BRUW, NSF DRAGON, and the DICE Group It is expected that this will evolve as standards bodies and other groups work on developing interface specifications/agreements with the larger community Key features of the current architecture are noted in the following bullets. One InterDomain Controller (IDC) per domain which is responsible for inter-domain messaging A “Domain Controller” (DC) which takes care of provisioning inside the domain. The DC is really an internal domain concern The DC design will vary by domain (based on technology types, vendor capabilities, etc), and in some instances may be a very minimal set of functions The IDC/DC combination provides the basis for a two-level hierarchical network view. Where the DC will have knowledge of the real local topology and the IDC may have a reduced or abstracted view.

5. Hybrid Networks Control Plane - Today Four distinct phases are identified for IDC communications: User Request, Topology Exchange, Resource Scheduling, Signaling Topology Exchange: currently based on abstracted link states, with little to no dynamic information. We are also planning to investigate use of a path vector style of inter- domain information exchange. Resource Scheduling: multi-domain, multi-stage path computation process where the specific resources get identified and reserved for a specific signaling event. The Signaling Phase is where specific network elements are provisioned. This phase may be initiated by the user, or by the domains. The Signaling Phase actions are based on resources identified in the Resource Scheduling phase. User Request Phase provides a message set for users to request multi-domain circuits Current security and authentication models are based on signed soap messages and X509 Certificates (User to local IDC messaging; IDC to IDC messaging)

6. Hybrid Networks Control Plane - Today Four Primary Web Services Areas: Topology Exchange, Resource Scheduling, Signaling, User Request

7. Hybrid Networks Control Plane - Today Meta-Scheduler Approach Same set of Web Services used for linear instantiation model can be used by a high level process to build services: Topology Exchange, Resource Scheduling, Signaling, User Request A key issue is that this requires a trust relationship between the “meta- scheduler” and all the domains with which it needs to talk MetaScheduler Topology Signaling Scheduling Topology Scheduling Signaling

8. Hybrid Network Control Plane What can we do today? Dynamic provision of end to end (circuits) across multiple domains. Specify a few basic parameters regarding a single circuit request: edge technology/configuration, bandwidth, end points, domain sequence, specific start/stop times

9. Hybrid Network Control Plane What is interesting to think about for the future? Enhanced Circuit Parameters and User Request Mechanisms Richer set of flexible circuit request constraints such as technology type, flexible time period specifications, latency, jitter, adjustments to in-service circuits, arbitrary business /administrative/security constraints, flexible user requests mechanisms. Topology Building: combining multiple individual circuits together into a user specified topology. Network Virtualization - True Multi-Level, Multi- Technology, Multi-Vendor network control and provisioning Only talking about network resources here; correlation with application domain resources is considered a separate activity.

10. Enhanced User Options User has increased number of parameters to specify such as technology type, adjustments to in-service circuits, flexible time period specifications, latency, jitter, arbitrary business /administrative/security constraints, flexible user requests mechanisms Green Path might be chosen in response to user specifying domain preference, latency/jitter requirement, or something else

11. Multi-Level, Multi-Technology, Multi- Vendor Infrastructures Multiple Options, most will have vendor proprietary control and management mechanisms which only work across single vendor regions Routers Switched WDM Optical Layer Ethernet PBB-TE Ethernet Layer Switched WDM Optical Layer Ethernet Layer Switched SONET Layer (vcat, lcas)

12. Multi-Level, Multi-Technology, Multi- Vendor Infrastructures Current dynamic provisioning environment can be described as:  Static Topology, Dynamic Provisioning Next we want to enable:  Dynamic Topology, Dynamic Provisioning

13. Multi-Level, Multi-Technology, Multi- Vendor – Network Virtualization Network Virtualization and Topology Building in Multi-Level, Multi-Technology, Multi-Vendor Infrastructures Bandwidth Request was large enough to justify provisioning at WDM layer Bandwidth Request was smaller, so provision Ethernet, then router connection Provisioned Topologies Routers Switched WDM Optical Layer Ethernet PBB-TE Same Result with Either Approach End Points might attach at different levels: How do flexibly provision at what ever level an end point might appear?

14. What are the major issues looking forward? A key requirement for the architecture is to be able to handle the reality that the underlying networks will be very heterogeneous in terms of technology, control mechanisms, and vendors. In the current architecture this is abstracted out by the DC to IDC interface. Four types of underlying domain types have been identified in terms of how the DC interacts with them: GMPLS (I2 DCN is an example, regional networks based on ethernet switch dynamic provisioning is another example) MPLS (ESNet SDN is an example) Management Plane Controlled (USN is an example) Vendor Control Plane (I2 DCN also has a component of this)

15. Dealing with Heterogeneous Network Technologies and Vendor Equipment Adding regions of new technologies and vendors is not too difficult from the provisioning perspective The difficult issue is in terms of the routing exchange between/from the technology/vendor regions and path computation (intra and multi- domain) with multiple constraints. GMPLS MPLS Management Plane IDC DC IDC DC IDC DC DRAGON DRAGON GMPLS Control Plane Ciena Region uni, tl1 CD_a CD_z uni, tl1 subnet signaling flow IDC As an Example, DRAGON is used as the DOMAIN Controller for I2 DCN Ciena Core Directors GMPLS to other domains to other domain IDCs

16. Multi-Constraint Path Computation IntraDomain provisioning requires a path computation process to determine a path across the local network If the domain consists of multiple technologies, multiple levels, and multiple vendors this problem can be complex In order to realize the advanced control plane features multi-domain path computation needs to be augmented to operate in these environments. This will likely include addition of the following constraints to the path computation process: time domain flexible set of AAA and other user defined constraints Ability to look for paths as a group in the context of a entire topology build. These scheduling and flexible policy processing mechanisms will need to be tightly integrated/coupled with path computation and selection processes

17. Flexible and Policy Based Multiple Constraint Path Computation with Filtering/Pruning Processes Data source (raw link states from intra- and inter-domain flooding) and 3D constraints Snapshot of topology reduced by policy filters Constraint based path computation algorithm - CSPF heuristics

18. Path Computation with Multiple Dimensions Resource dimension Link availability, bandwidth capability & resource interdependence TE constraints, e.g. switching cap. AAA policy dimension User privileges App. specific requirements (SLA) Administration policies Time schedule dimension Integrate and translate network resource states and policies into shared control plane intelligence. Synergize AAA policy decision with TE based provisioning decision, resulting in fast, precise and simplified control process.

19. Control Plane Scalability and Performance – Simulations and Testing In order to collect some more data on the (current and future) control plane performance, we are planning to run some simulations on OPNET and/or User Mode Linux (UML) environments. This will allow us to evaluate the scalability/stability of inter-domain information exchange, the success/blocking probability/performance of Resource Scheduling and Signaling. User Mode Linux (UML) based simulation will also allow us to connect simulated networks to real networks, since the UML will run the same software as current networks. Current and future designs will be evaluated

20. Questions/Comments? Tom Lehman tlehman at east.isi.edu Thank You