Rationalizing ONAP Architecture for R2 and Beyond

ONAP R2+ Architecture – Current Draft View OSS BSS ONAP Portal – Design Studio & Dashboard Design-time (SDC) Run-time ONAP Operations Management (OOM) External Data Movement & APIs Resource Onboarding Orchestration (SO) Inventory/Topology (AAI) Data & Analytics (DCAE) Policy VNF/PNF SDK Service Level Orchestration Active / Available DCAE Analytic µServices Service & Product Design Policy Execution Engine Entitlements CDAP Holmes … Policy Creation & Validation Resource Level Orchestration Resource / Service Topology Data Distribution ESR Data Collection Layer Analytic Application Design Common Services DMaaP AAF Logging Micro Services Bus Catalog Multi-VIM Interface Open Stack AWS Azure MEC Infrastructure Adaption Layer Adaption Layer Service Logic Interpreter Config Database Yang Netconf CLI Life Cycle Mgmt Func L0-3 Network Controller (SDN-C) 3rd Party Network controller L4-7 Controller (App-C) Testing & Certification Vendor VNFM Adaption Layer (VF-C) Products Resources Eng. Rules Services Policies Analytics Config Database Service Logic Interpreter Life Cycle Mgmt funcs Life Cycle Mgmt funcs Standard Adaption Layer Vendor OA&M Adaptor Vendor A Vendor B Chef Netconf Ansible Recipe / Engineering Rules & Policy Distribution sVNFM sVNFM sVNFM Further Rationalization Opportunities … VNF1 VNF2 VNF3 PNF 1 VNFn Infrastructure OpenStack VMware RackSpace Azure Peripherals

Open Issues: What is the right Orchestration Implementation in ONAP & How to provide deployment flexibility? Rationalization of Controller Family within ONAP

Role Of Orchestrator in ONAP: ONAP Orchestrator must orchestrate Network Functions (VNF / PNF), Services and coordinate cross-controller activities driven by orders and events that indicate the need to instantiate, modify or remove Network Functions (VNF / PNF), services as well as multi- control layer remedy actions. Key attributes of Orchestrator are Model Driven Orchestration Process behavior driven by resource / service model designed in SDC Orchestrates network functions and service Instantiations Support software upgrade and planned change management Support open & closed-loop control actions initiated by DCAE /Policy or external API (from OSS for Open Loop) Tracks orchestrated activity for the life of the request but doesn’t control state of orchestrated components Interfaces to External OSS / BSS Systems Standard APIs exposed northbound to create, modify or remove services Request decomposition of service request into service / network function components Selects model / recipe that supports the request and maps order data to recipe / model Coordinating Activities Across Various ONAP Controllers Coordinates activities across Multi-VIM (infrastructure)/SDN-C//app-C/VF-C controllers Manage recording of service configurations Orchestrator Performs Following Additional Services Coordinates current inventories and placement/optimization recommendations Enforces business & technical policies Support standards-based workflows Validates and records network functions / service implementations

Examples of Orchestration Requests Install a single network function (VNF or PNF) – e.g. Deploy a new virtual PE / P router A virtual network element might have one or more components (e.g VNFCs or VDUs) Install multiple copies of network functions in one or more data centers – e.g. Deploy new virtual PE / P routers across n locations Deploy one or more instances of network services – e.g. RAN at Multiple sites, one or more sites of EPC, etc. Instantiate a customer services – e.g. multi-site VPN or IP SEC Could require deploying new instances of network elements (e.g. vCE) and using existing elements (e.g. vPE) Could be using existing elements without any new VNF / PNF being deployed (e.g. “Allocated Resources” – configuring a service to existing components) Create a new instance of a network service – e.g. IoT or eMBB slice Could be leveraging existing network elements (RAN, vEPC, Transport etc.) without any new VNF / PNF being deployed Implement a software upgrade / change management Could require Coordination of activities across Multi-VIM (infrastructure)/SDN-C/app-C/VF-C controllers Orchestrator requested to remedial action as part of Closed or open loop action Could require adding new compute / network resource to the existing functional elements (i.e. VNF) Create a new instance of VNF and migrate traffic Could require Coordination of activities across Multi-VIM (infrastructure)/SDN-C/app-C/VF-C controllers)

Model-Driven Orchestration “Generic” model-driven Service flow (limited existing) Each Resource Type has its own “Generic” model-driven flow. There currently exist such flows for “VNF” and “Network” Resource Types. Service Orchestration Resource Orchestration Cloud Resource Orchestration PNF Service X: topology_template: node_templates: PNF Network VNF Allotted Resource Network VF Module Note recursion VNF Service Orchestration Resource Orchestration Allotted Resource requirement: A PNF An Allotted Resource can be homed to an existing “underlying” network function or Service Instance, or homing could determine that a new network function Service Instance is needed. This would result in a 2nd level of Service Orchestration. Service Y: topology_template: node_templates: PNF Network VNF Allotted Resource capability: A Network Service Y is being treated as a “Resource” from the perspective of Service X. VNF Allotted Resource

N-Level Run Time Nesting? Let The Service Providers Decide Each Service/Resource “reuse unit” results in a separate thread of orchestration. This would allow for “on demand” spin up of “lower level” (Infrastructure) Service instances. Could be extended to allow multiple “higher level Services” to each have a “share” of a “lower level Service’s” instance See next slide for an alternative modeling approach that could be used when a “higher level Service” consumes the entirety of a “lower level Service’s” instance. This alternate approach may be appropriate for a Service Provider who is concerned about the run-time complexity that can be introduced with N-Level run time recursion.

N-Level Run Time Nesting? Let The Service Providers Decide For the case whereby a “higher level Service” consumes the entirety of a “lower level Service’s” instance, SDC should support the Design Time ability to construct an “upper” Service Definition from other Services definitions via substitution mapping (a.k.a., “Compile Time Nesting”) Service W: topology_template: node_templates: VNF_W Service_X Service X: topology_template: node_templates: VNF_X Service_Y Service Y: topology_template: node_templates: VNF_Y Service_Z Service Z: topology_template: node_templates: VNF_Z Substitution Mapping Design Time Distribution Time & Run Time VNF_W Service_W: topology_template: node_template: VNF_W VNF_X VNF_Y VNF_Z The “lower level Services” would not be visible at Distribution Time. Hence a “flattening” of the run-time orchestration would result. VNF_X VNF_Y VNF_Z

Allotted Resources – vPE/VRF Example 1 4 Every Resource can be exposed as a Service. The ONAP model supports this today through the “Allotted Resource” construct. This concept of “Allotted Resource” does not seem to appear in the ETSI model. Perhaps this is due to ETSI seemingly covering only instantiation of Infrastructure Services, and not instantiation of end Customer Services. An instantiation request for a L3VPN_Cust Service would result in a VRF being instantiated. That VRF would be “homed” to an existing vPE_Infra Service instance (i.e., the vPE VNF instance on which this VRF will be configured). “Higher Level” Service 2 L3VPN_Cust Service: topology_template: node_templates: VRF In this case, the vPE VNF has been packaged as an Infrastructure Service. An instantiation request for this vPE_Infra Service would result in a new vPE VNF being instantiated. VRF Allotted Resource requirement: VRF_Capability “Lower Level” Service vPE VNF vPE_Infra Service: topology_template: node_templates: vPE VNF vPE_Network capability: VRF_Capability 3 The vPE_Infra Service exposes a capability to provide “VRFs” (a “VRF_Capability”). The L3VPN_Cust Service consumes this capability through its “VRF Allotted Resource” construct. Network

Comparison with other industry work ETSI MANO: ONAP’s separation of infrastructure resource orchestration using multi-VIM layer aligns with MANO VIM module (with added scope in ONAP to support multiple clouds) ONAP SO (with service and resource orchestration) provides super set of functionality of ETSI MANO NFVO. However, ONAP has broadened the MANO scope by including the following new concepts: Concept of “Allotted Resource” is not discussed in MANO. An “Allotted Resource” is a resource that is provided by an underlying Service, and which is allotted to a customer’s (for example) Service Instance. E.g., a “VRF” would be modeled as an “Allotted Resource” provided by a virtual PE Router VNF. We see several examples of allocated resources in 5G (e.g. Slicing). Multiple level of service nesting (ETSI MANO has only one level of nesting in Network Services). MEF LSO: MEF’s main focus is to define standard interfaces between service provider to customers (including another service provider) Focus is on standardizing BSS to BSS interfaces Network Function virtualization is not main focus of MEF Therefore, MEF’s definition of “Infrastructure” conflicts with ETSI MANO as well as ONAP’s definition of infrastructure, as cloud resources and VNF are combined into “infrastructure”

Conclusions Service Providers require the ability to define Services that can be leveraged by higher order Services. Consistent with ETSI MANO, ONAP should not separate Service Orchestration and Resource Orchestration into two separate named components, because each Resource can be exposed as a Service What appears as a “Resource” from one Service’s perspective, may actually be a Service from another perspective. ONAP should provide Service Providers the ability to model their “Service reuse” to drive run-time behavior on a service-by-service basis, including: Allowing each Service/Resource level to result in a separate set of orchestration threads. Allow the Service Provider to “flatten” some levels of reuse at run time (e.g., through design time substitution mapping). ONAP should provide Service Providers deployment flexibility to address scalability, geographic distribution and redundancy / resiliency But should not impose extra complexity and cost on service providers by separating service / resource orchestration that provides little known benefit

Role Of Controllers in ONAP: ONAP Controllers configure and maintain the health of network services and functional elements throughout their lifecycle. Programmable network application management platform Behavior patterns programmed via models and policies Standards based models & protocols for multi-vendor implementation Operational control, coordinated state changes across devices, source of telemetry/events, etc. Manages the health of the network services & VNFs in its scope Policy-based optimization to meet SLAs Event-based control loop automation to solve local issues near real-time Telemetry source and action executor for outer control loop automation Self-healing and auto-scaling monitoring framework Local source of truth Manages inventory within its scope All stages/states of lifecycle Configuration audits

ONAP Controller Guiding Principles: Controllers manage the state of services & network functions (VNFs / PNFs) There could be multiple controllers to support different technology domains – e.g. L3 Networks, Optical networks, L4-7 network applications, etc. Controllers have intimate with domain specific protocols However, all ONAP controllers must support one set of common north bound APIs (for ONAP modules, all controllers look and behave the same) There should be functional parity amongst various controllers This is a critical requirement, otherwise SO and other ONAP module has to keep track of differences between the controllers We must leverage common controller framework, as much as possible (reduce complexity, development and maintenance cost) All controllers have common functional modules and structure (accepting models from SDC, parsing it, north bound APIs, topology database, lifecycle management functions, etc.) Differences are only at the VNF / PNF interface layer Leveraging common controller framework provides tremendous advantage, much smaller total code, changes are made once and leveraged by all instances, etc. Avoid having two controllers with ONAP supporting same technology domain (e.g. L3 Networking, Optical, IMS, etc.)

Leveraging Common Controller Framework Service & Resource LCM Actions VNFs PNFs Multi-VIM Common Controller Framework Plugins/Adapters* M-VIM APIs API Handler Rebuild VNF Configure Network Config Stop Audit Start Scale μSvcs APIs Models TOSCA engine DGs SLI Service Chain Heal Common Control Functions Model Parser Policy Engine Topology & Engineering Rules ODL / MDSAL * Plugins/Adapters could include Ansible, Chef, Puppet, Netconf / Yang, CLI, SNMP, standard APIs, EMS, VNFM, etc. Personas … Note: Common controller framework is a rich collection of libraries. Each domain specific controller can include only applicable modules.

ONAP Controller Rationalization Opportunities: L4-7 Controller (App-C) Vendor VNFM Adaption Layer (VF-C) Config Database Service Logic Interpreter Life Cycle Mgmt funcs Life Cycle Mgmt funcs Standard Adaption Layer Vendor OA&M Adaptor Vendor A Vendor B Chef Netconf Ansible L4-7 Generic ONAP Controller Model Parser API Handler DGs SLI TOSCA engine Policy Engine Models Topology & Engineering Rules Common Control Functions Rebuild Audit VNF Configure Stop Scale Network Config Start Heal Service Chain ODL / MDSAL ?? Plugins/Adapters* μSvcs APIs M-VIM APIs VNFs PNFs EMS sVNFM Multi-VIM VNFs PNFs VNFs PNFs Note: Team can decide which Common controller framework libraries are applicable for L4-7 VNF controller.

Conclusions ONAP community should create a rich library in Common Controller Framework (CCF) – This offers maximum reusability and benefits are shared amongst all ONAP members CCF can be used to create controller instances of varying “personalities” (L1-3, L4-7, OOM, Optical, RAN, etc.).  Initially these controllers are created statically and included in ONAP Platform but the goal is to create them more dynamically by ONAP community members to meet their requirements Avoid redeveloping functionality already available in CCF Encourage community to contribute to CCF functions that be used by other controller (today or in future) However, we should avoid multiple controllers covering same domain or family of VNFs (e.g RAN, IMS, etc.) Provide clear & consistent guidelines to network function vendors Each Controller must follow agreed upon architecture principles: All ONAP controllers must support one set of common north bound APIs (for ONAP modules, all controllers look and behave the same) There should be functional parity amongst various controllers Allow operator deployment flexibility meet their unique scalability, resiliency and geographic distribution requirements Functions which are unique or specific to a controller can be developed within a controller and can live outside CCF 

Backup Slides

Terminology Mapping ONAP ETSI MANO (1) Service Network Service (Infrastructure Services only) (2) Service Orchestration Network Service Orchestration (2) Resource VNF and Network (NFVI) network resources (3) Resource Orchestration Specific NFVO functions that manage or coordinate the VNF or NFVI network resource’s lifecycle (3) Cloud Resource NFVI compute, storage, network Cloud Resource Orchestration Functions, perhaps provided by the underlying Cloud Provider (4), that manage or coordinate the NFVI compute, storage, network resource’s lifecycle Network An NFVI resource Allotted Resource (5) This concept is not discussed in MANO VF Module Virtualisation Deployment Unit (VDU) VF Component VNFC Because of this, ETSI TERMINOLOGY IN AND OF ITSELF IS NOT SUFFICIENT TO DESCRIBE THE ONAP PROBLEM SPACE. Ref: ETSI - NFV Terminology for Main Concepts in NFV ONAP includes “Customer Services” that ride on top of what ETSI would call a “Network Service”. Also ETSI considers a “Network Service” as always including at least one dedicated VNF, whereas ONAP allows for “Services” that do not include any dedicated VNFs. Concept of “Allotted Resource” is not discussed in MANO. The concept of “PNF” seems to be out of scope for an NFV-MANO implementation. For example, the orchestration provided by OpenStack HEAT An “Allotted Resource” is a resource that is provided by an underlying Service, and which is allotted to a customer’s (for example) Service Instance. E.g., a “VRF” would be modeled as an “Allotted Resource” provided by a virtual PE Router VNF.

A&AI Instance Representation of Prior Example In this example, the entirety of VNF_W is dedicated to the Service_W Service Instance, but only a portion (represented in A&AI as AllottedResource_W) of the “lower level” Service_X Service Instance is dedicated to the Service_W Service Instance. This pattern repeats itself for the other Service Instances shown. Service_W Svc Instance Instance “1” AllottedResource_W AR Instance AllottedResource_X AR Instance Instance “2” AllottedResource_Y AR Instance Instance “3” Service_X Svc Instance Service_Y Svc Instance Service_Z Svc Instance Instance “4” VNF_W VNF Instance VNF_X VNF Instance VNF_Y VNF Instance VNF_Z VNF Instance