1. Rationalizing ONAP Architecture for R2 and Beyond

2. ONAP R2+ Architecture – Current Draft View
OSS
BSS
ONAP Portal – Design Studio & Dashboard
Design-time (SDC)
Run-time
ONAP Operations Management (OOM)
External Data Movement & APIs
Resource Onboarding
Orchestration (SO)
Inventory/Topology (AAI)
Data & Analytics (DCAE)
Policy
VNF/PNF SDK
Service Level Orchestration
Active / Available
DCAE Analytic µServices
Service & Product Design
Policy Execution Engine
Entitlements
CDAP
Holmes …
Policy Creation & Validation
Resource Level Orchestration
Resource / Service Topology
Data Distribution
ESR
Data Collection Layer
Analytic Application Design
Common
Services
DMaaP
AAF
Logging
Micro Services Bus
Catalog
Multi-VIM Interface
Open
Stack
AWS
Azure
MEC
Infrastructure
Adaption Layer
Adaption Layer
Service Logic
Interpreter
Config
Database
Yang
Netconf
CLI
Life Cycle Mgmt Func
L0-3 Network Controller (SDN-C)
3rd Party Network
controller
L4-7 Controller (App-C)
Testing & Certification
Vendor VNFM
Adaption Layer
(VF-C)
Products
Resources
Eng. Rules
Services
Policies
Analytics
Config Database
Service Logic Interpreter
Life Cycle Mgmt funcs
Life Cycle Mgmt funcs
Standard Adaption Layer
Vendor OA&M Adaptor
Vendor A
Vendor B
Chef
Netconf
Ansible
Recipe / Engineering Rules & Policy Distribution
sVNFM
sVNFM
sVNFM
Further Rationalization
Opportunities …
VNF1
VNF2
VNF3
PNF 1
VNFn
Infrastructure
OpenStack
VMware
RackSpace
Azure
Peripherals

3. Open Issues:
What is the right Orchestration Implementation in ONAP & How to provide deployment flexibility?
Rationalization of Controller Family within ONAP

4. Role Of Orchestrator in ONAP:
ONAP Orchestrator must orchestrate Network Functions (VNF / PNF), Services and coordinate cross-controller activities driven by orders and events that indicate the need to instantiate, modify or remove Network Functions (VNF / PNF), services as well as multi- control layer remedy actions. Key attributes of Orchestrator are
Model Driven Orchestration
Process behavior driven by resource / service model designed in SDC
Orchestrates network functions and service Instantiations
Support software upgrade and planned change management
Support open & closed-loop control actions initiated by DCAE /Policy or external API (from OSS for Open Loop)
Tracks orchestrated activity for the life of the request but doesn’t control state of orchestrated components
Interfaces to External OSS / BSS Systems
Standard APIs exposed northbound to create, modify or remove services
Request decomposition of service request into service / network function components
Selects model / recipe that supports the request and maps order data to recipe / model
Coordinating Activities Across Various ONAP Controllers
Coordinates activities across Multi-VIM (infrastructure)/SDN-C//app-C/VF-C controllers
Manage recording of service configurations
Orchestrator Performs Following Additional Services
Coordinates current inventories and placement/optimization recommendations
Enforces business & technical policies
Support standards-based workflows
Validates and records network functions / service implementations

5. Examples of Orchestration Requests
Install a single network function (VNF or PNF) – e.g. Deploy a new virtual PE / P router
A virtual network element might have one or more components (e.g VNFCs or VDUs)
Install multiple copies of network functions in one or more data centers – e.g. Deploy new virtual PE / P routers across n locations
Deploy one or more instances of network services – e.g. RAN at Multiple sites, one or more sites of EPC, etc.
Instantiate a customer services – e.g. multi-site VPN or IP SEC
Could require deploying new instances of network elements (e.g. vCE) and using existing elements (e.g. vPE)
Could be using existing elements without any new VNF / PNF being deployed (e.g. “Allocated Resources” – configuring a service to existing components)
Create a new instance of a network service – e.g. IoT or eMBB slice
Could be leveraging existing network elements (RAN, vEPC, Transport etc.) without any new VNF / PNF being deployed
Implement a software upgrade / change management
Could require Coordination of activities across Multi-VIM (infrastructure)/SDN-C/app-C/VF-C controllers
Orchestrator requested to remedial action as part of Closed or open loop action
Could require adding new compute / network resource to the existing functional elements (i.e. VNF)
Create a new instance of VNF and migrate traffic
Could require Coordination of activities across Multi-VIM (infrastructure)/SDN-C/app-C/VF-C controllers)

6. Model-Driven Orchestration
“Generic” model-driven Service flow (limited existing)
Each Resource Type has its own “Generic” model-driven flow. There currently exist such flows for “VNF” and “Network” Resource Types.
Service Orchestration
Resource Orchestration
Cloud Resource Orchestration
PNF
Service X:
topology_template:
node_templates:
PNF
Network
VNF
Allotted Resource
Network
VF Module
Note recursion
VNF
Service Orchestration
Resource Orchestration
Allotted Resource
requirement: A
PNF
An Allotted Resource can be homed to an existing “underlying” network function or Service Instance, or homing could determine that a new network function Service Instance is needed. This would result in a 2nd level of Service Orchestration.
Service Y:
topology_template:
node_templates:
PNF
Network
VNF
Allotted Resource
capability: A
Network
Service Y is being treated as a “Resource” from the perspective of Service X.
VNF
Allotted Resource

7. N-Level Run Time Nesting? Let The Service Providers Decide
Each Service/Resource “reuse unit” results in a separate thread of orchestration. This would allow for “on demand” spin up of “lower level” (Infrastructure) Service instances.
Could be extended to allow multiple “higher level Services” to each have a “share” of a “lower level Service’s” instance
See next slide for an alternative modeling approach that could be used when a “higher level Service” consumes the entirety of a “lower level Service’s” instance. This alternate approach may be appropriate for a Service Provider who is concerned about the run-time complexity that can be introduced with N-Level run time recursion.

8. N-Level Run Time Nesting? Let The Service Providers Decide
For the case whereby a “higher level Service” consumes the entirety of a “lower level Service’s” instance, SDC should support the Design Time ability to construct an “upper” Service Definition from other Services definitions via substitution mapping (a.k.a., “Compile Time Nesting”)
Service W:
topology_template:
node_templates:
VNF_W
Service_X
Service X:
topology_template:
node_templates:
VNF_X
Service_Y
Service Y:
topology_template:
node_templates:
VNF_Y
Service_Z
Service Z:
topology_template:
node_templates:
VNF_Z
Substitution Mapping
Design Time
Distribution Time & Run Time
VNF_W
Service_W:
topology_template:
node_template:
VNF_W
VNF_X
VNF_Y
VNF_Z
The “lower level Services” would not be visible at Distribution Time. Hence a “flattening” of the run-time orchestration would result.
VNF_X
VNF_Y
VNF_Z

9. Allotted Resources – vPE/VRF Example1 4
Every Resource can be exposed as a Service. The ONAP model supports this today through the “Allotted Resource” construct. This concept of “Allotted Resource” does not seem to appear in the ETSI model. Perhaps this is due to ETSI seemingly covering only instantiation of Infrastructure Services, and not instantiation of end Customer Services.
An instantiation request for a L3VPN_Cust Service would result in a VRF being instantiated. That VRF would be “homed” to an existing vPE_Infra Service instance (i.e., the vPE VNF instance on which this VRF will be configured).
“Higher Level” Service 2
L3VPN_Cust Service:
topology_template:
node_templates:
VRF
In this case, the vPE VNF has been packaged as an Infrastructure Service. An instantiation request for this vPE_Infra Service would result in a new vPE VNF being instantiated.
VRF Allotted Resource
requirement: VRF_Capability
“Lower Level” Service
vPE VNF
vPE_Infra Service:
topology_template:
node_templates:
vPE VNF
vPE_Network
capability: VRF_Capability 3
The vPE_Infra Service exposes a capability to provide “VRFs” (a “VRF_Capability”). The L3VPN_Cust Service consumes this capability through its “VRF Allotted Resource” construct.
Network

10. Comparison with other industry work
ETSI MANO:
ONAP’s separation of infrastructure resource orchestration using multi-VIM layer aligns with MANO VIM module (with added scope in ONAP to support multiple clouds)
ONAP SO (with service and resource orchestration) provides super set of functionality of ETSI MANO NFVO.
However, ONAP has broadened the MANO scope by including the following new concepts:
Concept of “Allotted Resource” is not discussed in MANO.
An “Allotted Resource” is a resource that is provided by an underlying Service, and which is allotted to a customer’s (for example) Service Instance. E.g., a “VRF” would be modeled as an “Allotted Resource” provided by a virtual PE Router VNF. We see several examples of allocated resources in 5G (e.g. Slicing).
Multiple level of service nesting (ETSI MANO has only one level of nesting in Network Services).
MEF LSO:
MEF’s main focus is to define standard interfaces between service provider to customers (including another service provider)
Focus is on standardizing BSS to BSS interfaces
Network Function virtualization is not main focus of MEF
Therefore, MEF’s definition of “Infrastructure” conflicts with ETSI MANO as well as ONAP’s definition of infrastructure, as cloud resources and VNF are combined into “infrastructure”

11. Conclusions
Service Providers require the ability to define Services that can be leveraged by higher order Services.
Consistent with ETSI MANO, ONAP should not separate Service Orchestration and Resource Orchestration into two separate named components, because each Resource can be exposed as a Service
What appears as a “Resource” from one Service’s perspective, may actually be a Service from another perspective.
ONAP should provide Service Providers the ability to model their “Service reuse” to drive run-time behavior on a service-by-service basis, including:
Allowing each Service/Resource level to result in a separate set of orchestration threads.
Allow the Service Provider to “flatten” some levels of reuse at run time (e.g., through design time substitution mapping).
ONAP should provide Service Providers deployment flexibility to address scalability, geographic distribution and redundancy / resiliency
But should not impose extra complexity and cost on service providers by separating service / resource orchestration that provides little known benefit

12. Role Of Controllers in ONAP:
ONAP Controllers configure and maintain the health of network services and functional elements throughout their lifecycle.
Programmable network application management platform
Behavior patterns programmed via models and policies
Standards based models & protocols for multi-vendor implementation
Operational control, coordinated state changes across devices, source of telemetry/events, etc.
Manages the health of the network services & VNFs in its scope
Policy-based optimization to meet SLAs
Event-based control loop automation to solve local issues near real-time
Telemetry source and action executor for outer control loop automation
Self-healing and auto-scaling monitoring framework
Local source of truth
Manages inventory within its scope
All stages/states of lifecycle
Configuration audits

13. ONAP Controller Guiding Principles:
Controllers manage the state of services & network functions (VNFs / PNFs)
There could be multiple controllers to support different technology domains – e.g. L3 Networks, Optical networks, L4-7 network applications, etc.
Controllers have intimate with domain specific protocols
However, all ONAP controllers must support one set of common north bound APIs (for ONAP modules, all controllers look and behave the same)
There should be functional parity amongst various controllers
This is a critical requirement, otherwise SO and other ONAP module has to keep track of differences between the controllers
We must leverage common controller framework, as much as possible (reduce complexity, development and maintenance cost)
All controllers have common functional modules and structure (accepting models from SDC, parsing it, north bound APIs, topology database, lifecycle management functions, etc.)
Differences are only at the VNF / PNF interface layer
Leveraging common controller framework provides tremendous advantage, much smaller total code, changes are made once and leveraged by all instances, etc.
Avoid having two controllers with ONAP supporting same technology domain (e.g. L3 Networking, Optical, IMS, etc.)

14. Leveraging Common Controller Framework
Service & Resource LCM Actions
VNFs
PNFs
Multi-VIM
Common Controller Framework
Plugins/Adapters*
M-VIM APIs
API Handler
Rebuild
VNF Configure
Network Config
Stop
Audit
Start
Scale
μSvcs APIs
Models
TOSCA engine
DGs
SLI
Service Chain
Heal
Common Control Functions
Model Parser
Policy Engine
Topology & Engineering Rules
ODL / MDSAL
* Plugins/Adapters could include Ansible, Chef, Puppet, Netconf / Yang, CLI, SNMP, standard APIs, EMS, VNFM, etc.
Personas …
Note: Common controller framework is a rich collection of libraries. Each domain specific controller can include only applicable modules.

15. ONAP Controller Rationalization Opportunities:
L4-7 Controller (App-C)
Vendor VNFM
Adaption Layer
(VF-C)
Config Database
Service Logic Interpreter
Life Cycle Mgmt funcs
Life Cycle Mgmt funcs
Standard Adaption Layer
Vendor OA&M Adaptor
Vendor A
Vendor B
Chef
Netconf
Ansible
L4-7 Generic ONAP Controller
Model Parser
API Handler
DGs
SLI
TOSCA engine
Policy Engine
Models
Topology & Engineering Rules
Common Control Functions
Rebuild
Audit
VNF Configure
Stop
Scale
Network Config
Start
Heal
Service Chain
ODL / MDSAL ??
Plugins/Adapters*
μSvcs APIs
M-VIM APIs
VNFs
PNFs
EMS
sVNFM
Multi-VIM
VNFs
PNFs
VNFs
PNFs
Note: Team can decide which Common controller framework libraries are applicable for L4-7 VNF controller.

16. Conclusions
ONAP community should create a rich library in Common Controller Framework (CCF) – This offers maximum reusability and benefits are shared amongst all ONAP members
CCF can be used to create controller instances of varying “personalities” (L1-3, L4-7, OOM, Optical, RAN, etc.).  Initially these controllers are created statically and included in ONAP Platform but the goal is to create them more dynamically by ONAP community members to meet their requirements
Avoid redeveloping functionality already available in CCF
Encourage community to contribute to CCF functions that be used by other controller (today or in future)
However, we should avoid multiple controllers covering same domain or family of VNFs (e.g RAN, IMS, etc.)
Provide clear & consistent guidelines to network function vendors
Each Controller must follow agreed upon architecture principles:
All ONAP controllers must support one set of common north bound APIs (for ONAP modules, all controllers look and behave the same)
There should be functional parity amongst various controllers
Allow operator deployment flexibility meet their unique scalability, resiliency and geographic distribution requirements
Functions which are unique or specific to a controller can be developed within a controller and can live outside CCF 

17. Backup Slides

18. Terminology Mapping
ONAP
ETSI MANO (1)
Service
Network Service (Infrastructure Services only) (2)
Service Orchestration
Network Service Orchestration (2)
Resource
VNF and Network (NFVI) network resources (3)
Resource Orchestration
Specific NFVO functions that manage or coordinate the VNF or NFVI network resource’s lifecycle (3)
Cloud Resource
NFVI compute, storage, network
Cloud Resource Orchestration
Functions, perhaps provided by the underlying Cloud Provider (4), that manage or coordinate the NFVI compute, storage, network resource’s lifecycle
Network
An NFVI resource
Allotted Resource (5)
This concept is not discussed in MANO
VF Module
Virtualisation Deployment Unit (VDU)
VF Component
VNFC
Because of this, ETSI TERMINOLOGY IN AND OF ITSELF IS NOT SUFFICIENT TO DESCRIBE THE ONAP PROBLEM SPACE.
Ref: ETSI - NFV Terminology for Main Concepts in NFV
ONAP includes “Customer Services” that ride on top of what ETSI would call a “Network Service”. Also ETSI considers a “Network Service” as always including at least one dedicated VNF, whereas ONAP allows for “Services” that do not include any dedicated VNFs.
Concept of “Allotted Resource” is not discussed in MANO. The concept of “PNF” seems to be out of scope for an NFV-MANO implementation.
For example, the orchestration provided by OpenStack HEAT
An “Allotted Resource” is a resource that is provided by an underlying Service, and which is allotted to a customer’s (for example) Service Instance. E.g., a “VRF” would be modeled as an “Allotted Resource” provided by a virtual PE Router VNF.

19. A&AI Instance Representation of Prior Example
In this example, the entirety of VNF_W is dedicated to the Service_W Service Instance, but only a portion (represented in A&AI as AllottedResource_W) of the “lower level” Service_X Service Instance is dedicated to the Service_W Service Instance. This pattern repeats itself for the other Service Instances shown.
Service_W Svc Instance
Instance “1”
AllottedResource_W AR Instance
AllottedResource_X AR Instance
Instance “2”
AllottedResource_Y AR Instance
Instance “3”
Service_X Svc Instance
Service_Y Svc Instance
Service_Z Svc Instance
Instance “4”
VNF_W VNF Instance
VNF_X VNF Instance
VNF_Y VNF Instance
VNF_Z VNF Instance