1. OPNFV projects and project life cycle
Frank Brockners, Cisco

3. OPNFV Creates, Composes, Deploys, Tests, and starts over... (iterate)

4. VOID

5. OpenSource Building Blocks For NFV
PaaS
Application Layer / App Server *
Additional PaaS platforms
Cloud Infra & Tooling
Network Data Analytics *
PaNDA
Orchestration
Open-O *
VIM Management System
Network Control *
Infrastructure
Operating Systems * * *
IO Abstraction & Feature Path
Hardware
CI/CD
Evolve/Integrate/Install/Test
*New to LF in 2015/2016

6. OPNFV Arno Release Application Layer / App Server
PaaS
Application Layer / App Server
Cloud Infra & Tooling
Network Data Analytics
Orchestration
VIM Management System
Network Control
Infrastructure
Operating Systems
IO Abstraction & Feature Path
Hardware
CI/CD
Evolve/Integrate/Install/Test

7. 5 Projects Participated in Arno
Lab Infrastructure: “Pharos”
Create and maintain a system-level CI/CD system: “Octopus”
System Composition and Installation: “Bootstrap/GetStarted”
Functional Testing: “FuncTest”
Documentation: “OPNFV Docs”

8. Bootstrap/Get Started was to compose the first system: Different Implementations of Components & Features
Compute
Storage
Network
Infra
Operations
Deployment
Test
VM control
Store Images (VNFs)
Identity Services /AAA
Message Bus
Cluster Comms
HA & Loadbalance
Portal
Network Control
Hypervisor
Virtual Forwarder
Virtual disks (block storage)
Data-Base
Orchestration
Object store
Telemetry
Unit Test
System Test
Performance
Installer
VM control
Store Images (VNFs)
Identity Services /AAA
Message Bus
Cluster Comms
HA & Loadbalance
Portal
Network Control
Hypervisor
Virtual Forwarder
Virtual disks (block storage)
Data-Base
Orchestration
Object store
Telemetry
Unit Test
System Test
Performance
Installer
VM control
Store Images (VNFs)
Identity Services /AAA
Message Bus
Cluster Comms
HA & Loadbalance
Portal
Network Control
Hypervisor
Virtual Forwarder
Virtual disks (block storage)
Data-Base
Orchestration
Object store
Telemetry
Unit Test
System Test
Performance
Installer

9. Bootstrap/GetStarted in OPNFV Arno Release: Initial composition of 2 base scenarios
Fixed Set of Hardware
Fixed Set of Components
Set of Tools (some choice)
6 Servers >= 3 Control nodes, >= 2 Compute nodes, 1 Provisioning node (“jumphost”) (Centos 7, runs test infra and installer VM)
Reference lab hosted by the Linux Foundation Blade servers with 80G connectivity each (Cisco UCS-B), CIMC management
Compute O/S Nova, KVM
Storage O/S Glance, O/S Cinder
Network OpenDaylight, OVS
Infra RabbitMQ, Corosync, Pacemaker, MySQL
Test Tempest, Rally, Robot Install
Options:
Fuel
Foreman/Quickstack
2 deployment scenarios
See also:

10. Functional Testing: Project FuncTest
A: vPing SSH
Verify private & public connectivity
A: vPing userdata
Verify nova-metadata service and private connectivity
A: Tempest test
OpenStack native tests (200+ tests)
A: Rally bench tests
Benchmark the OpenStack deployment
A: ODL suite
Robot framework, ODL functional testing
B: ONOS suite
TestON framework
B: vIMS
OpenSource solution by Clearwater
B: Promise
Resource reservation and management project
B: Doctor
Fault management and maintenance project
B: BGPVPN
OpenStack Neutron BGPVPN project integration
Brahmaputra Additions
Brahmaputra Additions

11. FuncTest Results Database / Dashboard http://testresults. opnfv
Credentials: opnfvuser/kibana

12. OPNFV Brahmaputra Application Layer / App Server
PaaS
Application Layer / App Server
Cloud Infra & Tooling
Network Data Analytics
Orchestration
VIM Management System
Network Control
Infrastructure
Operating Systems
IO Abstraction & Feature Path
Hardware
CI/CD
Evolve/Integrate/Install/Test

13. OPNFV Brahmaputra 20+ deployment scenarios Apex Fuel Compass JOID Arno
O/S – ODL Baremetal
O/S – ODL Baremetal
O/S – ODL Baremetal
O/S – ODL Baremetal
O/S – ODL VM
O/S – ODL VM
Arno SR1
O/S – ODL – L3 Baremetal
O/S – ODL – L3 Baremetal
20+ deployment scenarios
O/S – ODL – SFC Baremetal
O/S – ODL – BGPVPN Baremetal
O/S – ONOS Baremetal
O/S – ONOS Baremetal
O/S – ONOS Baremetal
O/S – ONOS Baremetal
O/S – OpenContrail Baremetal
O/S – OpenContrail Baremetal
O/S – OpenContrail Baremetal
Brahmaputra
Note: Not all scenarios shown. For an up to date view on scenarios see

14. OPNFV Colorado Application Layer / App Server Network Data Analytics
PaaS
Application Layer / App Server
Cloud Infra & Tooling
Network Data Analytics
Orchestration
VIM Management System
Network Control
Infrastructure
Operating Systems
IO Abstraction & Feature Path
Hardware
CI/CD
Evolve/Integrate/Install/Test

15. Enhancing the Stack while Integrating: A Map of OPNFV Projects
PaaS
Application Layer / App Server
Cloud Infra & Tooling
Network Data Analytics
Orchestration
NFV deployment architectures
(Multi-Site, EdgeNFV)
VIM Management System
Fault Management/Localization, Audit (Doctor, Pinpoint, Prediction, Bottlenecks, Inspector)
Resource Control/Scheduling
(Promise, Resource Scheduler (RS))
Model and Policy Driven Control
(Copper, Movie, Models, Domino, Parser)
Network Control
Network Policy
(FastDataStacks, Copper, PolicyTest)
NFV services (service chaining, VPN, ..)
(VNFFG, SFC, SDNVPN)
SDN Controller Performance Test
(Cperf)
Infrastructure
Operating Systems, virt. Infra
Hypervisor for NFV
(KVM4NFV)
IO Abstraction & Feature Path
Dataplane Performance Test
(Vsperf, Fastpath)
Dataplane Evolution & Test
(DPACC, OVSDPDK, FastDataStacks)
Hardware
Infra control & CI/CD
(Pharos, Releng, Octopus)
CI/CD
Integrate/Install
Create Scenarios/Stacks
(IPv6, FastDataStacks, Armband, Apex, Daisy ONOSFW, OpenContrail, Joid, Fuel, Compass)
Installation, Upgrade
(Apex, Joid, Fuel, Compass, Daisy
Escalator, Genesis)
Documentation
(OPNFVdocs)
System Test & Security Audit
System Testing
(Yardstick, FuncTest, Qtip)
Storage Performance Test
(Storperf)
Security
(Moon)

16. Evolve Features in lock-step with Upstream: Example: OPNFV SFC Project
PaaS
Application Layer / App Server
Cloud Infra & Tooling
Network Data Analytics
NFV Telemetry Data Acquisition
(VNF Event Stream)
Orchestration
NFV deployment architectures
(Multi-Site, EdgeNFV)
VIM Management System
Fault Management/Localization, Audit (Doctor, Pinpoint, Prediction, Bottlenecks, Inspector)
Resource Control/Scheduling
(Promise, Resource Scheduler (RS))
Model and Policy Driven Control
(Copper, Movie, Models, Domino, Parser)
Network Control
Network Policy
(FastDataStacks, Copper, PolicyTest)
NFV services (service chaining, VPN, ..)
(VNFFG, SFC, SDNVPN)
SDN Controller Performance Test
(Cperf)
Infrastructure
Operating Systems, virt. Infra
Hypervisor for NFV
(KVM4NFV)
IO Abstraction & Feature Path
Dataplane Performance Test
(Vsperf, Fastpath)
Dataplane Evolution & Test
(DPACC, OVSDPDK, FastDataStacks)
Hardware
Infra control & CI/CD
(Pharos, Releng, Octopus)
CI/CD
Integrate/Install
Create Scenarios/Stacks
(IPv6, FastDataStacks, Armband, Apex, Daisy, ONOSFW, OpenContrail,, Joid, Fuel, Compass)
Installation, Upgrade
(Apex, Joid, Fuel, Compass, Daisy,
Escalator, Genesis)
Documentation
(OPNFVdocs)
System Test & Security Audit
System Testing
(Yardstick, FuncTest, Qtip)
Storage Performance Test
(Storperf)
Security
(Moon)

17. Feature Evolution Example: OPNFV SFC Project
Objective: Verify ODL SFC in system level deployments
Brahmaputra Yardstick tests:
TC029: VM Creation
Verify that only 2 Service Function VMs are created for Service Chains Chain1 and Chain2
TC030: Block HTTP
Verify that Client1 can not do HTTP traffic, but can do SSH traffic
TC031: Block SSH
Verify that Client2 can not do SSH traffic, but can do HTTP traffic
See also:

18. System-level, ETSI-focused Testing Example Project: Yardstick
PaaS
Application Layer / App Server
Cloud Infra & Tooling
Network Data Analytics
NFV Telemetry Data Acquisition
(VNF Event Stream)
Orchestration
NFV deployment architectures
(Multi-Site, EdgeNFV)
VIM Management System
Fault Management/Localization, Audit (Doctor, Pinpoint, Prediction, Bottlenecks, Inspector)
Resource Control/Scheduling
(Promise, Resource Scheduler (RS))
Model and Policy Driven Control
(Copper, Movie, Models, Domino, Parser)
Network Control
Network Policy
(FastDataStacks, Copper, PolicyTest)
NFV services (service chaining, VPN, ..)
(VNFFG, SFC, SDNVPN)
SDN Controller Performance Test
(Cperf)
Infrastructure
Operating Systems, virt. Infra
Hypervisor for NFV
(KVM4NFV)
IO Abstraction & Feature Path
Dataplane Performance Test
(Vsperf, Fastpath)
Dataplane Evolution & Test
(DPACC, OVSDPDK, FastDataStacks)
Hardware
Infra control & CI/CD
(Pharos, Releng, Octopus)
CI/CD
Integrate/Install
Create Scenarios/Stacks
(IPv6, FastDataStacks, Armband, Apex, Daisy, ONOSFW, OpenContrail,, Joid, Fuel, Compass)
Installation, Upgrade
(Apex, Joid, Fuel, Compass, Daisy,
Escalator, Genesis)
Documentation
(OPNFVdocs)
System Test & Security Audit
System Testing
(Yardstick, FuncTest, Qtip)
Storage Performance Test
(Storperf)
Security
(Moon)

19. Example testing activity in OPNFV: System-Level testing: Project YardStick
Pre-Deployment validation of NFVI
YardStick Test-Framework
ETSI GS NFV-TST001
Chapter 6
Pre-Deployment validation
of NFV infrastructure
1. Define infrastructure
2. Identify VNF-type
3. Select Test Cases
4. Execute Tests
5. Collect Results
6. Integrate VNF application
Configure
Deploy
Test
Validate
System under Test (SUT)
Deployment rules
Execute benchmarks
Trigger External scripts for SUT disturbances
Collect results
Graphical results
Assertion
Generate template or
External templates
Control plane/User plane
Main components
Real time constraints
Hardware
Metrics
Test cases 1 … N
Test stimuli
Test environment

20. Reliability/Availability
YardStick Test Suite
Performance/Speed
Capacity/Scale
Reliability/Availability
Compute
- Latency for random memory access - Latency for cache read/write operations
- Processing speed (instructions per second)
-Throughput for random memory access (bytes per second)
- Number of cores and threads
- Available memory size
- Cache size
- Processor utilization (max, average, standard deviation)
- Memory utilization (max, average, standard deviation)
- Cache utilization (max, average, standard deviation)
- Processor availability (Error free processing time)
- Memory availability (Error free memory time)
- Processor mean-time-to-failure
- Memory mean-time-to-failure
- Number of processing faults per second
Network
- Throughput per NFVI node (frames/byte per second)
- Throughput provided to a VM (frames/byte per second)
- Latency per traffic flow
- Latency between VMs
- Latency between NFVI nodes
- Packet delay variation (jitter) between VMs
- Packet delay variation (jitter) between NFVI nodes
- Number of connections
- Number of frames sent/received
- Maximum throughput between VMs (frames/byte per second)
- Maximum throughput between NFVI nodes (frames/byte per second)
- Network utilization (max, average, standard deviation)
- Number of traffic flows
- NIC availability (Error free connection time)
- Link availability (Error free transmission time)
- NIC mean-time-to-failure
- Network timeout duration due to link failure
- Frame loss rate
Storage
- Sequential read/write IOPS
- Random read/write IOPS
- Latency for storage read/write operations
- Throughput for storage read/write operations
- Storage/Disk size
- Capacity allocation (block-based, object-based)
- Block size
- Maximum sequential read/write IOPS
- Maximum random read/write IOPS
- Disk utilization (max, average, standard deviation)
- Disk availability (Error free disk access time)
- Disk mean-time-to-failure
- Number of failed storage read/write operations per second

21. YardStick Results Database / Dashboard http://testresults. opnfv

22. OPNFV Testing Approaches
Functional:
Functest: base system functionality, component testing, VIM
Performance/System:
Yardstick: End to end system level based on ETSI TST001
Bottlenecks: Verify OPNFV boundaries in staging before bringing VNFs
StorPerf: Storage performance for any VNF
VSPerf: Virtual Switch characterization
CPerf: Controller performance
PolicyTest: Policy testing
QTip: Bare metal focus (compute, network, storage)
For discussion of testing ecosystem, see:

23. Integrate and Evolve Upstream in lock-step: Example Project: OPNFV FastDataStacks
PaaS
Application Layer / App Server
Cloud Infra & Tooling
Network Data Analytics
NFV Telemetry Data Acquisition
(VNF Event Stream)
Orchestration
NFV deployment architectures
(Multi-Site, EdgeNFV)
VIM Management System
Fault Management/Localization, Audit (Doctor, Pinpoint, Prediction, Bottlenecks, Inspector)
Resource Control/Scheduling
(Promise, Resource Scheduler (RS))
Model and Policy Driven Control
(Copper, Movie, Models, Domino, Parser)
Network Control
Network Policy
(FastDataStacks, Copper, PolicyTest)
NFV services (service chaining, VPN, ..)
(VNFFG, SFC, SDNVPN)
SDN Controller Performance Test
(Cperf)
Infrastructure
Operating Systems, virt. Infra
Hypervisor for NFV
(KVM4NFV)
IO Abstraction & Feature Path
Dataplane Performance Test
(Vsperf, Fastpath)
Dataplane Evolution & Test
(DPACC, OVSDPDK, FastDataStacks)
Hardware
Infra control & CI/CD
(Pharos, Releng, Octopus)
CI/CD
Integrate/Install
Create Scenarios/Stacks
(IPv6, FastDataStacks, Armband, Apex, Daisy, ONOSFW, OpenContrail,, Joid, Fuel, Compass)
Installation, Upgrade
(Apex, Joid, Fuel, Compass, Daisy,
Escalator, Genesis)
Documentation
(OPNFVdocs)
System Test & Security Audit
System Testing
(Yardstick, FuncTest, Qtip)
Storage Performance Test
(Storperf)
Security
(Moon)

24. FastDataStacks Development Areas: Evolving “Upstream”
Neutron
Neutron NorthBound
GBP Neutron Mapper
Topology Mgr vBD
Honeycomb (Dataplane Agent)
VPP
REST
Netconf/YANG
...
VPP renderer
DPDK
GBP Renderer Manager
System Install
System Test
OpenDaylight
GBP Neutron Mapper
GBP Renderer Manager enhancements
VPP Renderer
Virtual Bridge Domain Mgr / Topology Manager
FD.io
HoneyComb – Enhancements
VPP – Enhancements
CSIT – VPP component tests
OPNFV
Overall System Composition – Integration into CI/CD
Installer: Integration of VPP into APEX
System Test: FuncTest and Yardstick system test application to FDS
See also: FDS Architecture:

25. Project Lifecycle

26. OPNFV Project LifeCycle
Proposal
Project doesn’t really exist yet, may not have real resources, but is proposed and is expected to be created due to needs.
Incubation
Project has resources, but is recognized to be nascent.
Mature
Project is fully functioning and stable, has achieved successful releases.
Integration
Project provides value to and receives interest from a broad audience.
Archived
Project has been recognized as dead (could be for a variety of reasons, e.g. project successfully accomplished its goals, project failed, etc.), and has been archived as it's no longer a going concern.
Integration
Integration Review
Mature
Graduation Review
Incubation
Creation Review
Proposal

27. Participating in an OPNFV Project...
Find approved OPNFV projects at
Plus project proposals can be found at
Contact the project team members (e.g. the project lead) or attend project meetings if you are interested in contributing to a project.
You can find relevant project information under the “key project facts” section

28. Creating a new project is simple...
Create a proposal
Name of the project is appropriate (no trademark issues etc.); Proposed repository name is all lower-case without any special characters
Project contact name and
Description is complete
Scope and project plan is well defined
Reference of an existing NFV requirement or clear identification of a new requirement
Resources committed, Contributors identified
Initial list of committer identified (elected/proposed by initial contributors)
Meets BoD policy (IPR)
Proposal has been socialized with potentially interested or affected projects and/or parties
In the case where a project will require changes in other projects, those projects are listed in the proposal, and a sponsoring developer in the project has been identified
Post it and allow for a 2 week review period
Proposal to TSC mailing list
Review by TSC: Confirm that the proposal is complete and the above listed requirements are met.
Simple majority approval by voting TSC members

29. OPNFV Inaugural Plugfest

30. Plugfest Overview Hosted by Cable Labs
41 participants from 17 organizations (including 3 non-members)
Local pods (Huawei, Intel) and Pharos labs (worldwide)
Data collection through Functest, Yardstick and StorPerf
Many existing scenarios on new hardware with all installer groups (Apex, Compass, Fuel, JOID)
Many NFV use cases and VNF lifecycle management (vIMS, vEPC, Others)
Upstream enhancements planned and recommended
Report available at
Example: Bug in XXXX found through yardstick ping test

31. All Credit to the Guilty Parties!

32. Virtual Evolved Packet Core
Collaboration between SP and hardware, embedded system and software vendors.
The virtualization server platform and OPNFV Brahmaputra (Apex) are running on x86 Servers as NFVI
Security and application gateways, as well as complete vEPC system, are running on both NFVI platforms as VNFs
Mobile phone connected to femtocell base station
This demo provided by China Mobile, Intel, Wind River, Raisecom, and Red Hat.
The Titanium Server from Wind River and OPNFV Brahmaputra (Apex) are running on Intel Servers as NFVI
Raisecom VNF (vSecurityGW, vApplication GW, vEPC) are running on Titanium Server and OPNFV Brahmaputra as VNFs
Mobile phone connected to Nanocell (Raisecom) can access the Internet.

33. What Went Well, Thoughts for Next Time
Break out sessions: Functest, Yardstick, StorPerf
SDN controller integration with multiple installers
Daily Face to face meet-ups to recap progress and plans
On-site hardware, dedicated lab space, meeting rooms
Thoughts for Next Time
More on-site hardware resources, 24x7 lab availability
Divide into two parts: (1) install tools, testing; (2) VNF Interop
Generally, more end-to-end participation from VNF vendors
Note: Plugfest Session at Monday’s Design Summit

34. Create.Compose.Deploy.Test.
Iterate.

35. References OPNFV: https://www.opnfv.org/
OPNFV wiki:
OPNFV Brahmaputra release:
OPNFV stats:
Mailing lists: