2. Bringing Dynamism to OPNFV
Fatih Degirmenci, Jose Lausuch, Tim Rozet, Uli Kleber

3. Intro to Continuous Integration
OPNFV uses Continuous Integration (CI) for automatic build, deploy and testing.
Patches are automatically verified and periodic full deployments and tests provide fast feedback
Results are publicly visible, so the community can react
Community labs provide the resources
BUT: The nature of OPNFV brings additional challenges
Dependency on upstream projects (see XCI, presentation earlier today)
Support of multiple combinations of upstream components – so called scenarios

4. OPNFV‘s CI/CD principles and challenges
Need to continuously deploy using multiple installers and scenarios
5 installers, 64 scenarios (Danube), 3-5 hours to deploy and test each scenario, daily jobs over 40 hours of runtime
Resource dependencies
Installer specific environments need to be prepared in the labs/PODs
Current CI has fixed resource assignments per installers
2 CI PODs per installer
Limited execution time for installers that support high number of scenarios
Each OPNFV installer uses it's own form of input from CI to deploy scenarios, rather than a common method.
Currently scenarios and installers need to be assigned manually to PODs
We cannot react quickly on resource shortage even when hardware is available

5. The Principle of Dynamic CI?
Goal
Prerequisites
We will explain
Allow CI to assign lab resources dynamically
Any installer, any scenario, any option, any POD
Step 1: more flexibility for PODs, (see presentation at 2:20pm)
Step 2: full dynamic assignments
Introduction of common configuration files for PODs and Scenarios
All stakeholder use same file formats
Lab owners, Scenario owners
CI tools
Installers
Contents of POD Descriptor File (PDF) and Scenario Descriptor File (SDF)
Usage overview (workflows) for CI, installers, developers, testing and lifecycle

7. Users of PDF

8. Contents of PDF pod.yaml Metadata Hardware information per node
Labowner
Location
Hardware information per node
Cpu
Disks
OS (jumphost)
Remote management
Network Interfaces
Network.yaml
(Common for all PODs in a lab)
Metadata
Labowner
Location
Common Network Info
IP-address ranges
Subnets
Vlan configurations/tags

11. Contents of SDF Metadata Components Deployment Options
Name
History
Purpose
Owner
Components
e.g. SDN controllers
Versions
Optional features, e.g. NFV features
Deployment Options
Hardwaretypes
Virtual deploy
HA, NOHA
Deployment Tools
Supporting installers
Valid options per installer
Hardware Prerequisites
e.g. SRIOV, DPDK

12. CI using PDF and SDF for decision making
CI knows valid combinations of scenarios, options, installers
Use information from SDF
All valid combination need to be deployed according to certain rules
Some tests require a specific installer, some are flexible
Release testing requires all combinations to be tested
CI can select dynamically a free POD for a deployment
Use information from PDF (check hardware prerequisites)
Jenkins will distribute the load between available PODs

13. Installer consuming PDF and SDF for deployment
Get upstream component list from SDF
Get features for upstream components from SDF
Get deployment options from SDF
Get hardware and network details from PDF
Prepare deployment steps for each role
Customize steps with above information
Generate mapping on nodes and networks
Execute necessary additional network configurations on POD
Execute deployment
Generate summary file:
List of deployed components with exact versions including dependencies
Generated networks, usedids&passwords
Trigger test framework

14. Dynamic POD and Scenario Allocation and Testing
Type of test cases:
common to all scenarios and installers (e.g. vPing)
supported by certain installers only (e.g. security scan)
specific to certain capabilities and installed features (e.g SFC)
Test frameworks need to know the deployment options to trigger
the appropriate test cases.

16. Introduction Strategy
Creating PDF files in Euphrates Release
PDF Converter tool to minimize installer efforts
Preparing SDF template and workflows during Euphrates
Phase 1 of Dynamic CI (PDF) during Euphrates
Work according to Scenario Lifecycle in F-Release
Phase 2 of Dynamic CI (SDF) during F-Release

17. Summary and Outlook Benefits of Dynamic CI concept
Better resource usage
Easier to add more resources for CI
PDF/SDF are important for us to have single source of truth (instead of information being distributed over gerrit, wiki, dashboard, etc.)
Everybody (humans and machines) can consume the same source information so there will be less errors
Outlook: End users can use SDF and PDF
In future, end users will be able to use SDF and PDF including the CI tools in their own environment
End users will use PDF to trigger deployment on their own POD
End users can use SDF to create customized scenarios

18. Thank You https://wiki.opnfv.org/display/INF/Continuous+Integration
PDF Template:
SDF Template (in review):
See also presentation on
Improving POD Usage in Labs, CI and Testing, 2:20 pm
Scenarios on Thursday, 3:20 – 3:50 pm