Common NFVI Telco Taskforce Paris Face-To-Face Sessions Compliance & Verification Heather, Kirksey LFN Rabi Abdel, Vodafone Group; July 2019

LFN CVC | An introduction Committee Charter: The Compliance and Verification Committee (“C&V Committee”) is a members-driven committee within LF Networking that recommends policies and oversight for compliance and certification program to the Governing Board of LF Networking (“Governing Board”). C&V Committee operates under the direction of the Governing Board and its recommendations are subject to approval by the Governing Board. The C&V Committee will meet by conference call, online conference or in face-to-face sessions. Participation in the C&V Committee is open to members of LF Networking, and the target makeup of the C&V Committee is to include a balance of representatives with active testing Alignment of testing and certification programs between projects (ONAP, OPNFV, etc.) Common governance and program designs Program structures, marketing, oversight, legal, laboratory, and escalation procedures

OVP: Compliance & Verification Program Accelerating Deployment – Reduce Operator & Supplier Integration/Interop Testing Intervals By 50% Operator Services on Boarding Operator Services on Boarding VNF SDK and VVP VNF VNF VNF VNF VNF VNF VNF VNF NFVO NFVO NFVO NFVO NFVO VIM VIM VIM VIM VIM VIM From: Too many separate solutions for VNF and NFVi lifecycle management and onboarding To: Simplified & VNF on-boarding, Reduced time and cost of operationalizing a VNF through a common NFV Framework* NFVI NFVI NFVI NFVI NFVI - Aligned

Key Benefits to Service Providers Accelerate time to deployment for new network services Improve interoperability and software quality Reduce in-house testing effort and reduce costs Key Benefits to Vendors All this can be voiceover. TTR is a benefit for both vendors and service providers. The big item in my mind is lowering OPEX and reducing cycle times for opeators. It is implied here but most operators will understand they are capital intensive and operationally intensive and as a result somewhat slow. Also there is an efficiency to be had because much of the industry validation is duplicative and/or redundant. Also, similar to the call discussion - it might be a a good place to say as an industry, we did a fairly good and ever improving job of open specifications and open source, we need to also address open validation. Improve time to revenue for new product offerings Achieve greater alignment with service provider customer requirements Demonstrate product quality through open ecosystem testing

Vision – End to End Lifecycle System Validation Operator Specific Req VNFRQTS Standards / Requirement Validated Common NFVI/VIM NFVI Compliance Check NFVI Validation Check NFVI Performance Check On-Boarding Module Alignment Running ONAP Instance -- Validated VNF VNF Compliance Tests VNF Validation Tests VNF Performance Tests Cross-Platform On-Boarding VNF Deploy Life Cycle Mgmt FCAPS Interoperability

Standards / Requirements Unified VNF/NFVI Certification CNTT | CVC Vision Keeping it simple ! Operator Specific Req VNFRQTS Standards / Requirements Compliance Validation Certified NFVI/VIM Report Unified VNF/NFVI Certification Repository/Portal VNFs √ Compliance A X Compliance B √ Validation A X Validation B √ Performance A X Performance B NFVI Test Tools Approval Process Test repository Performance Interoperability

CNTT | CVC/OVP Scope Approval Process Publish Result Standards Performance Compliance Interoperability Validation Testing Tools Operator Approved Lab Standards / Requirements VNF Compliance Tests VNFs NFVI NFVI Compliance Tests VNF Validation Tests VNF Performance Tests Operator’s specific tests/Third party tests VNF/NFVI Vendor VNFRQTS Operator Specific Req Request Certification Ad-hoc Testing Query Result Self Certification Publish Result VNF / NFVI Compliance Testing Tools Test repository VNF Compliance Tests VNFs NFVI NFVI Compliance Tests Submit Result Compliance VNF /NFVI Validation Testing Tools Test repository VNF Validation Tests VNFs NFVI NFVI Validation Tests Submit Result Validation VNF/NFVI Performance Testing Tools Test repository VNF Performance Tests VNFs NFVI NFVI Performance Tests Submit Result Performance Interoperability Testing Tools Test repository Interoperability Tests Interoperability Certified NFVI/VIM Report Unified Certification Repository Portal VNFs √ Compliance A X Compliance B √ Validation A X Validation B √ Performance A X Performance B NFVI Approval Process Publish Result

OVP Terminology Term Meaning Compliance Testing Testing a product against a set of requirements as outlined in a specification or standard (e.g., MUST, SHOULD, MUST NOT) statements Verification Testing Testing a product to validate it behaves as expected in real-life type conditions (e.g., testing failover for an NFVI/VIM or testing on-boarding and lifecycle management actions) Performance Testing Testing a product’s performance in defined environments; Note that this will depend on specifics of operator environments, system tuning, etc. System Under Test The commercial product being tested; e.g., VNF, NFVI/VIM, etc. Interoperability Testing that implementations from different vendors work with each other and that the end-to-end system works as desired.

CNTT | Relation between CNTT & CVC/OPV Interoperability Compliance Validation Performance Test Tools Test repository Reference Model RM Profile B Profile N Tests Profile C Reference Architecture RA Reference Implementations RI

CNTT | Integration with OVP vision Reference Architecture(s) RA Reference Architecture(s) RA Profile B Ref Impl R VNF A Ref Impl R Reference Implementations of Common NFVI Profiles are needed. Profile N Ref Impl R VNF B Ref Impl R Basic Virtual Compute, Network, and Storage Over-subscription HA Tests & Tools Profile C Ref Impl R VNF C Ref Impl R Network Intensive SDN controller CPU Pinning Virtual Compute, Network, Acceleration, and Storage Zero over-subscription HA Test Tools Compute Intensive Monitoring NVMe/SSD Virtual Compute, Network, GPU/FPGA, and Storage Zero over-subscription HA CVC VNF Test repository Industry Standard Certification Program for VNFs and NFVIs Reference Model RM VNF Compliance Tests VNFs NFVI NFVI Compliance Tests VNF Validation Tests VNF Performance Tests Test Scripts

Current OVP Toolchain Web Portal (Public view, Vendor profiles, Governance Workflow) Dovetail instantiation configuration test case execution result collection Test Tools Functest Yardstick Bottlenecks VNFSDK VVP API tests OS level HA tests VNF validation System under test: VIM + NFVI controller nodes, compute nodes, SDN controller SuT: VNF Heat + Tosca 11/13/201911/13/2019

Verified Lab Program Work with third party labs that have passed the rigorous badging program to indicate expertise in open source software and LFN projects in particular Lab requirements include CI/CD knowledge, community participation and commitment, ability to run the test tool suite Working with a Verified Lab can improve troubleshooting capabilities and accelerate a company’s ability to pass compliance testing.

CNTT | OVP - Expectation from VNF Vendors I have a VNF, I would like to certify it, what should I do? VNF Control VNF-C DP S B M N L Instance type Compute flavour Approved Lab VNF Test Tools CVC Test repository VNF Vendor Tests Reference Implementation Profile C Profile N Profile B

CNTT | OVP - Expectation from NFVI Vendors I have a NFVI solution, I would like to certify it, what should I do? Approved Lab VNF A Ref Impl R NFVI Test Tools CVC Test repository Tests NFVI Vendor NFVI Implementation Profile C Profile N Profile B Vendor A