| Status | Choose One: DRAFT |
|---|---|
| Submitter | Steven Wright |
| Contributors | Orange |
| Proposed Release | Casablanca |
| JIRA Ticket(s) | VNFRQTS-242 - Getting issue details... STATUS |
Objectives
Badging for ONAP must fit within the policy constrains of the LFN C&V committee. The currently inforce C&V policies, refer to lab qualification procedures. The numner/scope of individual badging efforts is controlled by the LFN projects' (in our case ONAP's) TSC. This committee seems to be converging on leveraging the OPNFV Dovetail framework for aggregating testing results.
The objective is to extract / summarize VNF Requirements into a VNF Badge. The VNF Requirements were original developed as prototype RFP text. Not all of the Requirements are testable ( either currently or in the near future).ONAP VNF Requirements evolve with ONAP releases. ( See e.g. Casablanca release notes ). The test implementations are also expected to evolve with ONAP releases. The VVP project is implementing HEAT related test scripts. The VNFSDK project is implementing TOSCA based test scripts. Test scripts may be invoked in a variety of different use cases. The proportion of testable requirements having tests implemented is expected to increase with each ONAP release. The scope of testing is expected to evolve from onboarding (Package inspection), to Life Cycle operations and Functional testing. In Casablanca test descriptions are available for VNF Package inspection tests. Test Case Descriptions for Life Cycle operations may be candidate work items for future releases.
The specific badge would need to be aligned with the LFN guidance on logos, but something along the following lines may be a place to start.
There is some discretion to adjust portions of the logo to reflect the scope of the assurance proposed by the badge. This example proposes a scope around "VNF Management" but other scopes may be possible. More than one badge may be possible if the material is sufficiently independent. The "2018.11" reflects the ONAP release which provides the conformance reference for the badge. VNF Badging should avoid overlap/ duplication with VNF validation based on testing, ideally the two should be linked.
One approach is to use a format similar to the CII Badging website see e.g. CII Badge for VVP. There are some differences for VNF badging vs CII badging ( beyond the obvious subject matter). This would allow human entered data to be supplemented by automated test results (e.g. from the Dovetail framework).
VNF Requirements Analysis
The Casablanca VNF Requirements are available for download from the docs.onap.org website.
The breakdown of requirements by keyword is
| Row Labels | Count of ID |
| MAY | 73 |
| MAY NOT | 1 |
| MUST | 553 |
| MUST NOT | 81 |
| SHOULD | 78 |
| SHOULD NOT | 3 |
| Grand Total | 789 |
Scoring with simple counts of Met/Unmet will not be sufficient:
R-89571 provides for alternatives (one or more of) : Netconf/ Chef/ Ansible
Section 7.4 has MAY or SHOULD requirements which should not have the same weight as a MUST requirement.
The breakdown of requirements by target is
| Row Labels | Count of ID |
| PNF | 19 |
| VNF | 559 |
| VNF PACKAGE | 1 |
| XNF | 204 |
| XNF PROVIDER | 4 |
| (blank) | 2 |
| Grand Total | 789 |
VNF Badge Scoping Alternatives
As an initial target, it has been proposed to focus on the requirements associated with VNF Management (i.e. Chapter 7 of the ONAP VNF Requirements) . This Chapter 7 represents ~190 of the >750 requirements ie ~25% of the VNF requirements identified.
The current CII badge has ~70 requirements, in 6 pulldown menus. The max requirements in a pulldown is 16.
For usability, we should constrain the number of pulldowns and the number of requirements per pulldown to a similar order of magnitude e.g. pulldowns < 10, reqts/pulldown <20
VNF Package Inspection
VNF Test Case Descriptions are provided for VNF Package Inspection from Bejing.
Of the ~192 requirements in section 7 about 25 are identified in the Test Description Annex as being inspectable in the VNF package delivered from the VNF provider to the ONAP operator.
Ch 7 includes a number of requirements on VNF Package or xNF Package that are not included in that Test Description Annex (yet) but are presumably inspectable in the Package at some point even if the inspection tests are not yet available in VVP or VNFSDK as part of the Casablanca release. VNFRQTS-241 - Getting issue details... STATUS provides for updates to this document as part of the Casablanca release.
Eliminating the requirements that are inspectable in the xNF Package would reduce the number of requirements in the table above - ~30 of the 190 requirements are on the xNF or VNF Package ie ~15%
ONAP supports several different VNF Package formats for onboarding : HEAT based, TOSCA Based (VFC:Generic VNFM, VFC:SVNFM) and (Proposed for Dublin: Helm based ( see Req #1)). In Casablanca the number of Requirements & tests for these formats is:
| VNF Package format | No. Requirements | No. Tests |
|---|---|---|
| HEAT | 298 (List) | 180 (List) |
| TOSCA (G/S VNFM) | 11 (List) | 6 (List) |
| HELM | 0 | 0 |
The VNF Package as a handoff between the VNF Provider and the ONAP Operator includes a variety of documentation about the VNF. This documentation is required by the Network operators staff to correctly use the VNF. It is not used directly by the ONAP platform in any automated fashion. Testing for documentation is problematic - at best, automated testing can detect the presence of some artifact, but not validate the "correctness" of the documentation. There is some overlap between VNF "package" requirements and VNF "documentation" requirements.
VNF Life Cycle Operations
VNF Test Case Descriptions for VNF Life Cycle operations are not available in Casablanca.
Developing new test case descriptions for Life Cycle operations is not trivial. The set of life cycle commands supported by the platform is expected to expand in future releases.
Life cycle operations require running instances of the VNF. The resource requirements of ONAP platform may make the use of the ONAP platform problematic as a part of the test case description, in which case some process may be required to maintain synchronization between the test cases and the platform command (API) evolution.
VNF Functional Requirements
TBD
VNF Performance Requirements
TBD
Further details
Some Analysis of candidates for VNF Badging are discussed below:
- VNF Badging vs CII Badging
- VNF Documentation Requirements
- VNF HEAT Requirements
- VNF Life Cycle Operations Requirements
- VNF Management Badge
- VNF Package Requirements
- VNF Requirements by Section Heading
- VNF Security Badge
- VNF TOSCA Requirements
6 Comments
Brian Hedstrom
A few items that might be added to this page (or include links to relevant info):
Steven Wright
Most of these points are still open for discussion as to what the ONAP community wants to achieve.
The LFN C&V committee sets general guidelines across multiple LF projects (including ONAP), but does not define the specific badge criteria and its implications - These are deferred to the respective TSCs for approval, and in general to the various projects below the TSC for formulation so the TSC has something concrete to approve.
VNFRQTS project manages the set of VNF requirements that would be the basis for any VNF certification program. VNFSDK manages tests for TOSCA based VNFs, VVP manages tests for HEAT based VNFs. VNFSDK and VVP also have tooling to invoke the test scripts.
Eric Debeau
The list of requirements include some requirements that can not be fulfilled by a single VNF.
For example, we have requirements for configuration
The requirements should take into account that a VNF is using one configuration mode
Amongst the 789 requirements, some are dedicated to PNF. Do we apply badging only for VNF ? Not for PNF ?
I believe that we decided at LFN level to focus first on packaging (Heat or TOSCA). The VNF must be 100% compatible with the associated MUST and MUST NOT requirements associated with tests. We should decide for SHOULD and SHOULD NOT.
Steven Wright
R-89571 provides for alternatives ansible/chef/netconf. There may be requirements on how to support those options that require further clarification in their wording to make that clear when the individual requirements are read stand - alone. Suggestions to improve the wording would be welcome as part of the Dublin work plan.
Steven Wright
There is considerable overlap between the requirements for VNFs and PNFs. In general common requirements are structured as "XNF" requirements. See the table above - about 200 of the requirements are identified as "XNF" requirements. The PNF work is expected to continue to evolve in Dublin timeframe - see PNF PnP and PNF S/W UPGRADE. because there is ongoing work expected to add a significant number of new PNF requirements, badging/ certification of PNFs should be deferred until this is completed. Though discussion of general processes and intentions for PNF certification may be helpful preparation.
Steven Wright
The LFN and OPNFV approaches envisage a bar that gets raised over time. One approach would be to start with the 634 MUST/MUST NOT requirements ( out of the 789 total) but this includes run time as well as package inspection requirements. Additional analysis of the scope of VNF package requirements is provided here. In general we have to be concerned with a couple of factors as the overall coverage increases over time...
1) scope - how many of the total VNF requirement base are considered for inclusion?
2) test coverage - how many of these requirements have tests implemented?