Objectives

  • As part of LFN Anuket Certification Program, ONAP wants test platform where VNF packages could be certified using ONAP requirements to drive industry adoption
  • Provide an platform where vendor/operator can develop, deploy, run test cases and query the results
  • Test cases, test results and VNF should be manageable .i,e with authorization, so only user with given roles is allowed to perform operation like
    VNF package upload/download, run compliance verification tests, allow only specific VIM for specific users, etc.
  • Test results should be persisted and should be available for human analysis later via LFN infrastructure.
  • Provides test flow where author make flow across different test cases for a given program like compliance verification and  VNFREQS/SOL0004.
  • Provide integration with OPNFV dovetail to run test cases across dovetail and VNFSDK.
  • Deployable as docker container.

Overview

A vert generic test platform for perfoming various VNF testing and could be used at different stage such as CI/CD, LFN OVP certification, onboarding, design and active&passive testing. 

Usecases

It has been used for following usecases:

  1. VSP compliance check (on-boarding phase)
  2. VNF validation (certification phase)
  3. VNF OVP certifications (certification phase)

  4. Test flow design

    1. TSC 2019-12-05

      1. https://wiki.onap.org/download/attachments/71835507/OVP%20Augment%20with%20ONAP-v1.1.pdf?version=1&modificationDate=1575595693000&api=v2

Demos

Above usecases are demoed in following ONS events

  1. ONS NA 2019
  2. ONS EU 2019

Others

Press Releases

https://www.prnewswire.com/news-releases/onap-doubles-down-on-deployments-drives-commercial-activity-across-open-source-networking-stack-with-dublin-release-300881744.html

 — In ONAP, VTP plays this role.

 — mentioned about the VNF Test Platform (VTP) as "publically available VNF compliance testing tool"

REST API

Casablanca 

"/onapapi/vnfsdk-marketplace/v1/vtp/tests": {
      "get": {
        "tags": [
          "VNF Test Platform"
        ],
        "summary": "VTP Test cases",
        "description": "Returns the list of test cases",
        "operationId": "listTests",
        "produces": [
          "application/json"
        ],
        "parameters": [],
        "responses": {
          "200": {
            "description": "successful operation",
            "schema": {
              "type": "string"
            }
          },
          "500": {
            "description": "Failed to retrieve the tests",
            "schema": {
              "type": "string"
            }
          }
        }
      }
    },
    "/onapapi/vnfsdk-marketplace/v1/vtp/tests/{testName}/run": {
      "post": {
        "tags": [
          "VNF Test Platform"
        ],
        "summary": "Run VTP testcase",
        "description": "Runs the given test case and returns the result",
        "operationId": "runTest",
        "consumes": [
          "application/json"
        ],
        "produces": [
          "application/json"
        ],
        "parameters": [
          {
            "name": "testName",
            "in": "path",
            "description": "test Name",
            "required": true,
            "type": "string"
          }
        ],
        "responses": {
          "404": {
            "description": "Test case not found",
            "schema": {
              "type": "string"
            }
          },
          "500": {
            "description": "VTP internal failure",
            "schema": {
              "type": "string"
            }
          }
        }
      }
    }

Dublin

Provides required API for qureying and executing the test cases as part of VSP Compliance check use case in dublin.

More details VTP REST API v1



  • No labels

6 Comments

  1. Trevor Lovett

    I'm trying to explore how we could integrate the vvp/validation-scripts for Heat valiation into this framework via Dovetail, but I'm not having much luck finding details on how to do that. Can someone point me to some documentation or example of how we might run our validation scripts within dovetail in a way that produces suitable output to be consumed by the framework?


  2. Morgan Richomme

    I think it would make/have made sense to consider xtesting framework for some synergies

    A presentation had been done in ONS Amsterdam: https://events.linuxfoundation.org/wp-content/uploads/2017/12/LFN-Xcommunities-Testing-Morgan-Richomme-Cedric-Ollivier-Orange.pdf

    xtesting is already used within LFN network, in OPNFV CI/CD supporting AMD and ARM for functional testing (functest project).

    It is even runnable on raspberry pi since last week (https://wiki.opnfv.org/display/functest/Run+Functest+containers+on+Rasperry+PI) (smile)

    In Orange, we have been using this framework to automatically test all our community infra/ONAP within our CI/CD chains

    xtesting is natively connected through the test API to the Mongo Test DB (same API/DB mentioned for Dovetail by Stamatis Katsaounis)


    • The outputs are consistent through the Test API/DB and can be post-processed, we got the status PASS/FAIL and lots of information on the test conditions.

    Infrastructure tests and VNF tests are launched and managed the same way, leveraging the existing API/DB.

    • The dockers are very light well and then well adapted to CI/CD (Robot healthcheck is less than 50M)
    • The integration of new tests is simple and already supporting multi languages (Bash, python, Robot).

    I did integrate VVP several months ago to test automatically all the heat VNF templates hosted in ONAP/demo using xtesting (demo done to ONAP integration team).

    you can try it if you want,  docker run morganrol/xtesting-onap-vnfpkgcheck

    you should get something like

    it means that currently none of the templates referenced in ONAP demo repository is OK from a VVP perspective (smile)

    And if I would have run with the -t option all the results would have been automatically pushed to the Mongo DB.


    If you think it makes sense to share it during a VVP, VNFSDK meeting, feel free to ask.

    I hope it would be possible to converge on ONAP side on testing framework - I also saw another proposal in this area OTF - Open Test Framework and initiated a mail thread some time ago on this question.

    Anyway it took time in OPNFV to converge from a testing group perspective - took several versions to draw the "big picture" (https://docs.opnfv.org/en/stable-gambia/_images/OPNFV_testing_working_group1.png) and try to clearly identify the role of each projects. It could be one of the goal for Dublin to draw such pictures, it would be helpful to know who is doing what? try to avoid - when possible - overlapping.


  3. Junfeng Wang

    Does this project has weekly meeting that I can listen?

    1. user-67d6f

      not really, pls let me know if you have any queries and we could schedule a meeting for the same. we could use onap-discuss group for any discussion.

  4. Zu Qiang (Ericsson)

    I understand that this project was started with VNF package testing. But now it covers not only VNF but also PNF. and it also covers not only the onboarding package testing as it was in VNFSDK project, but also LCM and more functions of a package. Proposal is to change the project name and make the scope more clear. 

    1. user-67d6f

      Hi Zu Qiang, yeah, you are right. would like to hear some proposal for naming (smile)