...
The testsuites usually include several tests(e.g. onap_vnf include basic_ubunutu, freearius_nbi and clearwater_vims, security includes 6 tests). Tests can be run sequentially or in parallel.
To be integrated in CI, you must integrate your tests in of of the defined test categories (see Integration categories & testsuites discussions):
- infrastructure-healthcheck
- healthcheck
- smoke-usecases
- candidate-usecases
- security
- benchmark
The testing part
1) Your tests must be fully automated...
Your tests must be fully automated. It means that it shall take into account the env on which it is executed, setup the resources, run the tests, delete the resources created for the test
if you are using non open source VNF, proprietary third components/equipments you will not be able to fully automate.
...
You can develop your test using any language (python, bash, robotframework, go..) and any framework.
Ideally use case owner shall be able to provide the automatic procedure at this stage.
Please note that this part is the most important and represents about 95% of the work...writing good tests requires a good knowledge of the system under test.
2) ...and integrated in a docker based on Xtesting
...
As any framework is allowed, at the end we may have a huge diversity of tests, test frameworks, test artifacts. In order to simplify the integration of the tests in any CI chains, we decided to leverage the OPNFV xtesting framexork.
Whatever your tests, they will be launched always the same way and generate consistent artifacts. As they will be dockerized, they will be portable in any CI/CD system.
Your test suites must then However your test suites must be "xtestingized", it means they must be embedded in a docker file leveraging Xtesting.
This task can be done by the integration team - the most difficult is the test itself... If writing the test takes 90% of the the time, building a xtesting docker is about 4%.
Xtesting is a light framework aiming to harmonize test inputs and outputs, which is very helpful for integration.It has been developped in OPNFV and already proposed several infrastructure test dockers for Openstack and kubernetes, leveraging upstream tests (https://wiki.opnfv.org/pages/viewpage.action?pageId=13211751).
...
*: note that Xtesting is open source so if your favorite framework is missing you can add it
The CI part
This last part represent the missing 1%, it mainly deals with declaration of the case in the DB and in the CI system.
1) Your tests must be declared in the DB...
for that you need to have access to the common test DB used by OPNFV and ONAP: testresults.opnfv.org
An account onap-integration has been created. Public SSH keys of the contributors have been pushed to grant access to this machine.
So to log in use
| Code Block | ||
|---|---|---|
| ||
$ ssh -i <key path> onap-integration@testresults.opnfv.org |
Once log you can directly access to the database
...
it is obviously possible to update, delete using usual mongo commands.
2) ... and in the CI
Once everything is declared in the DB, you can add the test in the CI chain.
...