Integration tests in target deployments using OOM
CSIT tests currently running in Jenkins are misleading. They are not CSIT tests because they are not executed in the target deployment environment (kubernetes deployed by OOM).
They are functional tests, which is important and mandatory from a project perspective, executed in a mocked environement most of the time based on docker composed.
It was historically setup by integration team to provide an execution environement to the projects to beyond the unit/functionnal maven tests.
It is still fully conveient as functional tests but not acurate for integration tests.
Integration tests must be done on the target environment.
It can be realized by an adapation of the gating: OOM Gating
This gating has been setup in February 2019, initially for OOM then extended to SO and suggested to other projects.
Compared to the existing functional test, we need to adapt it in order to reduce the footprint to what is necessary for the testing. There is no need of a full ONAP for all the integration tests, simulators may also be needed.
The gating must then deal with the installlation of ONAP using OOM but limiting the components to what is really needed.
As for SO, it shall also include a build chain to test the patchset.
Improve robot healthcheck across projects
Robot healthcheck tests are very important to get the status of ONAP components.
Today the level of maturity of the tests is not really homogeneous.
Some projects already reached a good level of quality and coverage. But it is not the case for every projects.
In theory the healthcheck should clearly indicate if the component is running well or not.
It is not acceptable to get green status when the project is crashing, some of the pods are not running, ...
A new feature must systematically include a new test in the healthcheck.
Healthcheck tests are living tests and shall be updated by the teams for each feature/version.
As an illustration any new GUI shall come with its associated tests
Integration team initiated an overview for the main components. The work is in progress, it is just a draft but a place to collect rationales and initiate discussions for future releases: Healthcheck tests evaluation
Anyone is welcome to help completing.
More projects added to CI pipeline and more project-specific testing
Lots of use cases have been created since Amsterdam version. Only a few have been automated in CI.
The goal is to improve the list of tests integrated in CI to consolidate the trust to the system.
In Frankfurt, several categories have been defined by the community: Integration categories & testsuites discussions
Several "old" tests are already candidates.
Moreover a stronger automation culture is needed to consider automation when possible at the beginning of the tests.
It also means that better synergies shall be found on the reusability of simulators, traffic generators,..