You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 22 Next »

Motivation

The most obvious solution for taking full advantage of CSIT is to add the test cases under the same repository with the functionalities that they are testing (instead of having them in separate centralized CSIT repository as we currently have). This would have the following benefits:

  • CSIT could be triggered by any commit to the project repo
  • CSIT tests the code (or specifically, docker images that have been built) from the committed branch
  • CSIT could have a vote on the commit based on the result of the test run
  • If the implementation changes require changes in CSIT tests (to cover new functionality or to pass in the first place), that could be handled within the same commit
  • ideally, local verification would become less complex (no need to work between CSIT repo and project repo) 

Issues

Given the fact that CSITs are currently a very colourful collection of various suites with different scopes and strategies, the transition of CSITs to project repositories is not necessarily trivial:

  • CSIT suites that test components from multiple project repositories at the same time 
    • such CSIT tests may have to be separated using additional simulators, or
    • project repository structures themselves may have to be reconsidered
  • Jenkins templates may have to be redesigned to support unified approach for triggering review branch-specific docker build, CSIT execution and voting chain as part of review verification
  • CSITs would become blockers for merging code
    • local pre-commit verification should be supported better by common CSIT tools
    • are all projects and suites mature enough to deal with that?

Technical details to be decided

  • Should we keep separate docker build and CSIT jobs and just chain them into review verification, or should we try to incorporate docker building and CSIT execution into existing review jobs?
    • Reusing existing jobs and chaining them would require some docker image tag tuning to make sure CSIT tests pick up the exact image that was produced by preceding docker build job 
    • Either way, JJB templates will probably have to be touched
  • Should we still have common CSIT scripts (run-csit.sh etc) in CSIT repo and related procedures (setup, tests, teardown and related result collection) as the basis of project-specific test execution? 
  • Execution of CSIT tests and incorporating locally built test images should be made as easy as possible following common guidelines
    • Setting up testing environment (specific project-specific dependencies should be handled by the setup scripts)
    • Specific environment variables expected by the test suite (like GERRIT_BRANCH)

Project status and readiness at the end of Guilin

Projects with CSITCSIT jobs CSIT job triggersTested docker images and repositoriesDocker build jobsDependenciesComments
aaf

aaf-frankfurt-csit-certservice

Frankfurt branch only? The tests triggered from here do not even exist in master (nor are there any jobs for still remaining aafapi and sms-test-plan?)

nexus3.onap.org:10001/onap/ org.onap.aaf.certservice.aaf-certservice-api:frankfurt-latest

(hardcoded)

The source repository is aaf/certservice

aaf-certservice-maven-docker-stage-frankfurt

(note that aaf-certservice-release-verify and aaf-certservice-release-merge also build aaf-certservice-api image, but not with frankfurt-latest tag)

primekey/ejbca-ce:6.15.2.5 (hardcoded, real image used)


How relevant are any of these in Honolulu?
appcappc-master-csit-healthcheck

Daily timer

nexus3.onap.org:10001/onap/appc-image:1.7.2-SNAPSHOT-latest

nexus3.onap.org:10001/onap/appc-cdt-image:1.7.2-SNAPSHOT-latest

(hardcoded)

docker-compose.yml as well as appc-image and appc-cdt-image come from appc/deployment


No job is producing these images at the moment! 

appc-deployment-master-docker-java-daily produces 1.8.0-SNAPSHOT-latest and 

appc-deployment-frankfurt-docker-java-daily is producing 1.7.3-SNAPSHOT-latest

1.7.2-SNAPSHOT-latest images have not been created since May 27


mariadb:10.4.3

nexus3.onap.org:10001/onap/ccsdk-dgbuilder-image:0.6.0 (no longer produced, last update to Nexus on Aug 30 2019)

nexus3.onap.org:10001/onap/ccsdk-ansible-server-image:0.4.4 (no longer produced, last update to Nexus on Jun 06 2019)

(hardcoded, real images used)

Note that APPC is in maintenance mode and its CSIT is to be disabled: https://gerrit.onap.org/r/c/ci-management/+/113561

The tested images are obsolete Frankfurt versions, but the dependencies are even more ancient? Is it any use to update tests to use 1.8.0 if the CCSDK dependencies are not up to date? 

ccsdk (disabled)




See

CCSDK-2850 - Getting issue details... STATUS

CCSDK-2851 - Getting issue details... STATUS

CCSDK-2856 - Getting issue details... STATUS


ccsdk-oran

ccsdk-oran-master-csit-polmansuite

ccsdk-oran-guilin-csit-polmansuite

  • nexus3.onap.org:10003/onap/ccsdk-oran-a1policymanagementservice (1.1.0-SNAPSHOT for master, 1.0.1-SNAPSHOT for Guilin)

The source repository is ccsdk/oran

ccsdk-oran/job/ccsdk-oran-maven-merge-master and ccsdk-oran-maven-merge-guilin produce the tested images respectively (note that these are different images that the STAGING-tagged ones from triggering jobs!)

nexus3.onap.org:10003/onap/sdnc-image (2.1.0-STAGING-latest for master, nexus3.onap.org:10003/onap/sdnc-image:2.0.1-STAGING-latest for Guilin)

nexus3.o-ran-sc.org:10004/o-ran-sc/nonrtric-controlpanel:2.0.0
mysql/mysql-server:5.6
nexus3.o-ran-sc.org:10004/o-ran-sc/a1-simulator:2.0.0
consul:1.7.2
nexus3.onap.org:10001/onap/org.onap.dcaegen2.platform.configbinding.app-app:2.3.0
mrstub:latest
callback-receiver:latest

These tests are currently cloned from ORAN repository and they are essentially plain shell scripts that are just wrapped into Robot

In Honolulu the test cases should be brought under ONAP repository and rewritten with Robot and ONAP CSIT scripts as much as possible

clamp

clamp-master-csit-APIs

clamp-master-csit-UIs

Daily timer

nexus3.onap.org:10001/onap/clamp-backend:4.1-STAGING-latest

nexus3.onap.org:10001/onap/clamp-backend:4.1-STAGING-latest

(hardcoded)

The docker-compose.yml and source of the images is clamp

clamp-maven-docker-stage-elalto

mariadb:10.5.4

python:2-slim

The versions are a mess; master docker builds are producing 5.1 versions, frankfurt branch 5.0 versions, and el-alto 4.1 versions. However, CSIT master and frankfurt branches are testing versions 4.1 and CSIT does not even have el-alto branch?
dcaegen2

dcaegen2-services-son-handler-master-csit-testsuites

dcaegen2-services-pmsh-master-csit-testsuite

dcaegen2-services-bbs-event-processor-master-csit-bbs-testsuites

dcaegen2-pmmapper-master-csit-pmmapper

dcaegen2-master-csit-testsuites

dcaegen2-master-csit-prh-testsuites

dcaegen2-collectors-restconf-master-csit-testsuites

dcaegen2-collectors-hv-ves-master-csit-testsuites

dcaegen2-collectors-datafile-master-csit-ManagementInterface-suite

dcaegen2-collectors-datafile-master-csit-Functional-suite

dcaegen2-services-son-handler-master-merge-java

dcaegen2-services-pmsh-docker-merge-master

dcaegen2-services-master-bbs-event-processor-merge-java

(pmmapper has no trigger job)

dcaegen2-collectors-ves-master-merge-java

dcaegen2-services-prh-master-merge-java

dcaegen2-collectors-restconf-master-merge-java

dcaegen2-collectors-hv-ves-master-merge-java

(datafile CSITs have no trigger jobs)

nexus3.onap.org:10003/onap/org.onap.dcaegen2.services.son-handler:latest

nexus3.onap.org:10001/onap/org.onap.dcaegen2.services.pmsh:latest

nexus3.onap.org:10001/onap/org.onap.dcaegen2.services.components.bbs-event-processor:latest

nexus3.onap.org:10001/onap/org.onap.dcaegen2.services.pm-mapper:latest

nexus3.onap.org:10003/onap/org.onap.dcaegen2.collectors.ves.vescollector:latest

nexus3.onap.org:10001/onap/org.onap.dcaegen2.services.prh.prh-app-server:latest

nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.restconfcollector:latest

nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.hv-ves.hv-collector-main:latest

nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.datafile.datafile-app-server:latest

(all hardcoded in various places in the tests)

Some of the triggering jobs produce the tested images, others do not produce any  image; requires a deeper lookA lot of simulators used, requires deeper analysis
dmaap

dmaap-buscontroller-master-csit-ssl

dmaap-buscontroller-master-csit-with_dr

dmaap-buscontroller-master-csit-with_mr

dmaap-datarouter-master-csit-ssl-dr-suite

dmaap-master-csit-mrpubsub


dmaap-datarouter-maven-docker-stage-master

(other jobs have no triggers)





multicloud





music





oom-platform-cert-service





optf





policy





sdc (special case)





sdnc





so





usecases (disabled?)





vfc





vid





vfsdk






  • No labels