...
- CSIT suites that test components from multiple project repositories at the same time
- such CSIT tests may have to be separated using additional simulators, or
- project repository structures themselves may have to be reconsidered
- Jenkins templates may have to be redesigned to support unified approach for triggering review branch-specific docker build, CSIT execution and voting chain as part of review verification
- CSITs would become blockers for merging code
- local pre-commit verification should be supported better by common CSIT tools
are all projects and suites mature enough to deal with that?
Technical details to be decided
- Should we keep separate docker build and CSIT jobs and just chain them into review verification, or should we try to incorporate docker building and CSIT execution into existing review jobs?
- Reusing existing jobs and chaining them would require some docker image tag tuning to make sure CSIT tests pick up the exact image that was produced by preceding docker build job
- Either way, JJB templates will probably have to be touched
- Should we still have common CSIT scripts (run-csit.sh etc) in CSIT repo and related procedures (setup, tests, teardown and related result collection) as the basis of project-specific test execution?
- Execution of CSIT tests and incorporating locally built test images should be made as easy as possible following common guidelines
- Setting up testing environment (specific project-specific dependencies should be handled by the setup scripts)
- Specific environment variables expected by the test suite (like GERRIT_BRANCH)
Project status and readiness at the end of Guilin
- In any case, the division between the images under test and images that are just necessary dependencies should be clearly made and documented
- the images under test should be coming from the commit branch
- in the case of necessary dependencies it must be decided whether they should be provided as simulators or real components
- if provided as real components, they should be referred to with a fixed, unchanging version number and should be stable and mature enough to develop on
- ONAP's (unintentional?) practice of allowing the same versioned image to be changed might prove problematic
- if provided as real components, they should be referred to with a fixed, unchanging version number and should be stable and mature enough to develop on
- project-specific simulators are trivial case from dependency handling point of view, but if common simulators are in use, dependency considerations for them are the same as with real components
- In any case, the division between the images under test and images that are just necessary dependencies should be clearly made and documented
- Jenkins templates may have to be redesigned to support unified approach for triggering review branch-specific docker build, CSIT execution and voting chain as part of review verification
- What is the most elegant and effortless solution for ensuring that the docker images built from commit branch are taken in use in the CSIT (both in Jenkins and in local development environment)?
- CSITs would become blockers for merging code
- local pre-commit verification should be supported better by common CSIT tools
are all projects and suites mature enough to deal with that?
Technical details to be decided
- Should we keep separate docker build and CSIT jobs and just chain them into review verification, or should we try to incorporate docker building and CSIT execution into existing review jobs?
- Reusing existing jobs and chaining them would require some docker image tag tuning to make sure CSIT tests pick up the exact image that was produced by preceding docker build job
- Either way, JJB templates will probably have to be touched
- Should we still have common CSIT scripts (run-csit.sh etc) in CSIT repo and related procedures (setup, tests, teardown and related result collection) as the basis of project-specific test execution?
- Execution of CSIT tests and incorporating locally built test images should be made as easy as possible following common guidelines
- Setting up testing environment (specific project-specific dependencies should be handled by the setup scripts)
- Specific environment variables expected by the test suite (like GERRIT_BRANCH)
Project status and readiness at the end of Guilin
| Projects with CSIT | CSIT jobs | CSIT job triggers | Tested docker images and repositories | Docker build jobs that create the tested images | Dependencies | Comments | |||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| aaf (disabled) | aaf-frankfurt-csit-certservice Frankfurt branch only? The tests triggered from here do not even exist in master (nor are there any jobs for still remaining aafapi and sms-test-plan?) | aaf-certservice-frankfurt-merge-java - this job neither builds a docker image nor triggers another job that does it, so why trigger CSIT? | |||||||||||||||||||||||||||||||||||||||||||||
| Projects with CSIT | CSIT jobs | CSIT job triggers | Tested docker images and repositories | Docker build jobs that create the tested images | Dependencies | Comments | |||||||||||||||||||||||||||||||||||||||||
| aaf (disabled) | aaf-frankfurt-csit-certservice Frankfurt branch only? The tests triggered from here do not even exist in master (nor are there any jobs for still remaining aafapi and sms-test-plan?) | aaf-certservice-frankfurt-merge-java - this job neither builds a docker image nor triggers another job that does it, so why trigger CSIT? | nexus3.onap.org:10001/onap/ org.onap.aaf.certservice.aaf-certservice-api:frankfurt-latest (hardcoded) The source repository is aaf/certservice | aaf-certservice-maven-docker-stage-frankfurt (note that aaf-certservice-release-verify and aaf-certservice-release-merge also build aaf-certservice-api image, but not with frankfurt-latest tag) | In maintenance mode, CSIT disabled:
| appc (disabled) | appc-master-csit-healthcheck | - | nexus3.onap.org:10001/onap/appc-image:1.7.2-SNAPSHOT-latestnexus3.onap.org:10001/onap/ appc-cdt-image:1.7.2-SNAPSHOTorg.onap.aaf.certservice.aaf-certservice-api:frankfurt-latest (hardcoded) docker-compose.yml as well as appc-image and appc-cdt-image come from appc/deployment No job is producing these images at the moment! appc-deployment-master-docker-java-daily produces 1.8.0-SNAPSHOT-latest and appc-deployment-frankfurt-docker-java-daily is producing 1.7.3-SNAPSHOT-latest 1.7.2-SNAPSHOT-latest images have not been created since May 27 | mariadb:10.4.3 nexus3.onap.org:10001/onap/ccsdk-dgbuilder-image:0.6.0 (no longer produced, last update to Nexus on Aug 30 2019) nexus3.onap.org:10001/onap/ccsdk-ansible-server-image:0.4.4 (no longer produced, last update to Nexus on Jun 06 2019) (hardcoded, real images used) | The source repository is aaf/certservice | aaf-certservice-maven-docker-stage-frankfurt (note that aaf-certservice-release-verify and aaf-certservice-release-merge also build aaf-certservice-api image, but not with frankfurt-latest tag) | primekey/ejbca-ce:6.15.2.5 (hardcoded, real image used) | In maintenance mode, In maintenance mode, CSIT disabled:
| |||||||||||||||||||||||||||||||||
| appc ( | temporarilydisabled) |
| Jira | ||||||
|---|---|---|---|---|---|---|
|
| Jira | ||||||
|---|---|---|---|---|---|---|
|
| Jira | ||||||
|---|---|---|---|---|---|---|
|
-
nexus3.onap.org:10001/onap/appc-image:1.7.2-SNAPSHOT-latest
nexus3.onap.org:10001/onap/appc-cdt-image:1.7.2-SNAPSHOT-latest
(hardcoded)
docker-compose.yml as well as appc-image and appc-cdt-image come from appc/deployment
No job is producing these images at the moment!
appc-deployment-master-docker-java-daily produces 1.8.0-SNAPSHOT-latest and
appc-deployment-frankfurt-docker-java-daily is producing 1.7.3-SNAPSHOT-latest
1.7.2-SNAPSHOT-latest images have not been created since May 27
mariadb:10.4.3
ccsdk-oran-maven-docker-stage-master
ccsdk-oran-maven-docker-stage-guilin10001/onap/ccsdk-
oran-a1policymanagementservice (1.1.0-SNAPSHOT for master, 1.0.1-SNAPSHOT for Guilin)The source repository is ccsdk/oran
ccsdk-oran/job/ccsdk-oran-maven-merge-master and ccsdk-oran-maven-merge-guilin produce the tested images respectively (note that these are different images that the STAGING-tagged ones from triggering jobs!)
nexus3.onap.org:10003/onap/sdnc-image (2.1.0-STAGING-latest for master, nexus3.onap.org:10003/onap/sdnc-image:2.0.1-STAGING-latest for Guilin)
nexus3.o-ran-sc.org:10004/o-ran-sc/nonrtric-controlpanel:2.0.0
mysql/mysql-server:5.6
nexus3.o-ran-sc.org:10004/o-ran-sc/a1-simulator:2.0.0
consul:1.7.2
nexus3.onap.org:10001/onap/org.onap.dcaegen2.platform.configbinding.app-app:2.3.0
mrstub:latest
callback-receiver:latest
These tests are currently cloned from ORAN repository and they are essentially plain shell scripts that are just wrapped into Robot
In Honolulu the test cases should be brought under ONAP repository and rewritten with Robot and ONAP CSIT scripts as much as possible
dgbuilder-image:0.6.0 (no longer produced, last update to Nexus on Aug 30 2019)
nexus3.onap.org:10001/onap/ccsdk-ansible-server-image:0.4.4 (no longer produced, last update to Nexus on Jun 06 2019)
(hardcoded, real images used)
In maintenance mode, CSIT disabled:
| Jira | ||||||
|---|---|---|---|---|---|---|
|
See
| Jira | ||||||
|---|---|---|---|---|---|---|
|
| Jira | ||||||
|---|---|---|---|---|---|---|
|
| Jira | ||||||
|---|---|---|---|---|---|---|
|
- 10003/onap/
- ccsdk-oran-a1policymanagementservice (1.1
nexus3.onap.org:10001/onap/clamp-backend:4.1-STAGING-latest
(hardcoded)
The docker-compose.yml and source of the images is clamp
mariadb:10.5.4
python:2-slim
- .0-SNAPSHOT for master, 1.0.1-SNAPSHOT for Guilin)
The source repository is ccsdk/oran
ccsdk-oran/job/ccsdk-oran-maven-merge-master and ccsdk-oran-maven-merge-guilin produce the tested images respectively (note that these are different images that the STAGING-tagged ones from triggering jobs!)
nexus3.onap.org:10003/onap/sdnc-image (2.1.0-STAGING-latest for master, nexus3.onap.org:10003/onap/sdnc-image:2.0.1-STAGING-latest for Guilin)
nexus3.o-ran-sc.org:10004/o-ran-sc/nonrtric-controlpanel:2.0.0
mysql/mysql-server:5.6
nexus3.o-ran-sc.org:10004/o-ran-sc/a1-simulator:2.0.0
consul:1.7.2
nexus3.onap.org:10001/onap/org.onap.dcaegen2.platform.configbinding.app-app:2.3.0
mrstub:latest
callback-receiver:latest
These tests are currently cloned from ORAN repository and they are essentially plain shell scripts that are just wrapped into Robot
In Honolulu the test cases should be brought under ONAP repository and rewritten with Robot and ONAP CSIT scripts as much as possible
Invalid trigger jobs removed in
| Jira | ||||||
|---|---|---|---|---|---|---|
|
nexus3.onap.org:10001/onap/clamp-backend:4.1-STAGING-latest
nexus3.onap.org:10001/onap/clamp-backend:4.1-STAGING-latest
(hardcoded)
The docker-compose.yml and source of the images is clamp
mariadb:10.5.4
python:2-slim
dcaegen2-services-son-handler-master-csit-testsuites
dcaegen2-services-pmsh-master-csit-testsuite
dcaegen2-services-bbs-event-processor-master-csit-bbs-testsuites
dcaegen2-pmmapper-master-csit-pmmapper
dcaegen2-master-csit-testsuites
dcaegen2-master-csit-prh-testsuites
dcaegen2-collectors-restconf-master-csit-testsuites
dcaegen2-collectors-hv-ves-master-csit-testsuites
dcaegen2-collectors-datafile-master-csit-ManagementInterface-suite
dcaegen2-services-son-handler-master-merge-java
dcaegen2-services-pmsh-docker-merge-master
dcaegen2-services-master-bbs-event-processor-merge-java
(pmmapper has no trigger job that actually exists)
dcaegen2-collectors-ves-master-merge-java
dcaegen2-services-prh-master-merge-java
dcaegen2-collectors-restconf-master-merge-java
dcaegen2-collectors-hv-ves-master-merge-java
(datafile CSITs have no trigger jobs that actually exist)
Ticket to be raised about the non-existent trigger jobs
nexus3.onap.org:10003/onap/org.onap.dcaegen2.services.son-handler:latest
dcaegen2-services-son-handler-master-csit-testsuites
dcaegen2-services-pmsh-master-csit-testsuite
dcaegen2-services-bbs-event-processor-master-csit-bbs-testsuites
dcaegen2-pmmapper-master-csit-pmmapper
dcaegen2-master-csit-testsuites
dcaegen2-master-csit-prh-testsuites
dcaegen2-collectors-restconf-master-csit-testsuites
dcaegen2-collectors-hv-ves-master-csit-testsuites
dcaegen2-collectors-datafile-master-csit-ManagementInterface-suite
dcaegen2-services-son-handler-master-merge-java
dcaegen2-services-pmsh-docker-merge-master
dcaegen2-services-master-bbs-event-processor-merge-java
(pmmapper has no trigger job that actually exists)
dcaegen2-collectors-ves-master-merge-java
dcaegen2-services-prh-master-merge-java
dcaegen2-collectors-restconf-master-merge-java
dcaegen2-collectors-hv-ves-master-merge-java
(datafile CSITs have no trigger jobs that actually exist)
Ticket to be raised about the non-existent trigger jobs
nexus3.onap.org:10003/onap/org.onap.dcaegen2.services.son-handler:latest
nexus3.onap.org:10001/onap/org.onap.dcaegen2.services.pmsh:latest
nexus3.onap.org:10001/onap/org.onap.dcaegen2.services.components.bbs-event-processor:latest
nexus3.onap.org:10001/onap/org.onap.dcaegen2.services.pm-mapper:latest
nexus3.onap.org:10003/onap/org.onap.dcaegen2.collectors.ves.vescollector:latest
nexus3.onap.org:10001/onap/org.onap.dcaegen2.services.prh.prh-app-server:latest
nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.restconfcollector:latest
nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.hv-ves.hv-collector-main:latest
nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.datafile.datafile-app-server:latest
(all hardcoded in various places in the tests)
dmaap-buscontroller-master-csit-ssl
dmaap-buscontroller-master-csit-with_dr
dmaap-buscontroller-master-csit-with_mr
dmaap-datarouter-master-csit-ssl-dr-suite
dmaap-datarouter-maven-docker-stage-master
(other jobs have no trigger jobs that actually exist)
nexus3.onap.org:10001/onap/dmaap/buscontroller
nexus3.onap.org:10001/onap/dmaap/datarouter-provorg.onap.dcaegen2.services.pmsh:latest
nexus3.onap.org:10001/onap/dmaap/datarouter-nodeorg.onap.dcaegen2.services.components.bbs-event-processor:latest
nexus3.onap.org:10001/onap/dmaap/datarouter-subscriber
(all hardcoded, version not given so latest by default)
At least three different source repositories:
dmaap/messagerouter/messageservice
There are no jobs producing onap/dmaap/buscontroller - the latest image in Nexus has been updated Thu Feb 21 2019 and belongs to Casablanca (?!) - ticket raised:
| Jira | ||||||
|---|---|---|---|---|---|---|
|
multicloud-master-csit-functionality1
nexus3.onap.org:10001/onap/multicloud/vio:1.4.1
nexus3.onap.org:10001/onap/multicloud/framework:1.4.1
nexus3.onap.org:10001/onap/multicloud/openstack-starlingx
nexus3.onap.org:10001/onap/multicloud/vio
(all hardcoded, default latest for those that do not define version explicitly)
org.onap.dcaegen2.services.pm-mapper:latest
nexus3.onap.org:10003/onap/org.onap.dcaegen2.collectors.ves.vescollector:latest
nexus3.onap.org:10001/onap/org.onap.dcaegen2.services.prh.prh-app-server:latest
nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.restconfcollector:latest
nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.hv-ves.hv-collector-main:latest
nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.datafile.datafile-app-server:latest
(all hardcoded in various places in the tests)
TBD: most or all of these are in their own dedicated repositories, TBD: are the repo-image-CSIT relations straightforward 1:1 in all cases (or any case)?
dmaap-buscontroller-master-csit-ssl
dmaap-buscontroller-master-csit-with_dr
dmaap-buscontroller-master-csit-with_mr
dmaap-datarouter-master-csit-ssl-dr-suite
dmaap-datarouter-maven-docker-stage-master
(other jobs have no trigger jobs that actually exist)
nexus3.onap.org:10001/onap/dmaap/buscontroller
nexus3.onap.org:10001/onap/dmaap/datarouter-prov
nexus3.onap.org:10001/onap/dmaap/datarouter-node
nexus3.onap.org:10001/onap/dmaap/datarouter-subscriber
(all hardcoded, version not given so latest by default)
At least three different source repositories:
dmaap/messagerouter/messageservice
There are no jobs producing onap/dmaap/buscontroller - the latest image in Nexus has been updated Thu Feb 21 2019 and belongs to Casablanca (?!) - ticket raised:
| Jira | ||||||
|---|---|---|---|---|---|---|
|
multicloud-master-csit-functionality1
nexus3.onap.org:10001/onap/multicloud/vio:1.4.1
nexus3.onap.org:10001/onap/multicloud/framework:1.4.1
nexus3.onap.org:10001/onap/multicloud/openstack-starlingx
nexus3.onap.org:10001/onap/multicloud/vio
(all hardcoded, default latest for those that do not define version explicitly)
Latest in Nexus:
onap/multicloud/vio:1.4.1 is from Sep 12 2019
onap/multicloud/framework:1.4.1 is from Sep 06 2019
...these seem to be some early El-Alto versions?
Ticker raised: Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key MULTICLOUD-1229
https://jenkins.onap.org/view/multicloud/job/multicloud-openstack-maven-docker-stage-elalto produces version 1.4.3 of onap/multicloud/openstack-starlingx
https://jenkins.onap.org/view/multicloud/job/multicloud-framework-maven-docker-stage-master produces version 1.6.0 (and also latest) of onap/multicloud/framework
multicloud-openstack-maven-docker-stage-master produces version 1.5.6 (and also latest) of onap/multicloud/openstack-starlingx
multicloud-openstack-vmware-maven-docker-stage-master produces version 1.4.2 (and also latest) of onap/multicloud/vio
music-master-verify-csit-music-distributed-kv-store-test-plan
nexus3.onap.org:10001/onap/music/distributed-kv-store (hardcoded, latest by default)
In maintenance mode, CSIT disabled: Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key MUSIC-613
oom-platform-cert-service-master-merge-java (triggers two CSITs but does not produce or trigger docker image? Ticket raised:
| Jira | ||||||
|---|---|---|---|---|---|---|
|
Latest in Nexus:
onap/multicloud/vio:1.4.1 is from Sep 12 2019
onap/multicloud/framework:1.4.1 is from Sep 06 2019
...these seem to be some early El-Alto versions?
Ticker raised: Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key MULTICLOUD-1229
https://jenkins.onap.org/view/multicloud/job/multicloud-openstack-maven-docker-stage-elalto produces version 1.4.3 of onap/multicloud/openstack-starlingx
https://jenkins.onap.org/view/multicloud/job/multicloud-framework-maven-docker-stage-master produces version 1.6.0 (and also latest) of onap/multicloud/framework
multicloud-openstack-maven-docker-stage-master produces version 1.5.6 (and also latest) of onap/multicloud/openstack-starlingx
multicloud-openstack-vmware-maven-docker-stage-master produces version 1.4.2 (and also latest) of onap/multicloud/vio
music-master-verify-csit-music-distributed-kv-store-test-plan
music-distributed-kv-store-master-docker-golang-shell-dailynexus3.onap.org:10001/onap/music/distributed-kv-store (hardcoded, latest by default)
org.onap.oom.platform.cert-service.oom-certservice-api:latest
nexus3.onap.org:10001/onap/org.onap.oom.platform.cert-service.oom-certservice-post-processor:latest (but started in Robot test and defined in Robot variable so it would be theoretically overridable)
In maintenance mode, CSIT disabled:| Jira | ||||||
|---|---|---|---|---|---|---|
|
oom-platformmaven-certdocker-servicestage-master-csit-certserviceoom-platform-cert-service (for both)
primekey/ejbca-ce:6.15.2.5
optf-osdf-master-csit-postprocessorosdf
oom-platform-cert-servicemergejava| Jira | ||||||
|---|---|---|---|---|---|---|
|
nexus3.onap.org:10001/onap/optf-cmso-service:latest
nexus3.onap.org:10001/onap/optf-cmso-dbinit:latest
nexus3.onap.org:10001/onap/optf-cmso-topology:latest
nexus3.onap.org:10001/onap/optf-cmso-ticketmgt:latest
nexus3.onap.org:10001/onap/orgoptf-cmso-optimizer:latest
nexus3.onap.oom.platform.cert-service.oom-certservice-api:latestorg:10001/onap/optf-cmso-robot:latest (!! The actual Robot tests run here in setup phase instead of as separate Robot tests from CSIT repo)
nexus3.onap.org:10001/onap/org.onap.oom.platform.cert-service.oom-certservice-post-processor:latest (but started in Robot test and defined in Robot variable so it would be theoretically overridable)
oom-platform-cert-service-maven-docker-stage-master (for both)
primekey/ejbca-ce:6.15.2.5
optf-has:2.0.2-SNAPSHOT-latest
nexus3.onap.org:10001/onap/optf-osdf:2.0.3-SNAPSHOT-latest
Repositories:
optf-cmso-maven-docker-stage-master
optf-has-maven-docker-stage-master
optf-osdf-maven-docker-stage-master
(word of warning: the frankfurt versions of the above jobs are also pushing the same images with "latest"! Ticket
(all CSITs have non-existent trigger jobs, ticketto be raised)
...