...

• The project must define a review verification job based on "{project-name}-unified-review-{stream}-PoC" template (which is defined in ci-management/jjb/integration/integration-templates-csit.yaml)
• The following JJB variables must be defined in the project definition that defines the review verification job:
  • project-name (project identifier that can be freely chosen)
  • project (should correspond project's name in Gerrit)
  • stream (list of supported branches)
  • artifact-type (should match the type of docker snapshot PoC template)
  • java-version
  • mvn-version
  • mvn-settings
  • mvn-global-settings
• The project must define a job for building snapshot docker images from artifacts of given artifact-type based on any template following naming convention "{project-name}-{stream}-{artifact-type}-docker-snapshot-PoC"
  • only golang available at the moment! New templates for java, python etc. should be added to support more projects
  • The docker job definition must specify the project-specific script or method that
    • builds the artifacts
    • puts the artifacts into docker image(s)
    • tags the docker image(s) with <version>-SNAPSHOT-<unique docker tag> where UNIQUE_DOCKER_TAG environment variable (passed down as Jenkins job input parameter) is used for the last part
    • pushes the docker image(s) to ONAP snapshot Nexus
• The project must define a job for project-specific CSIT based on "{project-name}-{stream}-project-csit-PoC" template defined in ci-management/jjb/integration/integration-templates-csit.yaml
  • The project must have csit/run-project-csit.sh that is responsible of the entire CSIT execution
  • run-project-csit.sh should use the UNIQUE_DOCKER_TAG environment variable (passed down as Jenkins job input parameter) to specify the exact docker images to pull and run in startup phase
  • Following the integration/csit subdirectory structure and using its run-csit.sh (by cloning the integration/csit and copying the common scripts to workspace) to execute the actual tests is recommended (at least for existing Robot CSITs) but not mandatory
• The project should define a merge verification job based on "{project-name}-unified-merge-{stream}-PoC" template (which is defined in ci-management/jjb/integration/integration-templates-csit.yaml)
• The project should define a job for tagging verified snapshot docker images as STAGING based on "{project-name}-{stream}-{artifact-type}-docker-staging-PoC" template (in ci-management/jjb/global-templates-docker.yaml)
  • The docker job definition must specify the project-specific script or method that
    • pulls the verified snapshot docker images identified by the UNIQUE_DOCKER_TAG environment variable (passed down as Jenkins job input parameter) from ONAP snapshot Nexus
    • tags the docker image(s) with <version>-STAGING-latest
    • pushes the docker image(s) to ONAP snapshot Nexus

Example

...

projects

Music PoC

PoC implementation of the above design is available in music project for reference - note though that the CSIT is currently disabled and that artifact build jobs are not yet supported by the templates

• All the related job and template changes done to ci-management repository are available in master, see
  • ci-management/jjb/
    • global-templates-docker.yaml
    • global-templates-golang.yaml
  • ci-management/jjb/integration/
    • integration-macros.yaml
    • integration/integration-templates-csit.yaml
  •  ci-management/jjb/music/
    • music-distributed-kv-store.yaml
    • music-distributed-kv-store-csit.yaml
    • build-music-distributed-kv-store-image.sh
    • tag-music-distributed-kv-store-image-staging.sh
• The changes to music repository itself are available in open Gerrit review in https://gerrit.onap.org/r/admin/crepos/music/distributed-kv-store/+/114637
  
  • the entire csit subdirectory
  • changes to docker scripts to support unique tagging and staging

CCSDK/distribution

First actively used example of this design (excluding artifact build jobs) is now available

• All the related job and template changes done to ci-management repository are available in master
  • ci-management/jjb/integration/
    • integration-maven-jobs.yaml
    • integration/integration-templates-csit.yaml
  •  ci-management/jjb/ccsdk/
    • distribution.yaml
    • ccsdk-csit.yaml
    • tag-ccsdk-distribution-images-staging.sh
• The changes to ccsdk/distribution repository itself are available under https://gerrit.onap.org/r/admin/repos/ccsdk/distribution
  
  • the entire csit subdirectory
  
  
  • The entire csit subdirectory
  • changes to docker scripts to support unique tagging and staging

Open questions and remaining tasks

• The There are only two snapshot docker build template templates currently available is for golang
  • templates for java, python etc. should be added if no common template is feasible
  • maven
  • golang
  Do we need umbrella jobs for review and merge verification, or should we trigger docker jobs and chain CSIT to them directly?
  • Making changes to the overall structure might be more complex without umbrella jobs
• Is BUILD_ID the best possible identifier to be used for unique docker tag?
  • Most docker builds already use unique timestamp to identify the images, but using it between jobs is more complex (especially in case of umbrella jobs) 
• This proposal replaces plain Maven/Makefile/other artifact build jobs completely
  • Sonar builds seem to be completely separate scheduled builds anyway, and unsupported by any templates?
  • Ideally all reviews should start with Sonar build and fail the review on violations