PyPi configuration in Nexus3

There are 5 PyPi repositories present in Nexus3:

Pypi (proxy) - This repository is used to proxy https://pypi.python.org/

https://nexus3.onap.org/repository/PyPi/

Pypi.public (group) - This group consolidates PyPi.release and PyPi. This group should be accessed for reading/pulling purposes.

https://nexus3.onap.org/repository/PyPi.public/

Pypi.release (hosted) - This repository hosts our official releases. This repo does not allow re-deployment

https://nexus3.onap.org/repository/PyPi.release/

Pypi.snapshot (hosted) - Used to host our snapshots meanwhile the release candidate is being worked on. This repo allows re-deployment. 

https://nexus3.onap.org/repository/PyPi.snapshot/

Pypi.staging (hosted) - Used for our daily/staging non official publications. This repo allows re-deployment.

https://nexus3.onap.org/repository/PyPi.staging/

PyPi configuration in Jenkins and JJB

Projects can leverage this set of templates when they are a python project that needs to upload python packages to a PyPi repository.

These templates are located in the ci-management repo at "ci-management/jjb/onap-jjb/onap-pypi-template.yaml".

There are currently 4 templates available:

  1. {project-name}-python-staging-{stream}
  2. {project-name}-python-release-{stream}
  3. {project-name}-{subproject}-python-staging-{stream}
  4. {project-name}-{subproject}-python-release-{stream}

General Job Information

The "staging" jobs will trigger a Jenkins job that builds the python package and pushes the package to a staging repo located here https://nexus3.onap.org/repository/PyPi.staging/.

The staging jobs can be triggered by the following events:

  1. A gerrit merge event
  2. A comment on a gerrit patch of "pypi-remerge"

The "release" jobs will trigger a similar job that builds the python package and pushes it to the release repo located here https://nexus3.onap.org/repository/PyPi.release/.

The release jobs can be triggered by the following event:

  1.  A comment on a gerrit patch of "pypi-release"


Both jobs require the following variables to be defined:

  • project-name
  • stream
  • tox-dir (optional, default: '')
  • tox-envs (optional, default: '')
  • python-version

Optional params

tox-dir typically only need to be specified for subprojects.  You can specify it though if your tox.ini is somewhere other than the root of your project.  tox-envs are set to blank always so they do not overwrite your tox.ini;  Setting them blank gives the option to set them if you manually kick off a job for troubleshooting purposes though.


Python projects with a subproject

Some projects have a structure where one repo contains several projects.  LF has built templates to support them as well; you will use

these templates:

  • {project-name}-{subproject}-python-staging-{stream}
  • {project-name}-{subproject}-python-release-{stream}


Here is a concrete example of how to set up pypi jobs for a project with subprojects using dcaegen2/utils.

Example.yaml
---
- project:
    name: dcaegen2-utils
    project-name: 'dcaegen2-utils'
    python-version: python3
    project: 'dcaegen2/utils'
    stream:
      - 'master':
          branch: 'master'
    mvn-settings: 'dcaegen2-utils-settings'
    # due to a strange macro / variable translation problem this needs
    # to be passed as a string block to properly get the properties
    # correctly defined in the job
    maven-deploy-properties: |
      deployAtEnd=true
    files: '**'
    archive-artifacts: ''
    subproject:
      - 'dcaeapplib':
          tox-dir: dcaeapplib/
          tox-envs: ''
      - 'onap-dcae-dbs-docker-client':
          tox-dir: onap-dcae-dbs-docker-client/
          tox-envs: ''
      - 'onap-dcae-dcaepolicy-lib':
          tox-dir: onap-dcae-dcaepolicy-lib/
          tox-envs: ''
      - 'python-discovery-client':
          tox-dir: python-discovery-client/
          tox-envs: ''
      - 'python-dockering':
          tox-dir: python-dockering/
          tox-envs: ''

    jobs:
      - '{project-name}-{subproject}-python-staging-{stream}'
      - '{project-name}-{subproject}-python-release-{stream}'

   
    build-node: 'ubuntu1604-docker-8c-8g'



Note that the use of these templates implies a workflow where the staging artifact will be overwritten on each merge event.  We are working on features that will allow immutable staging as well.