KPI 1: CII Badging (Tony)

CII security requirements

  • Assurance case requirement: 50% of the projects have "Met" that requirement
    • project needs to produce documentation to satisfy this requirement and link to it from the CII badge page (wiki, readthedocs)
  • Application quality security requirements at the silver level: fewer than 10% of the projects not answering

CII non-security requirements with canned responses: 100% "Met" response

  • Note: All projects need to upgrade response to Passing (Vulnerability Report Private) to "Met"

KPI 2: Closed OJSI Tickets (Krzysztof)

  • No HTTP ports exposed.
    • All port expose HTTPS, or
    • HTTP port waiver granted by the SECCOM and documented in readthedocs
  • All OJSI tickets with CVEs assigned are closed (Security level set to None).

KPI 3: Remediating Known Vulnerabilities in Third Party Packages (Amy)

  • All Jiras for upgrading direct dependencies are closed (tickets with label= ComponentUpgrade).
  • If the project is unable to upgrade a direct dependency, they must have a TSC exception with documentation of the reason the direct dependency was not upgraded.

KPI 4: Code coverage tests (Pawel, Amy)

Frankfurt

  • All projects achieve at least 55% code coverage.
  • If a project is unable to achieve 55% they must:
    • Request a TSC exception including:
      • Reason 55% coverage cannot be achieved,
      • % coverage they can achieve.
  • KPI measurement
    • Projects without exceptions: passing = at least 55%
    • Projects with exceptions: passing = at least committed %
  • All projects document the % coverage in the readthedocs and the location of the test suites.

Guilin and beyond

The desire is for projects to concentrate on code coverage tests for new code and core components. Until we have tooling available that reliably measures this, we will use the following measures to assess code coverage.

  • All projects commit to the % coverage they can meet.
  • KPI: passing = at least committed %
  • Code coverage below 55% requires a TSC exception as documented in the Frankfurt code coverage tests above.


  • No labels

4 Comments

  1. Krzysztof Opasiak

    I think that we should define this criteria on a per project basis not for a whole ONAP at once as the state of every project is way way different.

    Having said that for OJSI I'd propose:

    • No HTTP ports exposed:
      • All port expose HTTPS, or
      • HTTP port waiver granted by the SECCOM and documented in readthedocs
    • No publicly known (Security level set to None) OJSI tickets with CVEs assigned.

    1. Paweł Pawlak

      Krzysztof's proposal makes sense for me.

  2. Paweł Pawlak

    For Code coverage part taking into account feedback from PTLs we will most probably try to get firm improvement proposals from projects that would reflect availability of resources.  

  3. Natacha Mach

    Projects are managed separately in terms of resources, security priorities and other topics.

    So KPI should be managed accordingly, per project. Otherwise it will not reflect anything, as there is an important difference between each project.