The proposal

The Details

Best Practices

Best practices are functional or non-functional design patterns. Some potential examples:

  • Container rootfs has to be mounted read-only
  • Secret material should not be placed in logs
  • Components in maintenance mode cannot be used as a dependency

(just examples, not an initial set of best-practices)

Best practices should be either a separate Jira project or a repo in RST that can be included in the documentation. The details of creation process depends on the form chosen by then community.

Anyone in the community can propose a new best practice at any time.

Best practice can be in one of below states:

  • WIP
  • Under review
  • Waiting for approval
  • Approved
  • Rejected
  • Removed

WIP - default state for every new BP that allows the author to work on it without bothering other people

Under review - the BP is in the review with impacted parties (Every BP should be socialized with PTL. BP that impact the architecture (functional) should be consulted with the architecture Team. BP that impacts docs should be consulted with DOC team. BP that impacts security aspects should be consulted with SECCOM)

Waiting for approval - The BP has been socialized with required parties and now it is ready to be voted by the TSC

Approved - The BP has been approved by the TSC and it should be followed by all New Code (the definition is below)

Rejected - The BP has been rejected by the TSC, which means that there is no agreement in the community that this is the right pattern to follow

Removed - The BP that has been previously in the Approved state may turn out to be out of date or impossible to follow due to some project constraints. In such case anyone may ask the TSC to revise such a BP and revoke it.


Best practices has to be followed by the new code that is arriving to the ONAP tree. The definition of new code may depend on the project thus it's recommended that every PTL clarifies the rules for his project.

For example:

In OOM project typically we consider every new chart that arrives for a review to be a new code

In DOC project a new code may be a new page or a new module

In typical java development project a new code may be new module or new microservice depending on PTL criteria.


It's advised, but not mandatory, for the PTL to ensure, as soon as resources allows, that the code that is already in ONAP repos and is under his responsibility follow all approved BP. However this is not going to be gated as long as given BP is not approved as a Global Requirement for a given release.

Global Requirements

Global requirement is an already approved  best practice, that has been chosen by the TSC to be the gating criteria for a given release.

Before every release TSC agrees on the list of BP that are view by them as High Priority Issues and should be applied to whole ONAP code base, not only to a new code.

Components that does not meet GR at Feature Freeze will be reverted back to the container version used in the previous release. This means that no new functionality can be added to this container in a given release if it doesn't meet the GR.

Feature

This is practically the same as our current "Use case". It's basically an ONAP-level functionality that impacts multiple components and is visible to the end user. The name "Feature" can be replaced with the current "Use case" if community prefers this name. This can be represented as a Jira ticket in a dedicated project or an RST page in a dedicated repo.

Feature may be in one of below states:

  • WIP
  • Under review
  • Waiting for approval
  • Approved
  • Rejected
  • Active
  • On Hold
  • In testing
  • Finished
  • Dropped

WIP - default state for every new feature that allows the author to work on it without bothering other people

Under review - the BP is in the review with:

  • Architecture
  • Requirements subcomittee
  • SECCOM
  • PTLs

To save submitter time, the review should happen asynchronously by default (unless submitter asked for a VCC to present) and only if subcommittee has additional questions a VCC meeting to answer them should be scheduled. Questions and doubts should be written down before the VCC.

Waiting for approval - The feature has been socialized with required parties and now it is ready to be voted by the TSC

Approved - The Feature has been approved by the TSC. Implementation may be started if the feature is put in the "Active" state before "Spec freeze" for a given release.

Rejected - The Feature has been rejected by the TSC, which means that there is no agreement in the community that this is the right change

Active - The feature is being actively worked on in a current release.

On Hold - The implementation has been started but there is no progress planned for a current release due to some constraints.

In testing - The feature is complete and is now being tested by the integration team

Finished - implemented and tested

Dropped - people lost interest in this feature

Anyone in the community can propose a new feature at any time but the implementation should be started only when Feature is Active for a given release.

A feature may be linked directly to tasks if it's relatively simple or consist of multiple specs from which every one may be tested separately in "early release model".

Spec

This is "a smaller feature" which involves architectural change in only single component ie. some internal design refactoring etc. This can really be a Jira epic that describes where associated tasks are going.

Spec is scoped for single component thus it has to be approved only by people who know this component best - PTL and his or her committers.

PTL may request additional review from subcommittees if needed.

Spec has to be marked as Active before it can be implemented. In fact it behaves similar to Features but is just smaller


Kickoff

Official start of the new release. Community presents to TSC GR proposals with reason why they believe this GR is important.

GR Approved

Global requirements has been approved by the TSC.

Spec freeze

All specs and features that will be worked on in this release has been put in the Active state. This is the maximal set of features that can be achieved in this release.

Feature freeze

We no longer accept commits that adds new features. Only bug fixes are accepted (Jira ticket has to be of type Bug). All projects that didn't met GR will be reverted to the container image from previous release.

At this point we know exactly what has been implemented.

RC0

We branch out release branch. On the master branch the work for next release may start and on our release branch we focus on stabilization, bug fixing and testing.

Sign-off

We officially announce our new release.




  • No labels

15 Comments

  1. David McBride

    Krzysztof Opasiak - it would be helpful to have some detail about the role of the release manager in supporting the new process.

    1. Krzysztof Opasiak

      David McBride

      The role of release manager would be:

      1) Keep track of global requirements for a given release

      2) Reminding about important dates (code freeze, feature freeze etc)

      3) Keep track of issues to be fixed during bug fixing period

      4) Keep track of features and specs that are getting implemented and reminding owners about their testing

      1. Catherine Lefevre

        In order to kick-off the Honolulu release, what does the ONAP Community (i.e. Subcommittee, PTL, TSC, requirement owners etc.) need to prepare ? thanks

      2. David McBride

        Thanks, Krzysztof Opasiak

        Would we continue to have release management tasks tied to milestones, as we do now.  Would they be the same tasks, or different?

  2. krishna moorthy

    Since we are moving towards Agile, it is important to pick up the features and spec of appropriate size, so that the scope reduction can be avoided during the course of the release.

    With respect to the global requirements, the requirement owners can work with one of the projects to complete the requirement(which I guess is already being done) which can be published as tutorial page somewhere in the wiki, so that other projects can use it for understanding and meeting the requirement. I personally felt a few requirements were somewhat time consuming due to the lack of these resources or pointer to the resources. The 30 minutes practice is really helpful on covering this, but it is also difficult to cover all the requirements over the course of release.

    There is still a possiblity of not meeting all the GR at feature freeze due to time and resource constraints from the project teams. There has to be some alternative to it.


    1. Krzysztof Opasiak

      krishna moorthy

      So the main difference between Feature and spec is that Feature touches multiple components while spec is focused on a change in a single component.

      I hope that specs would be small but even if they span across release it's not an issue we'll still use Jira for day to day work


      I agree with you regarding global requirements. That's why the proposal is to firs create set of best practices and only then pick some of them as a global requirement. By definition we opt for best practices that are well documented and widely understood by the community.


      TSC may grant an exception but this should be very rare. Number of GR for every release should be kept small to ensure that they are really doable and only projects that are not actively maintain cannot meet them


  3. Catherine Lefevre

    Please find some comments

    1.  "Best practices should be either a separate Jira project or a repo in RST that can be included in the documentation." Based on what was discussed and part of the above deck. Best practices should be documented following the EPIC template ( REQ-1 - Getting issue details... STATUS ). It is unclear why these should in a separate JIRA project or a repo in RST? Can you please clarify?
    2.  "It's advised, but not mandatory, for the PTL to ensure, as soon as resources allows, that the code that is already in ONAP repos and is under his responsibility follow all approved BP. However this is not going to be gated as long as given BP is not approved as a Global Requirement for a given release." How can the project team review all the code (old and new) align wiht the best practice? It might not always be possible depending on the best practice content.
    3.  Components that does not meet GR at Feature Freeze will be reverted back to the container version used in the previous release. It is not possible because it means that you are reverting all the use case, the HELM Charts in progress and some GR already completed without considering cross-project dependencies. I believe this should be reviewed case per case by the ONAP TSC.
    4.  If GR is an approved best practice and best practice is a functional or non-functional design patterns, it means that only the use cases do not require any TSC prioritization
    5.  GR = functional best practice should also be reviewed by Requirements/Control Loop subcommittees and Architecture prior GR TSC approval. GR = non functional best practice could be reviewe by SECCOM prior TSC approval
    6.  what's a VCC?
    7.  Do you still consider a TSC prioritization to help the PTL as part of the process or just a TSC approval per GR?
    8. As part of Kick-Off, it is expected that all the EPIC Template for any GR is created as well as Release impact view per component wiki page updated to help the PTLs to understand the potential release scope i.e. GR + Feature
    9.  It is understood that none of M1 tasks are requested as part of the new release cadence - nevertheless the PTLs will need to assign issues that they will develop to the new release. When the PTLS communicate to the other projects their API Changes? No more tracking of platform maturity chart - we believe it will be replaced and driven by the GRS. When PTL needs to update their FOSS information, CII badging and documentation plan?
    10. When should PTL inform about any OOM port list change?
    11. Is there any check to ensure that no merge requests exceed a long period?
    12. When does the PTL share Data models with Modelling Teams
    13. When PTL review his security issues? or how it is ensured it is done prior the release sign-off
    14. When PTL address their license issues? or how it is ensured it is done prior the release sign-off
    15. When do we check the test coverage is met?
    16.  When does PTL update their test plan i.e. Integration weather board ?
    17.  Do the tasks on the slide #1 are enough to ensure that the release is on track i.e. mini M2/M3/M4, RC0?
    18.  How do we manage requirements impacting multiple releases i.e. REQ-341 when we want to deliver a piece of it per release

    1. Krzysztof Opasiak

      Ad. 1:

      Because those are going to be long living tickets and sth that you want to sent to developers as a checklist of practices that should be followed. Keeping those in repo in RST format has an advantage that it automatically becomes a part of ONAP documentation but if we are not ready for this we can start with Jira tickets


      Ad 2:

      Fully agree. We should try to make our best practices as much testable as possible but I agree that it may be not always possible. In that case code review is the only tool that we can use. For old code I believe that most of PTL at least roughly knows the code base that they maintain thus they should be able to at least identify places to be checked.


      Ad 3

      Sure there is a place for a TSC exception here but as a rule we should keep the old version. Current practice shows that most of the projects are releasing 1-2 images per release (excluding bug fixes in last stage of the release) so it's not an issue at all we would just reject the new image unless TSC grants the exception. If projects are releasing more often (which is great) they should not make any breaking changes to their API or depend on APIs introduced by components that have not met GR yet. This is to ensure that we can always revert back.


      Ad 4

      I don't get this question. Could you please explain it a little bit?


      Ad 5

      From my point of view GR can be reviewed by the whole community but let's make sure that the process happens "asynchronously". Currently it takes ages to get sth approved becuase:

      1. You are ready with your proposal so ask a week before to be included in arch subcom agenda
      2. You go to arch VCC but you drop from the agenda due to time limit
      3. You wait yet another week and finally you present and it gets apprved
      4. In next week you go to requirements subcommittee you present and get approval
      5. You ask to discuss sth at TSC level but once again you don't manage to fit into agenda
      6. You wait one more week
      7. Finally you present sth to TSC and it gets approved

      So all in all you've just spent 5 weeks to just get your proposal approved and reviewed by various people and approved by the TSC and this time can be even way longer if any of subcommittees asks for some fixes

      Instead I'd propose the following:

      1. You submit your GR proposal for a review (gerrit/Jira ticket whatever form)
      2. Anyone from the community can review the proposal, ask questions or request fixes
      3. If there are some serious doubts subcommittee may request to present the proposal during their call
      4. If the proposal gets +1 from all interested parties or a week has passed since submission and no one is interested in reviewing it it automatically goes to closest TSC gets presented and voted

      So I expect this would reduce the time required to get GR approved significantly and that's really the goal. Reduce the paper work and the time that has to be invested to the minimum.

      Ad 6

      VCC - Video Conference Call (Zoom meeting;)


      Ad 7

      I'd prefer that anyone can assign priorities on their own so I'd keep all GR as a strong gating requirements.


      Ad 8

      Yes. When GR are approved the table with impact per component should be updated.

      For features the table should be kept "live" and as soon as feature is approved the table should be updated not only at the kickoff


      Ad 9

      a) It is understood that none of M1 tasks are requested as part of the new release cadence - nevertheless the PTLs will need to assign issues that they will develop to the new release.

      Not really. The idea is tat people declare that they are working on features/specs and that's it. Obviously if those team want to collaborate it's fine that they create all tasks upfront and assign them but it's not a mandatory requirement.

      b) When the PTLS communicate to the other projects their API Changes?

      As far as I understand our API versioning strategy components should not break the existing API but introduce their new versions. I believe that that every component should put all the API changes in their release notes whenever they are release new docker images. If there is a need to share additional information feel free to propose additional method of sharing that information

      c) No more tracking of platform maturity chart - we believe it will be replaced and driven by the GRS. When PTL needs to update their FOSS information, CII badging and documentation plan?

      Whenever they want but not later than Code freeze


      Ad 10

      When they are submitting an image update to OOM


      Ad 11

      I'm not sure how this is related but we can always have some kind of gerrit bot that checks open reviews


      Ad 12

      As long as there is no change the Data Model is constant right? So I believe that those models should be shared as a part of work on Feature/Spec right?


      Ad 13

      Do you mean OJSI or Nexus scan?


      Ad 14

      License issues should be fixed before the code freeze provided that we notify PTL early enough


      Ad 15

      At the code freeze


      Ad 16

      I'd encourage PTLs to do this as soon as possible. For example we've just shipped a docker image that contains the implementation of new spec and feature so let's assign this feature to be tested by the integration team even if it's only the beginning of the release

      Ad 17

      Release is always on track as we are switching from "Task bound planning" to "Time bound planning" this means that we'll stabilize whatever is in the master branch at a particular deadline. If sth is not there it can be integrated for the next release.


      Ad 18

      We deliver whatever is possible to be delivered based on the Team's priorities. Feature owner may formulate a long term plan by creating all tickets and assigning them to particular releases or just describe everything in a single ticket. I;d leave this to the feature owner.

      1. Catherine Lefevre

        Q4 - clarifications. 

        Up to now (i.e. Guilin release) - we have 3 categories: use cases, functional requirements and non functional requirements

        If you look at the definition of Best pratices - best practices = Best practices are functional or non-functional design patterns and Global requirement is an already approved  best practice

        It means that the use cases are the only items that do not require any more TSC approval.

        Did I get it right? thanks


        Q5 It would be good to review your proposal with subcommittees & PTLs

        Q7 In the past we noticed that lack of prioritization is an issue because reqs backlog exceeds nbr od developers & testers. In Guilin 31 non functional reqs + 20+ functional reqs = now called best practices; it was helping a lot the PTLS to define their scope and commitments. If Honolulu scope is about 40-50 best practices - how do we proceed if no prioritization?

        Q9 If M1 tasks not requested then In order to kick-off the Honolulu release, what does the ONAP Community (i.e. Subcommittee, PTL, TSC, requirement owners etc.) need to prepare ? thanks

        Is it not too late to document API changes in their release note? How can we proactively help the PTLs to detect API Change before the end of the release? Or are you suggesting to create the release note at M2/M3 milestone?

        Q12 Spec freeze sounds good

        Q13 yes this is correct

        1. Krzysztof Opasiak

          Q4

          Yes Best Practices has to be approved by the TSC and GR also.

          For Specs, in my opinion  PTLs & committers agreement should be enough as the change affects a single project

          For Features which affect multiple projects I believe that we can try without the TSC approval first and if there is some disagreement between submitter and one of reviewing parties then someone may ask TSC for their opinion. Does it sound good to you?


          Q5

          Fully agree, I'm open for a discussion and their feedback


          Q7

          Best practices are worked as a "best effort" for already existing code, global requirements are what is enforced. Best practices are mandatory to be followed only by a new code. For example:

          Best practice is to have a container rootfs mounted read only.

          We would enforce this practice for any new component/microservice that appears in ONAP/

          But we would not "complain" on anyone if the still have an RW rootfs until this Best Practice becomes a GR for a given release.

          I took a look into list of "functional requirements" that we have now and in my opinion most of them will fall under Feature and Spec category (see diagram below) so this shouldn't be an issue.


          Q9

          Community:

          • propose best practices
          • start proposing specs and features that they want to implement

          TSC:

          • review & vote on proposed best practices
          • establish GR

          Subcommittees:

          • review incoming best practices & features

          Regarding the API change, The goal here is to encourage people to release new images more often and not wait to particular milestone. As soon as new API is ready it should be submitted to OOM for testing and to allow other people to use it.

  4. Catherine Lefevre

    Clarify the category where SP3/Nonfunctional requirements; Use Cases and Functional Requirements fit to the new definition Features, GR, best practicees

    1. Krzysztof Opasiak

  5. Michela Bevilacqua

    Hi, 

    Trying to elaborate Spec and  Features management in the new WoW: 

    A Feature is a new ONAP UC that must be approved by ONAP TSC, submitted to requirement subcommittee and architectural subcommitte while Spec is any new content part of the current ONAP Use Cases.

    Can a Spec involve  multiple components if it is part of an already existing ONAP Use Case ? I would expect yes.

    Spec does not need any requirement and architecture subcommittee review but a presentation and agreement of the task with the specific ONAP PTL/committers. In case PTL may request additional review from subcommittees if needed. 

    I would then expect that a new spec can be add up to the Spec Freeze date. 


    In the picture the term Feature Freeze is used to indicate the functional freeze but then what is the deadline for the submission for a new Function ? (that I could call a Feature Freeze similarly  to Spec Freeze) . Maybe it corresponds to GR approval date

    1. Krzysztof Opasiak

      Michela Bevilacqua

      Please see the diagram that I've put above in the comment it should make it more clear to you.


      Now I realize that the naming is confusing but the deadlines are:

      Spec freeze - deadline when we stop accepting new features and specs for a given release (documents)

      Feature freeze - deadline when we stop accepting implementation of new features for a given release (code)


      So at spec freeze we know what at most is going to be implemented while at Feature freeze we know what we've really managed to deliever


  6. Michela Bevilacqua

    Krzysztof Opasiak  : An additional question, 

    In my understanding we are considering a global requirement, a functional or non functional requirement for a given release, to be prioritized by TSC. Use Cases are not considered global requirements and they do not need any prioritization.

    It is a bit blurry what we mean as community for a Use Case and it would be very useful spend some words trying to define it. 

    Moreover, when it is planned to extend an existing Use Case for a given release, adding new functionalities, I would question if the new functionality should not be anyhow identified by a functional requirement/global requirement.