This is just a draft version put together to capture the feedback provided during the meeting and present my point of view based on my experience in other open source projects

Current state

  • Patches are submitted via gerrit
  • Architecture is described on a wiki and (sometimes) later migrated to read the docs
  • We have a waterfall-ish development model with clearly distinguish planning, designing, implementation and testing phases
  • There is a lot of paper work to be done at every milestone
  • Features for every release are described globally (per release) and approved by TSC in the beginning of each release
  • We have 2-3 releases per year

Glossary

Feature - a high level design of new functionality that impacts multiple components. Has to be approved by subcommittees, impacted PTLs and TSC. Just technical details, no resource allocation.

Spec - a detailed level design design of a change that is planned but focused on a single component. Has to be approved by impacted project Team. Just technical details, no resource allocation.

ONAP best practice - "coding standard" (may be code related, security related, configuration related etc.) recognized by the community that should be followed. Approved by TSC, has to be followed by any new code entering the tree. Enforced for a code arriving for a review after approval.

Global requirement - best practice, non functional requirement chosen by the TSC to be applied to whole ONAP code base during a given release. Enforced since beginning of release X for whole ONAP code base. Stays forever

Assumptions

  1. DON'T BREAK ANY THE MASTER!
  2. All approved best practices are checked in gerrit review and enforce for any new code that is entering the tree
  3. Global requirements are mandatory for all projects. If project fails to deliver global requirements, it is descoped from the release (previous release containers are used).
  4. TSC defines the vision and sets the direction for ONAP project
  5. Based on recommendations from subcommittees, TSC makes the final decision whether the new feature should be approved or not.
  6. TSC makes the final decision whether the best practice should be approved or not.
  7. Architecture team consists of ONAP specialists (both on functional and implementation view) that can clearly assess whether the proposed feature is aligned with long term ONAP vision and is aligned with ONAP design principles
  8. Security Team consists of ONAP security specialists that can clearly assess whether the proposed feature does not compromise ONAP security standards
  9. All other subcommittees consists of ONAP specialists in a given domain who help to asses the proposal if it touches their domain of interest
  10. Anyone is free to propose a new feature into ONAP at any point of the time but it has to be scoped to the release before the deadline in order to be considered as active
  11. Anyone is free to assess the proposed feature from a technical point of view
  12. Anyone is free to propose new best practice at any point of time
  13. Before project meets all global requirements for a given release, it cannot make backward incompatible changes that are impacting other components.
  14. Project may release a new docker image at any point of the release (taken into account previous point)
  15. PTL is free to define additional rules and quality metrics that has to be met before the patch can be merged
  16. Anyone is free to submit any patch at any point of the time
  17. It's PTL & committers responsibility to make sure that patches they merge for a given branch obey to the rules set by TSC (best practices, release phase)
  18. PTL and committers are responsible for the project condition
  19. PTL and committers have a full control over what and when should be merged (ie. They can prevent functional patches from being merged until global requirements for their project are met)
  20. It's up to the feature/spec owner to organize resources to implement it.
  21. Features can be worked on since approval until they are ready, no matter if it takes a release of 5 years. The release is always on time just like a train, always on time. You missed one, no issues, you can release new docker as a part of the next release as soon as you are ready.

Development

  • Keep 3 releases in a year:
    • W8
    • W24
    • W43
  • Divide the release in 3 sections:
    • R - 16 Release kickoff  (example October 2020 for Honolulu right after Guilin signoff)
      • Start estimating which global requirements can be met in this release
      • Start socializing global requirements proposals with PTLs so that every one knows what has to be done in this release
      • Features and specs work is ongoing during this phase as well.
    • R - 14 Global requirements defined
      • TSC approves a list of global requirements that has to be address by the teams in this release
    • R - 10 Spec freeze   (example December 2020 for Honolulu)
      • All features and specs that are going to be implemented in this release should be approved and marked as active
      • At this point we know what we will get in this release (at most)
    • R - 5 Feature freeze  (example February 2021 for Honolulu)
      • End of accepting patches containing new features for this release
      • Start of bug fixing period
      • Release branch is being created for all participating projects
        • particularly OOM so that we can start to focus on the release package of helm charts and corresponding docker containers
    • R - 3 Beginning of RC-phase (example February 2020 for Honolulu)
      • RC every end of week
      • Bug fixes submitted for both master and release branch
      • Feature development may continue for approved specs in the master branch
    • R - 0 - final testing, release notes and Sign-off  (example March 2020 for Honolulu)
    • R - 16 of next release (example November 2020 for H release)
  • Every spec has below stages in its life cycle:
    • Proposed
      • The spec has been proposed by the author for a review by affected parties
    • Approved
      • The spec has been reviewed by interested parties and approved
      • In this point we agree that the design is good and if agreed we find the resources so it can be implemented
    • Scoped for the release
      • There is a volunteer to work on this particular spec within this release
      • The work may be continued in next release with PTL and TSC consent.
    • Implemented
      • The spec has been fully implemented and should be mentioned in the release notes
    • Dropped
      • The spec has been dropped because no one is willing to work on it or it became out of the date

Recording from 08.07.2020:

zoom_0.mp4

Please move the calendar to March to see the impact of our proposal on the release cycle.

  1. EDIT THE CALENDAR

    Customise the different types of events you'd like to manage in this calendar.

    #legIndex/#totalLegs
  2. RESTRICT THE CALENDAR

    Optionally, restrict who can view or add events to the team calendar.

    #legIndex/#totalLegs
  3. SHARE WITH YOUR TEAM

    Grab the calendar's URL and email it to your team, or paste it on a page to embed the calendar.

    #legIndex/#totalLegs
  4. ADD AN EVENT

    The calendar is ready to go! Click any day on the calendar to add an event or use the Add event button.

    #legIndex/#totalLegs
  5. SUBSCRIBE

    Subscribe to calendars using your favourite calendar client.

    #legIndex/#totalLegs

  • No labels

6 Comments

  1. Morgan Richomme

    3 releases per year does not look very reasonable for me... 2 is already very challenging (smile)

    I am also a bit puzzled with the proposal, I do not see lots of difference with the current process.

    It provides a high level very ideal roadmap. The reality of the previous releases look relatively far from this theory (freeze are rarely frozen, new docker versions provided very late in the process, integration windows compression).

    I do not see how it address the waterfall-ish development mode.

    You mention 3 sections but I can see 5 milestones. How do we manage the R-16, R-14, ... milestones. What would be the criteria? What would happen if one project is not meeting them? how to manage the summer/winter vacations?

    If we vould be stable enough to maintain a rolling release (always deployable/testable Master branch), it would be great to be able to announce new releases when new good features are available.

    There are several approaches, if you look at elasticsearch they do not communicate on the timelines/cadences but it is about every 6 weeks when there is an interesting new feature..

    On the other hand Debian releases when the stable version is good and fully tested so it may take a while between 2 versions and it is never the same duration (https://wiki.debian.org/DebianReleases).


    1. Krzysztof Opasiak

      Morgan Richomme

      So the main difference is the change of the approach. We want to switch from "planning the release" to doing the work.

      Currently in the beginning of every release we have a quite long planning period which ends with some TSC voting on what should be included in every release based on time and resources available. So first of all we want to split the technical discussion on proposed change from the staffing discussion. So when you come with the proposal you present it technical aspects to PTLs and subcommittees and when you got the approval from them you know that our proposal is fine from a technical point of view and fits into ONAP direction. The real change happens on the staffing part. What we want to change is to get rid of most of the release planning. Basically everyone is able to contribute any of approved architectural changes that the want, there is no TSC approval required for that. The TSC defines only global requirements for every release which are things like python upgrade etc. those requirements has to be met by projects in order to participate in the release. Additionally you can come  up with a proposal any time in a year and you will be able to go through the approval process and start working on it but it's going to be released when it is ready not when the release is planned.


      It's true that it's not fully fledged agile approach but we really want evolution not revolution. The agile part is lack of release planning for features and the ability to work on features at any point in the time.


      Criteria has been described above. If some project is not meeting them it's just not going to be included in this release which means that we are going to reuse their artifacts from a previous release.


      Yeah I know debian release approach and I know that the "Debian is released when it is ready" almost killed them. Now to mention that comparing ONAP to Debian is comparing Apples to Oranges or even cucumber. Debian is a Linux distro which means that their most effort is integration and packaging. They relay on the open source projects to deliver the funcitonality. ONAP on the other hand is mostly focus on development and delivering features.

      1. Morgan Richomme

        ok thanks for the clarification.

        I fully support the idea of a switch from "planning the release" to doing the work...

        3 releases in the conditions you are detailing is fine (get rid of most of the release planning, more trust on the contributors, less admin constraints). It would not be far from being a revolution (very different from the current milestones, milestone exceptions, hundreds of admin JIras, centralization of the decisions).

        It is a very good first step to a rolling release model, which will be even more revolutionary. I think it is  a good target release model regarding the type of the project.

        I just mentioned Debian as an extreme example, I agree that it is not relevant to our project even if sometimes each project may look like an independant open source project and ONAP the packaging of these projects (there is still lots of place for integration enhancements...)


  2. Catherine Lefevre

    Dear All,

    I would like to suggest a different approach where the PTLs including the Integration Team will decide what it is feasible based on a potential scope well-defined by Subcommittees and Owners .

    Some pre-requisities:

    #0  Well-defined requirements upfront - ready to be consumed by any developer and any tester.

    #1 Common services must be stabilized  

    • AAF
    • DMaaP
    • Portal
    • SDC-SDK
    • Portal-SDK
    • Oparent

    #2 API and components backward compatible

    #3 Docker containers and matching helm charts as part of the Projects delivery to ensure Self-Release (SRx)

    #4 Need more verify jobs like Job Deployer

    #5 More automated test cases so we give early feedback on any breakage sooner

    #6 Cutting branches strategy to be reviewed i.e. only cut the branch is something is delivered for the main release (not the Self-Release i.e. SRx)


    The following diagram represents a "Release Train" that represents a subset of requirements that are developed and certified - ready to be released.

    Similar to what Windows provide as "Service Pack" on top of their major release at a frequent pace and the size and content vary based on what it is certified at a specific time.


    Thoughts?

    1. Krzysztof Opasiak

      Catherine Lefevre

      I certainly agree that we need global (global jjb, python migration, security, in general all the basic stuff to just ship sth that works) requirements defined upfront but why a whole scope? This way we are always delaying each and every milestone in our process, we spend a bunch of time just talking what can we fit in the release and what not. Why don't we just spend this time on the actual development?

      The key point of the proposal above is to limit the discussion on scope definition only to the global requirements. For rest of stuff we just provide a time window where we accept code that brings in new functionality. Every one is free to work on whatever they want without any planning but:

      #1 PTL abd committers can prioritize reviews that are related to global requirements

      #2 PTL abd committers may establish their own schedule. They may want to focus on global requirements first and not accept any new features before they are fulfilled or they may decide to work in parallel. It's just up to them.

      #3 If project does not manage to achieve global requirements at the deadline we are just not including them in the release (containers from previous release are reused)

      #4 Because of #3, backward compatibility is a strong requirement


      BTW

      I fully agree with your points #4 and #5 but I think they are independent of the release model;)


  3. Christophe Closset

    Thanks for the calendar and slide comparison, it is really helpful - conceptually, it's not very different from what was proposed by Catherine, excepted for the sprint coupling

    Discussed this with Catherine here are some questions :

    •  Global requirements VS Features : Commitment to help on Best Practices, Global requ is left on PTL to handle - enforcement but how to ensure company commitment upfront VS forcing when TSC approves GR
    • How to deal with a Maintenance build in this model ? do you just make it parallel to other tracks ?
    •  How do you decide that a component is ready for say H release ? Component will self release and proclaim itself ready ? When is i
    •  Isn't there a de sync risk at some point (in say 5 release) where some components would have implemented 10 GR and Best practices, implying changes in their behavior VS some others that have less traction and are left behind ?
    •  How about common backlog management and dependency management, who owns this item ?