PROPOSAL - following feedback from the ONAP TSC 2019-02-07 the POC definition for Dublin


Proof of concept

POC DEFINTION:

Proof of concept (PoC) is a realization of a certain method or idea in order to demonstrate its feasibility,[1] or a demonstration in principle with the aim of verifying that some concept or theory has practical potential. A proof of concept is usually small and may or may not be complete.(source: Wikipedia)


POC GUIDELINES:

  1. SPONSOR - Every POC shall have a named sponsor for the feature whom acts as spokesperson at all checkpoints/meetings.
  2. NON BLOCKING - POC (only) related defects shall not block the current release and not create any backward incompatibility
  3. COMPLIANCE - POC should be fully compliant with ONAP development guidelines, including development procedures.
    1. License/Vulnerabilities - POC development that includes entry points in the release version must comply with license scans and vulnerability fixes.
    2. Proprietary Components - PoC may include proprietary software or 3rd party software. It should be explicitly stated that the PoC is not contributing its software otherwise it is subject to CLAs. Note: The 3rd party software should comply with security guidelines otherwise there could be a backdoor vulnerability into ONAP.
    3. Security - PoC development shall be compliant to security guidelines and incorporate the latest security fixes so as not to create a security vulnerability.
    4. Performance - PoC should not impact the performance of the main code; this should be implied as it is kept in a separate Repo. The PoC development team should have an eye on ONAP performance if in the future it might be evolved to be incorporated into the ONAP platform.
    5. Back Doors - A POC should not introduce a back-door to the stable features so it must comply with security, code coverage, and license scans requirements.
    6. Resources - The PoC should perform an assessment to see that adequate resources are available such as PoC toolchains (Jenkins, Sonar, NexusID, LF resources, Test coverage resources etc).
    7. Approval for inclusion - The TSC or Use Case S/C shall approve the PoC to proceed as part of the current release.
  4. INDEPENDENT OF MAIN RELEASE - PoC are not part of the (current) ONAP release product.
    1. Documentation - The PoC may have documentation in a wiki developed by the community; but will not be incorporated into the official release documentation.
    2. Independence - The PoC shall be kept independent of the official (current) release. The PoC shall not introduce any new requirements in the current release.
    3. Integration/Testing responsibility - There is no responsibility from the integration or test teams to test the PoC S/W. The testing, integration and use of the PoC shall be in the purview of the PoC development team.
    4. PoC Release Notes - PoC release notes may be developed by the PoC team, they may include details of functionality available, known issues, and known limitations/incomplete features to keep of track development issues as they arise and serve as a communication vehicle for those evolving the PoC in the future.
  5. CODE INDEPENDENCE - The PoC software shall be kept separate and independent of the main ONAP branch/repository.
    1. Common Code Usage - PoC software using common code, does not change the treatment of the common code with regards to completeness, S3P, documentation and other common code expectations.
    2. Separate Branch - PoC should be kept in separate repository branch. There should not be PoC S/W introduced into the main ONAP software branch. This prevents complications in testing and integration of the ONAP software in the main branch. The PoC should not modify any existing (current release) core ONAP components.
    3. Evolutionary Platform Software - EXCEPTION CASE: Under the blessing of a PTL for the platform component, there might be a situation where a PoC is demonstrating something to evolve the Platform component (e.g. SO, VID, VNF-SDK etc) and some of the PoC software may be included with in the CURRENT release. Experience has shown that it was important to merge PoC code in the current release to make the task manageable. Note: it is expect this would be most useful in large, multi-release PoCs.
  6. PROCESS - The PoC may follow a process during development.
    1. Milestones - The PoC team may follow some sort of development process. It is suggested that they use the Mx milestones/gates as the release progresses or some similar process. Note: it is expected that using a process would facilitate formal inclusion of the PoC in the subsequent ONAP release(s).
    2. Final Approval - The final approval of a PoC shall ultimately be determined by the sponsor/leader of the PoC (see Point #1). It is suggested that a "report" of how the PoC went to socialize and inform the TSC or other relevant teams.
    3. PoC Overlap - It is recommended that the PoC team socialize their PoC so that other teams working on Use cases and PoCs are aware of potential cross-interactions that might occur and overlap in scope and functionality.
  7. SCOPE - A PoC can be used to demonstrate functionality related to a project concept, a Use Case, functional or non-functional requirements, project specific extensions.
    1. Definition - It is the responsibility of the PoC team to define the scope and extent of the PoC. 
    2. Project Review - the PoC team shall review the PoC with impacted sub-committees (such a security and modeling S/C) and projects PTLs (such as A&AI and SDC).
  8. POC INCORPORATION - The point of a PoC is to demonstrate something (hopefully) useful. PoCs teams that wish to incorporate the PoC code into the main repository should follow the standard procedures for introducing new functionality and use cases in subsequent ONAP releases
    1. Feature Proposals - There exists a standard procedure new functionality introduction leading up to M0 shall be followed by a PoC in subsequent releases by the PoC team if they wish to incorporate PoC Software into the main repository.
    2. Partial PoC Introduction - The original PoC scope may be altered from what is introduced in a future release. An ideal case might be that a PoC software might serve as seed code, but it does not have to. Lessons learned can guide what would get incorporated or proposed in the future release.
    3. Merge Responsibility - It will be the responsibility of the PoC Team to properly merge the PoC code into the main-line branch (S/W repo) if it becomes part of the future release. Merging should follow compliance principles (described in point #3 above).
    4. Proprietary Components - If the PoC used proprietary components or software, for formal including into ONAP it would need to be excluded in the final merge because ONAP is an open source initiative. Note: A PoC using proprietary components are "on" or "with" ONAP; a contributed PoC (without proprietary components) are said to be "in" ONAP.



==-=-=-=-=-=-=-=-=-=-=-CUT LINE, Previous content preserved for continuity... Delete after TSC review-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=


POC is equivalent to "Tech Preview" or "Experimental"


Guidelines

  1. Every POC has a named sponsor for the feature whom acts as spokesperson at all checkpoints/meetings
  2. remove POC feature development is second priority to committed feature/tech debt development efforts. 
  3. Dependencies on other projects for POC functionality should be clearly communicated as POC development
  4. POC (only) related defects cannot block the release
  5. POC release notes should include details of functionality available, known issues, and known limitations/incomplete features
  6. Integration and test resources are primarily for the committed features and should not be used for POC until either
    1. primary functionality is complete 
    2. testing on the primary functionality is blocked - and cycles are available while blocker gets resolved
  7. POC issues are secondary topics at PTL or TSC meetings
  8. POC development can waive "product completeness" requirements - like S3P, Localization/I18N
    1. Basic documentation is expected so others can access the feature - full commercial grade documentation is not expected
  9. POC development must comply with license scans and vulnerability fixes
  10. POC may be promoted to fully supported feature prior to the end of the release with approval of the TSC
  11. POC features may be withdrawn from the release at any time with notification to the TSC from the sponsor 
  12. POC functionality is targeted for full feature in a specific future release


ISSUES:

  • How to treat "use case" POCs that use the common code base? Defects in a use case could impact other functionality...

12 Comments

  1. Brian Hedstrom

    We need to define the lifecycle stages of a POC to understand the process a POC needs to go through to mature into a feature release (as a non-functional, functional, use case, S3P, etc addition).

    For example, what are the gating criteria for a POC to be promoted to a release feature? Can a POC be promoted in a current release cycle or does it need to be promoted for the next release as a non-functional, functional, use case, S3P?

    What is the criteria for a release feature being demoted to a POC? 

    If a POC does not complete during a release cycle/release timeframe, does it automatically become a POC for the next release, or can it enter the next release as a non-functional, functional, use case, or S3P?

    If a POC does not get promoted, is withdrawn, or has no intention of evolving into a release feature, how does it reach an end of life? Can POCs be designated as Experimental only (can there be different classifications of POCs)?

  2. Srinivasa Addepalli

    It is good to have some guidelines for JIRAs too.

    Have a label as 'POC' for JIRA stories related to POCs

    And fix version NA.

    Srini

  3. Keong Lim

    See also POC defintion (Copy) because people are using the Copy page instead of the original page.


    1. Keong Lim

      Benjamin Cheung , Alla Goldner

      It appears that the Copy page has been deleted. Everyone who made comments/edits should redo them here.


  4. Benjamin Cheung

    Brian Hedstrom  - These are good and important questions. We had also entertained them at the PoC definition meeting that we had last week. We concluded that it is up to the sponsor to follow a process (or not); and for the sponsor/lead to determine when the PoC had reached a point where they could declare "success" (See Point #6 in our definition). The team does not want to FORCE the PoC team to follow a specific process during development. When the team feels that there is something meaningful to contribute, the PoC would then follow the "standard" process to propose a new requirement or use case for inclusion in the following release(s). If a PoC "straddles" multiple releases, it is up to the PoC sponsor/leader to determine when to submit parts of the PoC for inclusion (this is all stated in Points #6 and #8 above).


    Srinivasa Addepalli  - I discussed this with the team July 16 2019, and we felt this is too specific of a rule. Of course, I get what you are saying and it certainly makes sense for people to have the JIRAs labeled as POC and Fix version to NA. We didn't want to force people working on PoCs to such specific rules just for a PoC definition.


    -Ben and Alla.

  5. Viswanath Kumar Skand Priya

    Need clarity on the following :

    3g says TSC needs to approve POC to be part of current release and 4b says POC code will be kept independent of official release. It would be good if the explanation is made lot clearer.

    5b talks about maintaining separate branch for PoC, however 5c talks about POC may be merged to current release branch. So should I conclude that, the decision to maintain the POC code in separate branch vs release branch will be made by respective PTLs only ?

  6. Pierre Close

    I would like some more clarifications on the following:

    3b. Proprietary Components - PoC may include proprietary software. It should be explicitly stated that the PoC is not contributing its software otherwise it is subject to CLAs.

    8d. Proprietary Components - If the PoC used proprietary components or software, for formal including into ONAP it would need to be excluded in the final merge because ONAP is an open source initiative. Note: A PoC using proprietary components are "on" or "with" ONAP; a contributed PoC (without proprietary components) are said to be "in" ONAP.

    • Why would a PoC include proprietary software? How would this be merged into ONAP code later if proprietary code is present?
    • The part "It should be explicitly stated that the PoC is not contributing its software" in 3b. is not totally clear to me either.
    • 8b. Does this mean that a PoC could live along with ONAP, but not merged into ONAP?

    6c. PoC overlap should be moved to 7c. since it is about scope overlap, not process.  Now, if moving it does not seem to make sense, I would add a note in either 7a or 7b indicating that the scope shall be verified so that no overlap exists with an existing feature or with a feature to come and already approved for release.

  7. Benjamin Cheung

    • Why would a PoC include proprietary software?

    ANSWER: It is very conceivable that a vendor or company would want to try out a test using proprietary software to see if they might wish to see how it works before they open source it .

    • How would this be merged into ONAP code later if proprietary code is present?

    ANSWER: This is answered in Point 8d. Those proprietary code content must be removed before open sourcing. It is up to the PoC team to handle this. The ONAP firm and fixed rule is that you can't contribute proprietary software which is what is stated in point 8d.

    • The part "It should be explicitly stated that the PoC is not contributing its software" in 3b. is not totally clear to me either.

    ANSWER: If a PoC team feels that they must use Proprietary software (or indeed maybe that is the point of the PoC), the TSC felt that it would be wise to state that. This also reinforces to the PoC team that they are cognizant that they cannot contribute their Proprietary software unless they intend to make it "public"; otherwise it would be subject to corporate license agreements (CLA)s. ... does that help? or do you have a more specific question in mind. ... the "this isn't clear to me" is a vague question.

    • 8b. Does this mean that a PoC could live along with ONAP, but not merged into ONAP?

    ANSWER: Separate Independent Repo: (PoC with Proprietary S/W using ONAP code)

                      "Main" branch ONAP Repo: (without PoC code)

    Benjamin Cheung

    1. Pierre Close

      Thanks Benjamin Cheung for your reply.

      • Why would a PoC include proprietary software?

      ANSWER: It is very conceivable that a vendor or company would want to try out a test using proprietary software to see if they might wish to see how it works before they open source it .

      So this means that a vendor or company, who has developed proprietary software internally, is willing to test it with ONAP as a PoC with the sole purpose of open sourcing it, is this correct?  That's at least what this sentence seems to explain: "This also reinforces to the PoC team that they are cognizant that they cannot contribute their Proprietary software unless they intend to make it "public".

      I think we need to classify separately

      • proprietary software to be open sourced in the form of a PoC for ONAP integration, and
      • proprietary software to be used as a dependency for building a PoC, either in plain source (which cannot be merged into ONAP source code as you mention) or as a 3rd party library (whose licensing model would certainly prevent its use with ONAP) – and this is this second category that I was relating to

      Based on this distinction, I think it is really important to clearly state how proprietary software is meant to be used: as testing for open sourcing purpose, or as 3rd party dependency. Licensing model (usually closed source) of 3rd proprietary software is incompatible with ONAP licensing (if it was, it would be open source).  And, more generally, even non proprietary software like GPL-based software is also non compatible (unless under really specific exceptions).  I therefore believe that indicating the licensing model (proprietary with the purpose of open sourcing, or 3rd party license type) must be indicated.  We do not want non-Apache 2.0 source code inside ONAP as much as possible, excepted for the exceptions granted by the TSC and based on recommendations originating from the IP Legal subcommittee.

  8. Srinivasa Addepalli

    I get a feeling that we are talking about two different types of PoCs (in regards to proprietary software)

    1. POC use case :  In this case, ONAP is used along with proprietary software (which is external to ONAP). In this case, a given company can be involved and hence proprietary software along with ONAP is okay.
    2. POC feature enhancement : In this case, it is internal to ONAP as changes are required in ONAP source code base.  For example: "K8S support" was PoC in R3 and it is made main stream in R4.   In this case, I feel that there would be a need for collaboration among companies. Hence, I think that only Apache 2 license shall be allowed. 

    Srini


    1. Pierre Close

      Srinivasa Addepalli I agree

  9. Fei Zhang

    In  COMPLIANCE

    • Is 'Back Doors - A POC should not introduce a back-door to the stable features so it must comply with security, code coverage, and license scans requirements.'

    covered by related

           a. License/Vulnerabilities - ... and

           c. Security -....