ObjectiveTo establish a ‘reasonably light’, efficient, well communicated, repeatable and traceable model approval process.


1. Model States:

DISCUSSION – This state means the model is proposed to be reviewed and discussed in the modeling subcommittee. The proposed model shall be captured in one (or multiple) wiki pages under the 'discussion' category of the modeling workspace for collecting and resolving comments.

CLEAN – This state means the model proposal has been officially reviewed and approved by modeling subcommittee. The model wiki page will be moved under the 'clean' category of the modeling workspace, and be referenced in the ONAP modeling specifications.

NOTE: ONAP modeling specifications can be found at ONAP Documentations (https://docs.onap.org/projects/onap-modeling-modelspec/en/latest/index.html) and ONAP wiki (ONAP Information Model - Clean and approved models across ONAP releases).


2. State Transitions

Proposal => DISCUSSION

  • The model proposer shall create modeling requirements in the 'High Level Requirements' page, and create corresponding JIRAs. It's suggested to provide use cases and relevant material for the discussions, and the requirements and JIRAs will be reviewed in the modeling subcommittee call.
  • The model proposer shall create the wiki page (contribution) under the 'discussion' category of the modeling workspace, and the proposal will be reviewed in the modeling subcommittee call
  • After the first presentation in the modeling subcommittee call, the model proposer MUST also “call for participation” of the page (contribution) via an email to the mailing list 
  • Reviews & Comments:
    • The proposal shall be presented and discussed in the modeling subcommittee call.
    • Comments should be captured directly on the wiki page (either at the bottom or preferably “in line”, see clause 3 below for the instructions)   
    • Responses must be made to EACH comment with heading:
      • <NOTED> The review comment has been dicussed and no action is required
      • <RESOLVED> The review comment has been resolved and acceptable to the reviewer
    • Review decisions are captured on the wiki, and in meeting minutes.

DISCUSSION => CLEAN

  • In order to transition to clean, the following occurs:
    • There is general consensus within the modeling team for the proposal. General consensus includes:
      • All wiki comments are discussed, and are addressed in either <NOTED> or <RESOLVED> status
      • Call for team agreement in meeting for the email poll, and captured in meeting minutes
    • Polling:
      • the Modeling Chair does a “call for approval” of the wiki discussion page to onap-modelingsub@lists.onap.org, providing a 2-week time period for collecting feedback from the community
      • People respond “Yes”, "No" (with reasons) or “Abstain” (may with reasons).
      • The goal of the poll is to gather opinions from the committee members.
    • Decision:
      • In the modeling subcommittee call after the end of the email poll, the Modeling Chair will address rough consensus of the poll:
        • If there's no objection received in the email poll and the modeling call, the contribution is approved.
        • If there are objections raised in the email poll or during the call, the objectors shall describe their comments, and the chair(s) will try to get rough consensus (based on reference: https://datatracker.ietf.org/doc/html/rfc7282). If a consensus/rough consensus can be reached, the proposal will either be approved, or will be revised for further review in the following modeling calls.
      • If the objections cannot be addressed in the end, a formal vote is conducted during the call.
        • approval requires 2/3 of the votes approving the contribution, 
        • if a member has given a response in the poll, but not join the modeling call, the response of the poll will be considered as their opinion (vote) in this formal vote, unless they change their mind
        • the vote requires at least 3 committee members to attend, if the quorum is not met, another email vote will be triggered on the mailing list for final decision
      • The Modeling Chair publishes the results to the mailing list, and record in meeting minutes
    • Following process:
      • If approved, the DISCUSSION wiki goes to CLEAN. If not approved, there is no transition
      • If the subcommittee is unable to reach consensus (e.g., members have strong objections against the decisions), those members may escalate the matter to the TSC and seek guidance or decisions.


3. Wiki Comment Examples & Handling

  • No labels

8 Comments

  1. Thanks for pulling this together Xu Yang 

  2. DISCUSSION – This state means the model is proposed to be reviewed and discussed in the modeling subcommittee. The proposed model shall be captured in one (or multiple) wiki pages under the 'discussion' category of the modeling workspace. These pages (contributions) will get reviewed in weekly meetings.  The contributor also must provide use cases, requirements and relevant material articulating why the model is required when first discussed in the weekly call.


    The highlighted statements are about the process and requirements for submitting the proposed model. These statement should be part of the Discussion State. 


    New Text: DISCUSSION – This state means the model is proposed to be reviewed and discussed in the modeling subcommittee. The proposed model shall be captured in one (or multiple) wiki pages under the 'discussion' category of the modeling workspace for colecting and resolving review comments

  3. CLEAN – This state means the model proposal has been officially reviewed and approved by modeling subcommittee. The model wiki page will be moved under the 'clean' category of the modeling workspace, and be referenced in the ONAP modeling specifications.


    There are multiple links of ONAP modeling specifications. which links and versioning?

    https://docs.onap.org/projects/onap-modeling-modelspec/en/latest/index.html

    ONAP Information Model - Clean and approved models across ONAP releases




  4. 2. State Transitions

    Proposal => DISCUSSION

    • The model proposer shall create modeling requirements in the 'High Level Requirements' page, and create corresponding JIRAs. The requirements and JIRAs will be reviewed in the modeling subcommittee call and be modified if any changes are needed.
    • The model proposer shall create the wiki page (contribution) under the 'discussion' category of the modeling workspace
    • After the first presentation in the modeling subcommittee call, the model proposer MUST also “call for participation” of the page (contribution) via an email to the mailing list 
    • Reviews & Comments:
      • The proposal shall be presented and discussed in the weekly modeling subcommittee call.
      • Comments should be captured directly on the wiki page (either at the bottom or preferably “in line”, see clause 3 below for the instructions)   
      • Responses must be made to EACH comment with heading:
        • <AGREE> agree with the comment and provide description of action,
        • <DISCUSS> no action will be taken until discussed by the team,
        • <INFORMATION> additional information provided,
        • <DEFER> no action until some later time.
      • Review decisions are captured on the wiki, and in meeting minutes.


    Remove the "...weekly.."  Rational: today, modeling subcommittee call is weekly. In the future, the call may have different reoccurance.  Make this change globally.


    • <AGREE> agree with the comment and provide description of action,
    • <DISCUSS> no action will be taken until discussed by the team,
    • <INFORMATION> additional information provided,
    • <DEFER> no action until some later time.


    too many category heading: it should be simple such as

    • <AGREE> agree with the comment and provide description of action,
    • <NOTED> The review comment has been dicussed and no action is required
    • <RESOLVED> The review comment has been resolved and acceptable to the reviewer.
    1. Xu suggest to remove <AGREE>

  5. DISCUSSION => CLEAN

    • In order to transi Bold to tion to clean, the following occurs:
      • There is general consensus within the modeling team for the proposal. General consensus includes:
        • All wiki comments are discussed, and further actions decided
        • Call for team agreement in meeting for the email poll, and captured in meeting minutes.
      • The Modeling Chair does a “call for approval” of the wiki discussion page to onap-modelingsub@lists.onap.org, providing a 2-week time period for approval of the page. People respond “Yes”, "No" (with reasons) or “Abstain” (may with reasons). The goal of the poll is to gather opinions from the committee members.
      • In the following modeling subcommittee call after the end of the email poll, the Modeling Chair will address rough consensus of the poll:
        • If there's no objection received in the email poll and the modeling call, the contribution is approved.
        • If there are objections raised, the objectors shall describe their comments, and the chair(s) will try to get rough consensus (based on reference: https://datatracker.ietf.org/doc/html/rfc7282). If a consensus/rough consensus can be reached, the proposal will either be approved, or will be revised for further review in the following modeling calls.
        • If the objections cannot be addressed in the end, a formal vote is conducted during the call. Approval requires 2/3 of the votes approving the contribution, and the vote requires at least 3 committee members to attend.
          • Note 1: if the quorum is not met, another email vote will be triggered on the mailing list for final decision
          • Note 2: if a member has given a response in the poll, but not join the modeling call, the response of the poll will be considered as their opinion (vote) in this formal vote, unless they change their mind
        • The Modeling Chair publishes the results to the mailing list, and record in meeting minutes. 
      • If approved, the DISCUSSION wiki goes to CLEAN. If not approved, there is no transition
      • If the subcommittee is unable to reach consensus (e.g., members have strong objections against the decisions), those members may escalate the matter to the TSC and seek guidance or decisions.


    The process of transition from DISCUSSION => CLEAN is very confusing.  The text is mixing “call for approval” vs The goal of the poll is to gather opinions from the committee members.

    mixing 2/3 votes with consensus

    mixing the formal vote result with consensus and than escaltion with TSC.

    Perhaps a simple two steps process such as


    1) step #1: a poll to get rough idea that the discussion page(s) is/are ready to approval.  The poll can be conducted during subcommittee call

    2) step #2: “call for approval”: 

    • The Modeling Chair does a “call for approval” of the wiki discussion page to onap-modelingsub@lists.onap.org, providing a 2-week time period for approval of the page. People respond “Yes”, "No" (with reasons) or “Abstain” (may with reasons).
    • Approval requires 2/3 of the votes. Note: the process, voting members, etc. must be consistence with TSC process.
    • Each of "NO" vote must have one following marking:
      • <AGREE> agree with the comment and provide description of action,
      • <NOTED> The review comment has been dicussed and no action is required
      • <RESOLVED> The review comment has been resolved and acceptable to the reviewer.


    1. Zu's comment on 3.28 call:

      -suggest to describe the (minimum/maximum) time needed between the poll and the vote

      -make it clear the differences between polling and voting

      (Andy's opinion is modeling subcommittee can only poll as we can only make suggestions, not decisions)

      (Kenny seems to be ok with call it "voting")

  6. The governance is important to support the following principles:

    1. The specifications are serving for the running code and is supposed to reflect the code.
    2. The gonvernance process should encourge participation and achieven consensus whenevern possible.
    3. The vote results should be reflecting the majority of the active contributing members.

    And based on the above principles, I suggest the following amendments for your kind considerations:

    1. Add the model status of archive to identify those archive models that do not have corresponding code implementations. And for the transition from clean->archive, if it is not implemented in the community code after two consecutive releases.
    2. It is recommended to limit the qualifications of the voting members of the model committee, and encourage participation in the meeting more. 
    For example, if a model committee member fails to attend the meeting for two consecutive times, they will lose the right to vote at the third meeting. Model committee members who attend 4 consecutive model committee meetings can regain voting rights at the next meeting. Each meeting and online voting requires 3/2 of the legal voting members to participate in the result to be valid. If the number of valid votes exceeds half, the vote will be passed.