TABLE OF CONTENTS
MEETING LOGISTICS
When:
7:00am to 8:00am
(GMT-07:00) America/Los Angeles
Repeats: Weekly on Thursday
Where:
ONAP4 https://zoom.us/j/519627903
OVERVIEW
PNF SOFTWARE UPDATE INTRODUCTION
PNF Software upgrade is one aspect of Software Management. The purpose is to modify the software running on the PNF. This could be used to update the PNF software to a newer or older version of software.
PNF SOFTWARE UPDATE TOPICS
PNF Software update includes:
- Support in-place software update for PNF.
- Define generic building blocks, which can be used for workflow design in the design time.
- Identify gaps and amend ONAP components to support different flows for software management and configuration (e.g., with/without external controller, different ONAP controller options, different adaptors in ONAP controllers).
- Support External Controller (e.g., EMS) as the main entity responsible for carrying out the detailed vendor-specific software upgrade steps.
- Enhance precheck and postcheck steps in the upgrade process by vendor-specific rules. Define procedures involved in the design time and run time for specifying these rules.
- Enhance PNF modeling needed for PNF software upgrades (e.g., expected software version) and define processes in ONAP components (e.g., SO, A&AI, DCAE, and ONAP Controllers) for successful/unsuccessful software upgrades.
- Demonstrate single PNF upgrade and batch upgrade.
BUSINESS DRIVERS
This section describes Business Drivers for this Use Case.
Executive Summary - PNF software updates are routine for network upgrades to support new features, improve efficiency or increase capacity on the field, and to eliminate bugs. This use case positions ONAP as a vantage point in orchestrating and managing PNF software upgrades inline with the business and service objectives.
Business Impact - Deployment and orchestration of new network services over both VNFs and PNFs in a model and software driven way simplifies the network management. As 5G networks will host a large number of PNFs from multiple vendors, streamlining service upgrades that involve PNF software changes through ONAP will reduce the OPEX substantially.
Business Markets - Carriers both in the mobile and fixed-line space host PNFs at the edge of the network. New 5G deployments as well as legacy 4G systems in the mobile carrier space should be considered as target markets.
Funding/Financial Impacts - Orchestrating PNF software updates via an ONAP deployment will enable better service planning, faster introduction of new network services over field-deployed PNFs, and reduce the operational costs.
Organization Mgmt, Sales Strategies - Harmonizing PNF and VNF software management in a model and workflow driven manner is essential in 5G systems where PNFs will continue to exist in large numbers and they are expected to have more frequent software upgrades (as they will have more capabilities that can be controlled or upgraded via software). Thus ONAP can be the "go-to" solution if this harmonization can be done successfully.
ASSOCIATED PRESENTATIONS:
PNF SOFTWARE UPGRADE FLOW
SOFTWARE UPGRADE FLOW FOR CASABLANCA
Bold items on the right will be tested (steps 6, 7, 10, 12, 14, 15).
LADDER FLOW DIAGRAM FOR PNF SOFTWARE UPGRADE:
DETAILED STEP DESCRIPTION
STEP 5: Pick W/F, Execute Work-flow
The Operator at the VID can select an appropriate work-flow which uses PNF Software Upgrade and execute the work-flow
STEP 6: Request ONAP Controllers & A&AI Query
SO identifier the appropriate PNF Controller
SO verifies and queries A&AI for the appropriate PNF entry
STEP 7: PNF Software Upgrade Pre-Check
SDN-C working with the External Controller performs Software upgrade Pre-Checks.
STEP 8: PNF Software Upgrade Precheck
[Vendor Specific] Apply checking rules, collect necessary data for pre-checking.
STEP 9: Retrieval Request/Response
[Vendor Specific] External Controller retrieves information from PNF.
STEP 10: Software Upgrade
Software upgrade command is issued via Ansible exchange to the External Controller
STEP 11: S/W Download & Activate
[Vendor Specific] Software download and software activation is performed between external controller and PNF.
STEP 12: Post Check
Post checks are requested by Controller to the External Controller via an Ansible exchange.
STEP 13: Check Rules
[Vendor Specific] Checking rules are applied. Information is collected and comparisons are done.
STEP 14: Result Notification
Success or fail result notifications are sent from external controller to SDN-C
STEP 15: Update A&AI Entry
A&AI Entry of the PNF is updated with the appropriate software versions.
OVERALL PROCESS FOR CASABLANCA:
- PNF upgrade process in Casablanca will follow the similar procedures used for VNF in-place upgrade in Beijing. Main differences:
- Extended payload parameters for PNF in NB API for SDNC (e.g., PNF Identifier, PNF specific rules, External Controller info)
- PNF specific DG branches that are processed based on the presence of PNF specific parameters
- PNF upgrade specific ansible playbooks
- Ansible adaptor that accepts External Controller end point as a parameter from API calls on interface 6.
- External controller that executes playbooks received from ansible adaptor.
DETAILED PROCESS:
Interface in the northbound of SDNC:
SO (or test script) ↔ DMAAP ↔ SDNC
Interfaces within SDNC
DmaapListener ← (REST) → LcmAPI - DirectedGraph – AnsibleAdapter ← (REST) → Ansible Server
Interface in the southbound of SDNC
SDNC (i.e., Ansible Server) ← (ssh) → External Controller ← (ssh or netconf) → PNF
PRE-CONDITIONS FOR TESTING:
- DMaaP is up and running
- A&AI is up and running with valid PNF entry
- SDNC is up and running
- Installed ansible playbooks for precheck, upgrade, postcheck on SDNC
- Q: how do we install playbooks? (need link to the process) (playbooks come prepackaged with SDNC and should be committed before code freeze)
- Q: how do we associate playbooks with the PNF? (In Casablanca playbook names will reside in SDNC configuration file. In Dublin this can be refined as below)
- A: pnf-name → PlaybookName
- PlaybookName := <equip-vendor, equip-model, sw-version>
- PlaybookName can be constructed via A&AI queries using pnf-name (globally unique identifier)
- A: pnf-name → PlaybookName
- Installed DGs to SDNC for the NB API of SDNC about lifecycle management (need links to SDNC API and to the process of installing DGs here)
- Installed ansible playbooks for precheck, upgrade, postcheck on SDNC
- EM and PNF emulators are up and running at manually assigned IP addresses and ports (eventually PNF PnP use case emulator and this emulator should be merged to have a full picture)
- Test script is up and running (should be able to pub/sub to DMaaP)
- Q: Is this homed at SO or as a separate test module?
LCM API PAYLOAD:
- pnf-flag: boolean ("VNF" if pnf-flag == false, "PNF" if pnf-flag == true)
- Used to pick up the correct processing branch of DG
- pnf-name: string (globally unique PNF identifier that matches to the field name in A&AI for the PNF)
ipaddress-v4-oam : string (IPv4 ( IPv6 will use separate field?) address for External Controller)
Precheck payload SoftwareUpgrade payload Postcheck payload "payload": "{\"pnf-flag\":\"true\", \"pnf-name\": \"5gDU0001\",\"pnfId\": \"5gDU0001\", \"ipaddress-v4-oam\": \"EC_ip\",\"oldSwVersion\": \"v1\", \"targetSwVersion\": \"v2\", \"ruleName\": \"r001\", \"Id\": \"10\", \"additionalData\":\"{}\"}"}}
"payload": "{\"pnf-flag\":\"true\", \"pnf-name\": \"5gDU0001\",\"pnfId\": \"5gDU0001\", \"ipaddress-v4-oam\": \"EC_ip\",\"oldSwVersion\": \"v1\", \"targetSwVersion\": \"v2\", \"Id\": \"10\", \"additionalData\":\"{}\"}"}}
"payload": "{\"pnf-flag\":\"true\", \"pnf-name\": \"5gDU0001\",\"pnfId\": \"5gDU0001\", \"ipaddress-v4-oam\": \"EC_ip\",\"oldSwVersion\": \"v1\", \"targetSwVersion\": \"v2\", \"ruleName\": \"r102\", \"Id\": \"10\", \"additionalData\":\"{}\"}"}}
- Note that:
- 1) playbookname is not needed, since it is configured in sdnc_container:/opt/onap/sdnc/data/properties/lcm-dg.properties to your playbookname prefix
2) pnf-name is also not needed(recommend to keep it), since vnf-id is regarded as a pnf-name when pnf-flag is true
3) pnfId is needed (in huawei's playbook) , since playbook will use it
4) Id is needed (in huawei's playbook) , since playbook will use i
5) ipaddress-v4-oam is needed, it is the ip address of EC, since it will be translated into NodeList array in ansible server.
DG PROCESS:
- Detect pnf-flag to proceed to the PNF branch
- Map pnf-name to PlaybookName
- Execute ansible module (pass PlaybookName and ipaddress-v4-oam as parameters)
- Post result on DMaaP
ANSIBLE MODULE AND ANSIBLE SERVER IN SDN-C:
- Ansible module passes PlaybookName and EC_IPaddr to Ansible Server over restconf interface
- ipaddress-v4-oam comes from NB API payload, PlaybookName is constructed from pnf-name during DG process (need to double check this)
- Ansible server playbacks the ansible playbook and issues ssh commands to External Controller
EXTERNAL CONTROLLER:
- Receives ssh commands regarding software management and sends vendor specific instructions to PNF.
- Note that:
- 1) the remote access to EC must be configured in Ansible_Inventory
- 2) the ip address must be provided in the LCM payload
PNF SUPPORT:
- This is an emulator that support vendor specific instructions.
DEVELOPMENT STATUS:
JIRA ITEM | DESCRIPTION |
---|---|
SDN-C 424: Create new directed graph (DG) for 5G PNF pre-check, upgrade and post-check | |
SDN-C 425: Create new module on SDNC to talk to EC (External Controller) using Ansible | |
CCSDK-464 Create and push Ansible playbooks for 5G RAN PNF pre-check, software upgrade and post check | |
INT-629 Setup environment to integrate and test software upgrade for 5G RAN PNF | |
INT-630 Integrate emulator for EC and PNF into ONAP test environment | |
SDN-C 426 Define north bound API payload updates for EC/PNF |
QUESTIONS & ANSWERS:
- Why not Netconf?
- For Casablanca, we are utilizing most of the workflows already exist in VNF in place upgrade. Eventually, netconf support will/might be needed.
- What happens if upgrade fails due to PNF, EM or even SDNC?
- A rollback mechanism should be defined. Depending on where the failure is occurring, how to detect it, how to rollback and how to communicate the rollback (e.g., during failure or post-recovery) need more detailed analysis.
- Why doesn't ONAP (i.e., SDN controller) directly handle PNF configuration and upgrades?
- There are too many complexities potentially involved in 5G PNF upgrades. EMS simplifies the processes in ONAP substantially and allows for easier adoption.
- We plan to support various options including with External Controller, without External Controller, hybrid (e.g., some workflows with direct PNF access, some workflows with External Controller).
- Options for determining which Protocol the PNF supports (Ansible, NetConf, Chef)
- Option (1): Directed Graph per Protocol - for example, one dedicated directed graph for Ansible and a different DG for Netconf each with their own SO building blocks. Multiple SO building blocks could be used for operations (new SO building blocks would need to be developed). The SO to Controller to API. This would require multiple yang files to work. WORK-EFFORT: x. PROS/CONS: y. TECHNICAL DEBT: z.
- Option (2): LCM API based - In the LCM API for S/W Upgrade w/ one Directed Graph but the API payload has an optional parameter that specifies the DG which is executed. This was used for VNF vs PNF S/W upgrade. This was used in R3/Casablanca. Wanted to pass an Ext. Mgmt IP@ in the API payload. In the LCM API specify whether it is PNF/VNF. WORK-EFFORT: x. PROS/CONS: y. TECHNICAL DEBT: z.
- Option (3): Self-Service API - A self-service API is being developed by the CDS team. This API allows you to specify the work-flow to use when that API is used. The API remains the same, but a parameter indicates which work-flow should be used. WORK-EFFORT: x. PROS/CONS: y. TECHNICAL DEBT: z.
- Alignment with VNF upgrades
- Harmonized Solution. One Workflow/One API for everything. VNF/PNF (different PNFs)
EVOLUTION OF PNF SOFTWARE UPGRADE
This section describes the potential evolution of PNF Software upgrade. There are many potential software management functions that could be introduced into the ONAP ecosystem. These functions represent potential future enhancements that could be explored.
SOFTWARE MANAGEMENT FUNCTIONS
The following table describes some of the Software Management Functions
FUNCTION | ICON | DESCRIPTION |
---|---|---|
Software Download | Software is downloaded to the NF (PNF or VNF) with the intention of updating or changing the currently running software on the NF. | |
Software Activation | After Software is downloaded, it is activated and becomes available to provider service. | |
DUBLIN ENHANCEMENTS
The following table summarizes some potential Dublin (R4) enhancements that will be developed and evaluated.
ENHANCEMENT | DESCRIPTION |
---|---|
PNF Reporting | PNF sends information regarding its Software Version directly to ONAP through a VES event. |
Design Time Components | PNF software update workflow design, artifact onboarding (e.g., playbooks), upgrade rule list, utility of software version list. |
PNF Package Update | A software update may require a change in the PNF Package before the software update or post update or both. |
Software Update Coverage Analysis | Find out what services, control loops, etc. are impacted during and post software update before the PNF software update |
Post/Pre PNF SW Update Event Workflows | Deregistration, re-registration |
Coherence across VNF and PNF in-place software upgrade processes | Currently PNF and VNF entries in A&AI diverge that requires DG branching. Can we unify the process for both? |
R5 EL ALTO
The following table summarizes some potential El Alto (R5) enhancements that will be developed and evaluated.
ENHANCEMENT | DESCRIPTION |
---|---|
PNF Reporting | PNF sends information regarding its Software Version directly to ONAP through a VES event. |
Design Time Components | PNF software update workflow design, artifact onboarding (e.g., playbooks), upgrade rule list, utility of software version list. |
PNF Package Update | A software update may require a change in the PNF Package before the software update or post update or both. |
Software Update Coverage Analysis | Find out what services, control loops, etc. are impacted during and post software update before the PNF software update |
Post/Pre PNF SW Update Event Workflows | Deregistration, re-registration |
Coherence across VNF and PNF in-place software upgrade processes | Currently PNF and VNF entries in A&AI diverge that requires DG branching. Can we unify the process for both? |
VID Support | Execute the e2e run time workflow starting from VID command. In Casablanca, the process exists starting from the SDNC Northbound. |
NetConf Support | Current workflow is driven by Ansible playbooks. |
SOFTWARE UPGRADE FLOW FOR DUBLIN
The following diagram shows another flow that is supported in the R4/Dublin release.
It compliments, the R3/Casablanca release PNF Software Upgrade Flow
A notable difference between the two flows is a focus on ONAP communicating directly with the PNF.
DETAILED STEP DESCRIPTION for R4/Dublin
STEP 1: Pick W/F, Execute Work-flow
The Operator at the VID can select an appropriate work-flow which uses PNF Software Upgrade and execute the work-flow
STEP 2: Request ONAP Controllers & A&AI Query
SO identifier the appropriate PNF Controller
SO verifies and queries A&AI for the appropriate PNF entry
STEP 3: PNF Software Upgrade Pre-Check
SDN-C working with the External Controller performs Software upgrade Pre-Checks.
STEP 3b: Retrieval Request/Response
Retrieve information from PNF.
STEP 4: Software Upgrade
Software upgrade command is issued via PNF supported communication protocol and exchange to the PNF
SO: Requests S/W Upgrade
STEP 4b: S/W Download & Activate
Controller Performs software upgrade steps including download of SW to the PNF and then activates it.PNF.
STEP 5: Post Check
Post checks are requested by Controller to the External Controller via an Ansible exchange.
STEP 5a: Check Rules
[Vendor Specific] Checking rules are applied. Information is collected and comparisons are done.
STEP 5b: Result Notification
Success or fail result notifications are sent from external controller to SDN-C
STEP 6: Update A&AI Entry
A&AI Entry of the PNF is updated with the appropriate software versions.
ROADMAP
This wiki page represents a "living" page and landing spot to learn about PNF Software upgrade. The roadmap captures developments that have been made and ones that are planned in future releases.
TEST CASES AND STATUS
# | Test Case Description | Test Status |
---|---|---|
1 EC registration test | Ensure ONAP controller can find out the EC | NOT NEEDED (Specify in Inventory and payload) |
2 PreCheck test | Ensure the EC can receive the precheck command from ONAP controller | PASS |
3 SwUpgrade test | Ensure the EC can receive the software upgrade command from ONAP controller | PASS |
4 PostCheck test | Ensure the EC can receive the postcheck command from ONAP controller | PASS |
5 New SW version report test | Ensure the software upgrade completion event can be reported to DCAE | DEFERRED (Moved to Dublin) |
6 New SW version update test | Ensure the new software version can be updated in A&AI | NOT TESTED |
7 E2E Workflow execution test | Ensure the e2e software upgrade workflow can be executed at run time | DEFERRED (Moved to Dublin) |
NB LCM Test Result
1) UpgradePreCheck
2) UpgradeSoftware
3) UpgradePostCheck
21 Comments
Rebecca Lantz
Given a PNFID, how does the ONAP Controller find the External Controller to communicate with?
How are new versions of PNF artifacts (data model, counter definitions, alarm definitions, etc) propagated and used by ONAP components? When and how does this happen?
yaoguang wang
Ans1: Firstly, the external controllers (EC) are provided by vendors. Even if there are many instances, ONAP should see just one, like a ‘virtual ip’. Secondly, config info is required inside the ONAP controller’ adaptor, mapping from vendor name to location information, while the vendor driver name is modeled and specified as input of execution workflow in SO.
Ans2: In R3, those PNF artifacts as well as PNF software packages, vendor specific checking rules/scripts need to be pre-provisioned to EC. However, these things may be in the PNF package from SDC in R4+. Information propagated from ONAP to EC is the expected S/W version, rule identifiers and necessary parameters used in checking rules.
Rebecca Lantz
Thanks yaoguang wang, sorry for slow response. I somehow missed your reply on this...
WRT Ans1: ONAP will be multi-vendor. I think you are saying each vendor should deploy or register a single EC with ONAP. So for a given PNF, A&AI will hold the vendor and the EC for that vendor will be well known to ONAP (somehow). Do I have it right? I'm also confused about the "mapping from vendor name to location information" - this is something that SDNC will be doing?
WRT Ans2: In Casablanca in order to use closed loops that make use of alarms, counters, etc. we will be manually attaching the necessary artifacts to the service before it is instantiated. I suppose that if these artifacts change on a software upgrade it would need to be hand-managed. This would be something to address in Dublin when the PNF package is better defined.
Rebecca Lantz
Hi yaoguang wang - I'd like to clarify answers based on latest information here in wiki - please let me know if this is incorrect.
Ans1: ONAP Controller is provided the IP address (and any other connectivity needed) to communicate with the External Controller. I guess it would be passed in by operator at VID. Wiki states: "Extended payload parameters for PNF in NB API for SDNC (e.g., PNF Identifier, PNF specific rules, External Controller info) "
I think Ans2 is still hand-managed for now. Can you confirm that a new PNFD is not needed or used for PNF upgrade in Casablanca release?
yaoguang wang
Ans1: yes, it needs to be implemented in R3. To make the endpoint of EC configurable, one JIRA was created, and Ruchira Agarwal is working on it.
Ans2: No. Generally speaking, PNFD or PNF pacakge issue is still under discussion in R3+R4, and may not be defined or used in SDC in R3(from my understanding). In this functionality in R3, upgrade related artifacts (like vendor-specific checking rules) need be provisioned in EC.
Rebecca Lantz
Thanks yaoguang wang! My question 2 isn't fully understood I think. My point is that when the PNF changes software version, usually the data model, alarms, counters, etc. also change. These artifacts (data model, alarms, counters) are being used by various ONAP components (Controller and various DCAE services). In R3 I believe the intention for these artifacts are to have them manually added to the service blueprint (since there is no PNF package yet). My question addresses the intention for supporting newer versions of these CM/FM/PM models in ONAP. Is this something that has been discussed in the context of PNF software upgrade?
Ulas Kozat
Rebecca, this comment is also valid for VNF upgrades as well. I am not sure if the community has worked on these issues. Would you like to bring it to Change Management use case team for discussion?
Rebecca Lantz
After upgrade completes, does the PNF re-register with ONAP via VES event (which contains the current software version)? Like it does for PNP / initial PNF deployment?
Ulas Kozat
No re-registration is performed. SO is supposed to update the software version entry in A&AI after successful software upgrade.
Rebecca Lantz
Thanks Ulas. It may be good to rethink this for the future. No re-registration would mean all PNF upgrades must happen via ONAP. If a new software version is not allowed to be registered via message generated from the PNF, then I suppose there isn't the possibility of a software change instigated directly at PNF or via PNF's EM. My guess is there may be situations where there is a need for quick fix updates to be applied to PNF software that would not want to be limited by need for ONAP processing.
Ulas Kozat
Well, this use case and the specific workflow only cover the software upgrade done through ONAP. But, I think it is in general a good idea to define a flow for "re-registration" (and also "de-registration") independent from the software upgrade flow.
Rebecca Lantz
In Casablanca, SDNC configuration needs to include the ansible playbooks and the targets (the external controller info). In Dublin release we should find a way to ensure that both playbooks and targets can be dynamically added to running SDNC.
Benjamin Cheung
Source file for Flow Diagrams (PPTX)
Rebecca Lantz
Thanks for creating this Ben. However it is misleading as this is way more than is developed or intended for R3. R3 is only
Maybe it is better to make this a living document for Dublin intentions?
Mark Scott
Some comments from today's discussion that warrant further discussion:
yaoguang wang
Since some key contributor will be absent this week, the meeting (10/25) will not be scheduled. Ulas Kozat I suggest that we start to discuss the plans in Dublin in the week (10/28-11/3) . Can you present any proposal on this functionality for R4+? Mark Scott Rebecca Lantz Benjamin Cheung
Benjamin Cheung
The list in the DUBLIN Enhancement on the PNF SW Upgrade page:
5G-PNFSoftwareUpdate-DUBLINENHANCEMENTS is pretty good. I think we should probably review and discuss these entries in a meeting to see if we can refine them.
I have added a link to the R4 UC Proposals page: 5G - PNF SW Upgrade (Casablanca carry-over items) back to this page.
-Ben
Rebecca Lantz
Is there an intention to merge PNF sw upgrade with existing in-place VNF sw upgrade? A single xNF in-place sw upgrade would be reasonable if there are no specific differences between VNFs and PNFs in this area.
Benjamin Cheung
Rebecca Lantz, I think that would be worthwhile objective to try to get a xNF SW upgrade ... they do have some things in common (a S/W package), but there are also lots of differences too (a PNF S/W goes down to a specific box) whereas a VNF/CU S/W can be created "dynamically". We should discuss this though ...
Rebecca Lantz
My understanding is that only "in place" VNF sw upgrade is currently supported in change management. So it should be the same as a PNF. Build-and-replace VNF sw upgrade has not yet happened... Yes lets discuss more.
Ulas Kozat
Folks, FYI, we have an assigned zoom link for Thursday time slot. I added the meeting info to the top of the page.