(warning) Re-arranging content...and, cleaning up....

General Background

A broad set of transformations are taking place:

  • Business transformation: OTT services, faster TTM, Monetization
  • Technical transformation: QoE, ULL, SDN/NFV/OMEC integration, Edge Analytics, Big data, Virtualization, Automation, C->E, R->E
  • Architectural transformation: 4 views “NORMA-like” Cloud, ECOMP, Flexible architecture (RAN, Core, CDN, Application delivery, Automation, IoT, fog,..)
  • Industrial transformation: ICT&E

To efficiently and effectively deploy 5G network supporting ultra low latency and high bandwidth mobile network, we need to deploy variety of applications and workload at the edge and close to the mobile end user devices (UE or IoT).  That would include various virtualized RAN and core network elements, content (video), various applications (AR / VR, industrial automation, connected cars, etc.).  We might deploy near-real time network optimization, customer experience / UE performance enhancement applications at edge.  Edge cloud must support deployment of third party application (e.g. Value added optional services, Marketing, Advertising, etc.).  We must deploy mechanisms to collect real time radio network information, process them in real-time (e.g. Geo Location data), summarize, anonymize, etc. and make them available to third party applications deployed at the edge or central location or outside service provider environment.  Edge data collection could also be used for training machine learning models and fully trained models can be deployed at the edge to support network optimization.

The need

End users and other devices, cyber-physical systems will benefit from a broad set of context information that can enhance and enrich the delivery of a broad set of applications. 

Service Deployment Goal

Deliver Application SLAs while minimizing TCO.

Edge Application Profiles


No

Application Classification

(based on required RTT)

Application ExamplesNetwork / Service Behavior TypeDeployment Component/ APIsONAP ManagedEdge Deployment Hard /Soft Constraint (Based on RTT)Potential Application ProviderCasablanca CandidateAdditional Information
1Real-time (20ms -100ms)In service path optimization applications which run in open CU-CP platform (also known as RAN Intelligent Controller, or SD-RAN controller). Real-Time   Network State ControlOpen 5G CU-CP (CU - Control Plane) – VNFC.YesHardNF Vendor/Service Provider/3rd PartyYes

These applications include load balancing, link set-up, policies for L1-3 functions, admission control and leverage standard interface defined by oRAN / xRAN between network information base (or context database) and third party applications. Data collection through is B1 and implemented using x technology.

2Near-real-time (500ms and above)Slice monitoring, performance analysis, fault analysis, root cause analysis, SON applications, Optimization (SON Drive Test Minimization etc.), ML methodologies for various apps.Network Analytics & OptimizationDCAEYesSoftNF Vendor/Service ProviderYes
3Near-real-time (500ms and above)Video Analytics, Video Optimization, Customer geoLocation information, Anonymized customer data etc.Workload Analytics,  Optimization & Context processingCloud Edge or Cloud CentralNoSoft3rd PartyNA. Out of scope for ONAPThe apps are OTT and the service provider is offering their infrastrcture as a service to OTT providers.
4Real-time (10-20 ms)Third party applications that directly interacts with the UEs, like AR/VR, factory automation, drone control, etc. Workload Automation / AR-VR / Content, etc.UE or Cloud EdgeNoHard3rd PartyNA. Out of scope for ONAP.

These are third party applications, developed by enterprise customers (e.g. factory automation) or content creators (AR/VR applications). In this case, messages or requests or measurements directly go from UE (via UPF or GWs) to the applications and applications respond back. 

5same as 3)same as 3)Value Added Services + same as 3)

same as 3) + MEC/Cloud APIs (Note 1)

Yessame as 3)same as 3)StretchService Provider could be oferring video surveillance (video analytics/optimization apps etc.) as a service to enterprises.
6same as 4)same as 4)Value Added Services + same as 4)same as 4) + MEC/Cloud APIs (Note 1)Yessame as 4)same as 4)StretchService Provider could be oferring factory automation as a service to enterprises.

Note 1:  API Details

  • e.g., MEC APIs - Location info, Radio control info etc.
  • e.g., Cloud APIs - IaaS/PaaS + Context Awareness (time, places, activity, weather etc.)  

Edge Infrastructure Profiles

((warning) example based on Akraino Edge Stack..but, need to generalize)  

ProfilesWorkloadsComputeNetworkingStorageControlSecurityEdge Application Infrastructure
Large

Support for VMs and containers.


Commentary:

  • VNFs from Operators and Edge applications from customers of Operators.
  • Number of tenants to be supported???

>50 Compute Servers

Accelerators:

SRIOV based QAT for Crypto and Compression acceleration.

ML/DL Accelerators

Compute profiles: Fixed number of profiles are expected to be supported. (Will add profiles)


SRIOV Networking for High performnace Data plane VNFs.

vSwitch (OVS-DPDK) based networking for all other workloads

Multiple leaf switches and two spine switches

WAN - Underlay :

  • L3VPN Support (BGPVPN)
  • L2VPN support (E-VPN, PBB VPN, VPLS?)

Underlay realization options

  • PE at the Edge (MPLS/BGP start at the Edge) as physical appliance
  • PE at the edge as virutal appliance
  • CE- Physical at the edge
  • CE - Virtual at the edge

Overlay realization options

  • GENEVE based networks (for workload migration, redundancy and scalability)

IPv4 and IPv6 support

NAT44 with LSN (Large Scale NAT) support by providers.

Support for dedicated public IP addresses

Commentary: Network sharing among container and VM workloads will need to be supported. DVR (Distributed Virtual Routing) for forwarding packets locally among vSwitch based networks. Leaf/Spine switches for forwarding traffic among SRIOV based networks and for networks between vswtich and SRIOV based networks.

Few fixed profiles for following:

  • Local network profiles
  • Fabric topology profiles
  • WAN connectivity profiles

Block device support using Ceph

Dedicated nodes for storage ( 3 nodes )

Storage profiles representing whether the nodes are dedicated for storage, use compute nodes for storage, Number of nodes for storage etc...

Is support for Object storage required in Edges?

Dedicated nodes for control stack

Automation Offload Platform (Offloading ONAP) at the Edge.

Few control profiles

  • Profile 1:
    • Openstack for VM workloads
    • K8S for Container workloads
    • Dedicated nodes for VMs and containers.
  • Profile 2:
    • K8S control for both VMs and containers. No need to dedicate the computes.

Automation Offload Platform profiles consists of following:

    • VNF Life Cycle management
    • Fabric Control
    • WAN Control
    • Analytics Offload


Transport : TLS 1.2 and above between ONAP and Edge Services

Infra Security: TPM 2.0/SGX for private key security and secret/password protection, Remote attestation to detect any software tampering of compute, storage and control nodes.



MEC Platform as a VNF to provide contextual information to Edge applications.



MediumSame as above

Same as above.

Number of compute nodes are >10 and < 50

Same as aboveSame as above, except that there is no dedication of nodes to Ceph cluster

Same as above with respect to control, but Automation Offload Platform is not part of the Edge. No dedicated control nodes. Control functionality is shared with compute nodes.

Support for K8S profile as it can support both VMs and containers

Same as aboveSame as above
SmallSame as above, but may support very less number of tenantsSame as above. Number of compute nodes are < 10Same as above, but no PE and CE at the Edge. Fabric itself acts as CE.Same as above, no dedication of nodes to Ceph cluster

No control at the Edge

No Automation offload platform at the Edge

Regional sites are expected to provide control and AOP services.

Support for K8S based control.

Same as aboveSame as above

Edge Infrastructure Profile Summary 

  • Distributed
    • 1000's of edge locations of varying capacity
    • Casablanca - Implementation
      • 10-100 edge locations (simple starting point)
  • Peformance-awareness
    • GPU, FPGAs, SR-IOV etc.
    • Casablanca - Implementation
      • SR-IOV desired for Data Plane (5G CU-UP)
      • NIC offload desired for tunnel encap/decap e.g. 5G CU-UP GTP tunnel
  • Resource Isolation through fine-grained QoS
    • Support both Latency-sensitive and General purpose applications
    • Support ONAP Management plane components in the same cloud with Workloads
    • Casablanca - Implementation
      • Min/Max resource reservation model desired
  • Security
    • Workloads are often deployed in external (non-dc-type) locations and need HW security (TPM etc.) 
    • 3rd party applications which need additional HW security (VM, Containers in VM etc.) and SW security (Inter-component TLS etc.)
    • Casablanca - Implementation
      • Edge Clouds with private IP addresses, i.e. reachable via private connections 
      • For example, edge cloud in a public cloud provider reachable via AWS direct connect or Azure express route or Google partner interconnect 
  • Capacity constraints
    • Very small footprint (few nodes per physical location), Medium footprint (10's of nodes per physical location), Large footprint (100's of nodes per physical location)
    • Casablanca - Deployment
      • Need number of cores per servers; Need storage capacity/pool
  • Cloud Diversity
    • Private and Public Cloud Providers

    • Casablanca - Implementation
      • Note: ONAP currently supports private edge clouds based on VMware VIO, Wind River Titanium Cloud, Upstream OpenStack
      • Desire to have at least one Public Cloud Provider (Azure, AWS, GCE etc.) as an Edge Cloud Provider
        • ONAP central instantiates an Edge Cloud instance (blue cloud provider in gliffy) via a IaaS API to cloud provider
        • ONAP central instantiates one or more ONAP edge components as need, e.g. DCAE
        • ONAP central instantiates one or more NFs, e.g. 5G CU-UP/CP
  • Configuration Diversity
    • 5G Factory Automation, 5G General Mobility Services etc. – User Plane components (DU, CU-UP, UPF etc.) 

Key Challenges w/Centralized ONAP Architecture for Network Function Edge App/Infra Profiles

WAN network bandwidth & latency issues for the following key functions

  • Conveying Application & Infrastructure metrics, faults in real-time from Edge Cloud to Centralized ONAP for closed-loop fault management for a single edge site with a single Cloud control plane
  • Conveying Application & Infrastructure metrics, faults, alerts in real-time from Edge Cloud to Centralized ONAP for dynamic network function closed-loop fault management/placement optimization across multiple edge sites with a single Cloud control plane 
  • Conveying Application & Infrastructure metrics, faults, alerts in real-time from Edge Cloud to Centralized ONAP for dynamic network function closed-loop fault management/placement optimization across multiple edge sites with multiple Cloud control planes 

Exemplary Network Function Placement/Service Assurance Policy for demonstrating the aforementioned challenges

  • "5G CU-UP VNF location to be proximal to a specific physical DC based on 5G DU, bounded by a max wire latency from physical DC"

 Landscape for addressing aforementioned Challenges

  • Cloud Providers
    • Public Clouds such as Azure, Amazon, GCP do not support dynamic network function closed-loop fault management/placement optimization across multiple edge sites with single/multiple Cloud control planes 
  • Akraino Open Source
    • Defines Blueprints for Edge Cloud using OpenStacks (VMs) and K8S (Containers)
    • Plans to use ONAP for VNF/Service Orchestration for addressing aforementioned challenges

Hierarchical (ONAP Central, Edge) Architecture

Single Provider - Hierarchical Architecture

Internal Edge - Orchestration


  • Cloud Provider Business Unit: 
    • Provides hosting of Workloads, ie., IaaS/PaaS
      • SP installs and manages ONAP in separate 'Management Cloud' instances
      • SP installs and manages Network Services + 3rd Party Apps in separate 'Services/Apps Cloud' instances
  • Cloud Provider Business Unit: 
    • Provides SaaS, eg., Analytics/Closed Loop as a Service, LCM of Apps, etc. 
      • ONAP Edge may not be needed
  • Cloud Provider Business Unit: 
    • Types of virtualized cloud resource tenant and their characteristics
      • Virtualized Network Workload Cloud Resource Tenant Category
      • Network Management Cloud Resource Tenant Category
      • Virtualized Application Workload Cloud Resource Tenant Category
      • Application Management Cloud Resource Tenant Category 
    • Physical Network Function and their characteristics
      • Part of Edge Cloud Orchestrator
  • Immediate interest to ONAP for Network Function use cases
    • Virtualized Network Workload Cloud Resource Tenant Category
      • Guaranteed
      • Burstable (with minimum guarantee)
      • Best Effort
    • Network Management Cloud Resource Tenant Category 
      • Burstable (with minimum guarantee) 

Single Provider - Edge Functional Decomposition 

FunctionStatefulnessONAP Project MappingDetails
InventoryyesA&AI
IP Address Management (IPAM)yesSDN-C
Multi-Cloud SupportnoMulti-VIM
Initial PlacementnoOOF
Closed Loop ControllernoAPP-C
Closed Loop PolicynoPolicy
Infra Closed Loop AnalyticsnoMulti-VIM, DCAE
App Closed Loop AnalyticsnoDCAE
Loggingno

Infra/App Monitoring events, statisticsno

Single Provider - Sequence Diagram

ONAP Activity Goal #1: ONAP requires IaaS/PaaS attributes (see ongoing work – Edge Cloud Infrastructure Enablement in ONAP5G Items for Casablanca) from Cloud providers for Infrastructure profiles that allow Distributed, Highly-secure, Config/Cloud-diverse, Capacity-constrained and Peformance/Isolation-aware

ONAP Activity Goal #2: Define hierarchical ONAP Central, Edge Architecture/functional interactions (API reference points) to support aforementioned Application/Infrastrcuture profile in Any "Cloud" (internal Business Unit or external Partner) at Any "Location" edge, regional or central. 


ONAP-Edge - sequence

  • No labels

24 Comments

  1. The picture was discussed in the meeting. The separation of "access network" vs. "last mile network" does not make much sense to me as access networks provide the last mile solution. Maybe more inline with the picture is to use "customer-premise network". Even in that case why would an Edge Cloud in a customer premise have larger latency than an Edge Cloud in an access network?

    1. last mile vs access network: yes, the terms are confusing. maybe think last mile = internal LAN/campus network/tall-shiny-building. As example, inside a home. most of delay is due to slower speed links and/or best effort queuing in a wifi network, etc.

      delay mentioned depends on location of measurement points (MEPs, MDs using IEEE CFM terms). As example, last mile network = host-to-CPE, whereas, access network = CPE-to-CO/metro

  2. The main work here is to define how a "Central ONAP" instance would communicate on its "Southbound" to/from "Edge Controller" instances. Where, "Edge Controller" instances could be of any type of: ONAP itself, other open source controllers (ONF CORD, Akraino, ...), or proprietary vendor controllers (Nokia, Huawei, ...).

    The "Central Controller ← to/from → Edge Controller" communication is required in both directions: Central Controller → down-to → Edge Controller AND Central Controller ← up-from ← Edge Controller.

    The main architectural work here is to define how ONAP's main "Southbound" components (Multi-Cloud, SDN-x, DCAE) would communicate both from a Central Controller to an Edge Controller (top-down) AND from an Edge Controller to a Central Controller (bottom-up).

    This means Multi-Cloud, SDN-x, & DCAE need to act as both clients and servers. Today, on the "Southbound", Multi-Cloud and SDN-x only act as clients and DCAE only as a server. What is new here is that they all need to act as both clients and servers and expose a set of APIs for "Central-Edge" communications.

    Another option is to create a "new" ONAP component that terminates all "Central-Edge" communications (APIs) and forwards them internally to Multi-Cloud, SDN-x, or DCAE.

  3. Good points Arash. Please take a look at the notes below the gliffy diagram – we also need to cover the cases where Edge ONAP is optional and also Edge ONAP has only operational components (DCAE etc.).

  4. We have standardized on Kubernetes not Docker Swarm currently.  We can collaborate the Multi-vim team around federated support for deployment and management of the clusters via helm.  The current CD system runs on a single 5 node cluster on AWS - I would like to extend this to a Multi-AZ deployment

    http://jenkins.onap.info/job/oom-cd-master/2904/console

  5. I can see only the DCAE and closed loop related components in the Edge cloud. Is that a reduced scope ?

    There is an indication of DCAE/MEC - Does it mean DCAE + other MEC (ETSI) architecture components (MEAO, MEPM-V) ?

    I believe the scope of edge cloud should also include support for slice provisioning based on instruction from central cloud. Also it should preferably based on intents with declarative workflows 

    Additionally  not sure if this diagram is just indicative or planned for Casablanca. As per presentation by 5G Use case subcommittee wiki on slicing in 3GPP - there should be an AMF shared by slices and SMF per slice. With this understanding, would like to know the edge cloud scope with respect to slices - I believe a single edge ONAP manages set of slices in edge infrastructure. 

    In the diagram I see vOLT and vBBU  - Would like to know the role of these two VNFs. Here BBU means C-RAN base band unit or Broadband base unit? If yes I guess DU along with CU  fulfils that functionality of C-RAN BBU if I am not wrong. With vOLT I believe this is meant for a backhaul transport link. Is this a correct understanding ? Is this going to be a R-vCPE/E-vCPE kind of connectivity between Edge and Central cloud (management/control plane) ? Is that going to be a separate infrastructure service outside the scope of edge cloud ONAP ? 

    1. >> scope of edge cloud should also include support for slice provisioning based on instruction from central cloud.

      So far, we have focused on IaaS API from onap-central to edge CP. If that IaaS API requests a 'slice', then, edge CP would be expected to support that.

      >> if this diagram is just indicative or planned for Casablanca.

      not using that diagram. We have gliffy instead.

      >> BBU means C-RAN base band unit or Broadband base unit? If yes I guess DU along with CU  fulfils that functionality of C-RAN BBU if I am not wrong.

      yes.

      >> With vOLT I believe this is meant for a backhaul transport link.

      no. for fixed access loop. Can be for Residential/Enterprise use cases.


      1. Thanks Raghu. The sequence diagram also seems to be bit confusing. From the sequence diagram it gives an impression that Edge ONAP stack is brought up on demand on the Cloud Provider infrastructure. Is this a right understanding ? If this is correct the assumption is that lifecycle and connctivity of EDGE ONAP is managed by Central ONAP.

        In this case I did not quite get the sequence of Service Order decomposed into Service Request results in an action on OOM. Are you suggesting changes to SO to have an interaction with OOM ? Or this is going to be a call on Multi-VIM by SO based on the Service Descriptor (for Edge ONAP bring up) ? 

        If the above understanding is correct, Edge ONAP is similar in nature to a VNF packaged in a Service - Is this a correct understanding ? I am asking this because this will have an impact on how we define the minimum viable product for Edge ONAP - If we bring up Edge ONAP as a VNF on cloud provider infrastructure how can we model the specific components (A&AI, DCAE etc) to be loaded in the Edge ONAP ? 



        1. >>Edge ONAP stack is brought up on demand on the Cloud Provider infrastructure.

          yes...but I would expect this only when the first instance of the edge domain is being used.

          >>did not quite get the sequence of Service Order decomposed into Service Request results in an action on OOM

          we will need to tweak the sequence, ie., if not already deployed for a service that requires edge-onap components, then request OOM to trigger deployment, etc.

          >>Edge ONAP is similar in nature to a VNF packaged in a Service

          an edge-onap instance would be expected to support many services deployed in that edge domain. So, the intent was not to do this per service request, etc.

          btw, we have not reviewed the sequence in detail during calls. hopefully, we can get to it in a future call.

  6. The "Sequence Diagram" is a little bit confusing when correlating it to the description "ONAP Edge MVP for Casablanca ". The diagram indicate with Option B there will be close loop on edge infrastructure, while the description tells only Option C support that. 

    Another question is: How DMaaP deployed with edge ONAP is related/collaborated with the one deployed with centralized ONAP? I mean, is it possible for some ONAP component (e.g. DCAE app ) deployed in centraizlied ONAP to subscribe a topic to DMaaP running within the edge ONAP?

    Thanks

    1. >>diagram indicate with Option B there will be close loop

      ..but, not in 'red color font'. See text on right about red text.

      for your other Q, see sub-bullets in text for 'key changes needed for option B'. this needs further discussion.

  7. May I suggest the following format for section ONAP Edge MVP for Casablanca:

    ONAP Edge OptionDescriptionKey ChangesComments
    A
    • No ONAP components in Edge 
    • Critical Edge Functionality is delivered by Cloud Providers and VNF vendors
      • Analytics (Infra/App)
        • Value: Summarize data in the edge and avoid WAN bandwidth deluge
          • Generate appropriate events and alarms
        • Edge Infra Analytics 
          • Cloud 
        • Edge App Analytics 
          • VNFs
        • Close Loop Use Cases which need only ONAP Central intervention
          • VNF Scale in/out - Proactive using app/infra predictive analytics 
          • Enhanced Alarm Correlation 


    Note: 

        • This assumes Analytics and Fault Management Policies in Clouds and VNFs are independently configured. 
        • Single pane of glass policy management through ONAP involves managing a multi-vendor distributed policy framework and out of scope for R3.

    BFinalized in call on 06/04/2018
    • Option A, plus 
    • DCAE Enhancements 
      • Support New microservice based Apps –  Centralized SON applications, Optimization (SON Drive Test Minimization etc.), ML methodologies for various apps
    • ONAP Edge 
      • DCAE Enhancements (described above) 
      • DMaaP (communication for onap-edge DCAE microservices) 
    • ONAP Central 
      • OOM Enhanacements
        • SP uses a central-OOM with a 'policy' for deployment of an onap-edge instance, e.g., xyz edge provider with abc components, etc.
          • However, onap-edge instance can be 'lighter weight' with subset of components needed (per MVP discussed below)
          • Desirable to managed as a separate K8s cluster (ie., separate from onap-central instance, of course) and, only for onap-edge use, ie., don't use for other 'workloads' like network apps or 3rd party apps
      • ONAP Central↔Edge GW (Can Multi Cloud play the role?)
        • GW functionality is part of either Central or Edge ONAP instance
        • Communication across ONAP-central and onap-edge instances or ONAP-central instance to Edge Cloud Provider's Systems
        • Interface with ONAP instances through APIs 
        • Internal ONAP instance communication through API or DMaaP
      • ONAP Central OOF enhancements
        • Example: Choosing the Cloud Region for deployment of Network Functions (PNF/VNF) based on various constraints
          • Leverage Infrastructure Events/Alerts/Faults/Metrics for aggregate objects (Tenant, Cluster etc.) from (onap edge/central) DCAE or Multi-Cloud
          • Leverage Application Events/Alerts/Faults/Metrics for aggregate objects (VNFs etc.) from (onap edge/central) DCAE
    • ... more todo 

    Note: 

      • New DCAE Apps are from the Application profile table 
      • Choose applications that are independent and which do not impact closed loop operations
    C
    • Option B, plus
    • Closed Loop Related Components
      • Static/Dynamic Policy - PDP 
        • Policy may depend on current deployment state and also might need service context for the service component such as VNFs? So,  other onap components may be involved at the edge? 
      • APP-C, VF-C, Multi-Cloud for Controller Function
    • Multi Cloud Current + Enhancements
      • Current
        • Standardize Infrastructure Events/Alerts/Faults/Metrics and communicate to onap-edge DCAE VES Collector through APIs 
      • Enhancements
        • Edge Cloud supports Infrastructure Analytics Micro services
        • Multi Cloud standardizes "ONSET" and "ABATE" of Infrastructure Alerts/Faults and communicates them to onap-edge's Policy 
    • CLAMP Enhancements in onap-central
      • Deploy a separate Control Loop per Edge DCAE/Multi Cloud
      • Associate the right Edge DCAE during Control Loop deployment 
    • ... more todo



  8. Certainly Dominic, we are already working on it.

  9. Edge Cloud Team,

    I believe there is a significant work item that is missed here which is Edge Modelling in the Central ONAP  (for the Modelling Group). From the perspective of the Central ONAP, an end to end 5G service would consist of a set of Edge Clouds connected together through a Central Cloud. For an end to end 5G service to be modeled in the Central ONAP, there needs to be an Edge Descriptor defined in the SDC (just like the VNF/PNF Descriptor today). The Edge Descriptor would define exactly how every part of the Central ONAP would communicate with the Edge entity whatever that entity may be: ONAP Edge, Akraino, ONF CORD, etc. The Edge Descriptor would also define how the Edge would be connected to the Core. Then at design time, for every Edge instance, an Edge Descriptor would be on boarded into the SDC of the Central ONAP to be used in end to end service definitions (templates).

    1. this would be part of the 'IaaS' work....in a future release (smile)

      1. But without this how would the end to end service be modeled in the Central ONAP is Casablanca?

        Without an Edge Descriptor to abstract out the Edge, every VNF/PNF (DU, CU, ...) at the Edge would have to be modeled in the Central ONAP.

        1. we don't....b'coz static set up. .. ....unless, of course, we want to be ambitious and do stuff you suggest. see 'grey region' in sequence diagram.

  10. There is one component called ESR.  One way is to enhance ESR (API and A&AI schema representation) to support edge specific configuration.

    1. I think you are assuming the problem statement of onap-central managing LCM of each edge node. Do we want to take that on in this work? 

      1. In other words, we want the edge cloud provider to use onap for their internal use.

      2. Edge LCM (First time, upgrades and patches) involving node/switch software installation/upgrades/patches is not part of ONAP.  Normally, it is done using installers (such as compass, APEX, Airship etc..).  Once the edge is deployed via installers, it needs to be made aware to ONAP for service orchestration.  I was referring to ESR in that context of making edges visible to ONAP. I hope this clarifies (smile).

        1. hmm...i would expect the edge node to register with edge provider's systems. what we have assumed so far is the edge provider will expose a service catalog.

          1. Hi Raghu, Looks like I confused you.  ONAP will only see the edge-cloud with respect to services, compute profiles (type of edge nodes for HPA). Individual nodes are not visible to ONAP. 

            In case of Openstack based edge, ONAP will see services exposed by Openstack.

            In case of K8S based edges,  ONAP will see API server endpoint and OVN North-DB endpoint etc...

            We are on the same page ( I hope ) (smile)

            1. ok.

              do we want to expand scope of work to include edge provider using onap for their internal domain orch?