Analytics as a Service Closer to Edges

Problem Statement:

• The goal of Analytics as a Service closer to edges is address edge Scalability, Constrained Environment and Service Assurance Requirements.

  • Avoid sending large amount of data to ONAP-Central for training, by letting training happen near the data source (Cloud-regions).
  • ONAP scale-out performance, by distributing some functions out of ONAP-Central such as Analytics
  • Letting inferencing happen closer to the edges/cloud-regions for future closed loop operations, thereby reducing the latency for closed loop.
• Reference: ONAP-edge-automation-update-arch-use-case-10-23-2018.pdf

• 5G use case relevance

  • 5G/performance Analysis and Optimization:  High Volume and RT Data Collection/Analytics/Closed Loop of performance metrics at the Edge Cloud

  • Reference: ONAP-edge-automation-update-arch-10-29-2018-followup-11-07-2018.pptx
  • Reference: 5G_UseCase_for_Dublin_v4.pptx (reference slide adapted from this deck)
     

Dublin Requirements Summary: Distributed Analytics as a Service (Dublin Summary) - Edge Automation

Architecture Scope:

• Instantiation of edge and connectivity to ONAP central (out of scope for ONAP)
  
  
• Edge Cloud Registration [Ref. Arch. Impact Details (1)]
  • Automation of registration when scale (>100s)
• ONAP edge functions or 3rd party edge functions deployed at edge (e.g. Analytics, Closed Loop Control) [Ref. Arch. Impact Details (21 , 22)]
  • Registration of the edge functions to ONAP central (Intent, capabilities, capacity)
    • Intent Example: “Infrastructure Analytics as service for Alerts at Cluster Level and Host Level”
• Deploy Network Services in an optimal way to the edges using edge/central functions [Ref. Arch. Impact Details (3)]
  • Includes multiple VNFs on multiple edges/core which make a service
  • Cloud region (means one control plane) choice
  • Connect the service to the functions

• Networking of ONAP Central and edge functions [Ref. Arch. Impact Details (5)]
  
  

  Reference: ONAP-edge-automation-update-arch-10-29-2018-followup-11-07-2018.pptx

ONAP-based Analytics as a Service Details: (see Distributed_analytics_v3.pptx in Edge Automation through ONAP)

• What does ONAP-based Analytics Service encompass?
  • Support analytics-as-a-service in the cloud-regions that have K8S site orchestrator.
  • Use same analytics framework to have analytics even in ONAP-Central.
  • Two packages - Standard package and inferencing package.
  • Use existing analytics applications - TCA to prove this framework.
  • As a stretch - Showcase one ML based applications
    • Training application
    • Inferencing application

• How to Develop?
  • Use PNDA as a base
  • Create/adapt Helm charts
  • Ensure that no HEAT based deployment is necessary.
  • Use components that are needed for normal analytics as well ML based analytics (Apache Spark latest stable release, HDFS, OpenTSDB, Kafka, Avro schema etc..)
  • Use some PNDA  specific packages - Deployment manager as one example.
  • Develop new software components
    • that allow distribution of analytics applications to various analytics instances
    • that allow onboarding new analytics applications and models.
    • that integrates with CLAMP framework (if needed)

• Impacted ONAP Projects
  • DCAE, CLAMP, A&AI (TBD), Multi-VIM/Cloud (TBD)
• How to Test?
  

  • TCA (Changes - Convert this as a spark application) 

  • New Machine learning models for KPI (packet loss) prediction (New use case)

3rd Party Analytics Application - Dublin Scope

• Register VMware VIO (OpenStack-based) edge cloud region(s) in ONAP Central (Note: These Edge Cloud Regions support K8S)
  • References: 
    • https://docs.vmware.com/en/VMware-vCloud-NFV-OpenStack-Edition/3.0/rn/vCloud-NFV-OpenStack-Edition-30-Release-Notes.html
    • https://docs.vmware.com/en/VMware-Integrated-OpenStack/5.0/rn/VMware-Integrated-OpenStack-50-Release-Notes.html
  • ONAP project impact: None
• Deploy VMware vROps 3rd party infra analytics framework/application in target edge cloud region to monitor multiple edge cloud regions 
  • References:
    • https://docs.vmware.com/en/VMware-vCloud-NFV-OpenStack-Edition/3.0/rn/vCloud-NFV-OpenStack-Edition-30-Release-Notes.html
    • https://docs.vmware.com/en/vRealize-Operations-Manager/6.7/vrealize-operations-manager-67-reference-architecture-guide.pdf
    • In vrops reference architecture this is known as multiple data centers with remote collectors.
  • ONAP project impact: None
  • Note: 3rd party infra analytics application configuration is out of scope for Dublin
• Deploy microservice in target edge cloud region for cloud infra Event/Alert/Alarm/Fault Normalization & Dispatching to ONAP Central
  • Note: This microservice uses the Multi-VIM/Cloud project repository for development
  • ONAP project impact:
    • Multi Cloud project impact
      • Cloud infra Event/Alert/Alarm/Fault Normalization & Dispatching microservice development
        • Integrate DMaaP (Kafka) client for communication to ONAP Central 
        • Receive Event/Alert/Alarm/Fault from 3rd party infra analytics application
        • Normalize from cloud specific Event/Alert/Alarm/Fault format to cloud agnostic (ONAP internal) Event/Alert/Alarm/Fault format
          • ONAP internal format references
            • Control Loop Design
            • Questions: Does each control loop need a separate policy component?

          • Alert examples (Note: Host CPU Threshold & Memory Contention Threshold in a cloud region are defined separately)

            • Exemplary Alert Definition: 

              • Standalone host has memory contention due to overpopulation of virtual machines.

              • Symptoms include the following: 
                • Host memory workload at warning/immediate/critical level 
                • Host memory contention at warning/immediate/critical level 
                • > 50% of child virtual machines have Virtual machine memory workload at warning /immediate/critical level

            • Exemplary Alert Action:

              • Scale out VNF

            • ...

            • References:

              • https://docs.vmware.com/en/vRealize-Operations-Manager/6.6/vrealize-operations-manager-66-reference-guide.pdf

        • Dispatch Event/Alert/Alarm/Fault to ONAP central using DMaaP (Kafka) client
      • Cloud infra Event/Alert/Alarm/Fault Normalization & Dispatching microservice deployment
        
        • Develop K8S Helm chart
      • Note: 
        • This microservice is focussed on operational workflow & and independent of the current deployment focused micoservices in multi vim/cloud.
• Register 3rd party infra analytics application in ONAP Central (Stretch Goal)
  
  • ONAP project impact:
    • Multi Cloud impact - Below

      • Populate Intent in A&AI 

        • Infra Analytics as service exemplary intent -- “Infrastructure Analytics as service for Alerts at Cluster Level and Host Level for a Cloud Region”

      • • Capabilities (not exhaustive) corresponding to intent in A&AI (Note: Cluster CPU Threshold & Memory Threshold are defined separately) 

          • Cluster has memory contention caused by more than half of the virtual machines 

          • Cluster has memory contention caused by less than half of the virtual machines

          • Cluster has unexpected high CPU workload

          • ...
      • Populate Cloud Region List in A&AI corresponding to Intent
    • A&AI: Leverage existing HPA/Intent key-value pair schema 

Fine Grained Placement Service (F-GPS)