Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Test Results @CMCC lab

Image Added

Category: External System Registration


V00001: WindRiver OpenStack VIM Resource OnboardingRegistration

Test Case IdV00001
Test Case NameWindRiver OpenStack VIM
Resource Onboarding
Registration
DescriptionWindRiver OpenStack VIM
Resource Onboarding
Registration
ReleaseAmsterdam
Preconditions
  1. The WindRiver OpenStack VIM is configured with tenant information and provider networks
  2. The WindRiver OpenStack VIM services (console, auth, neutron, image, glance, etc) are accessible from ONAP by IP
Testing Steps
  1. Add VIM from ONAP GUI with VIM location information, tenant authentication information
  2. VIM default service url can be configured based on real VIM settings
Expected Results
  1. VIM
resource
  1. is
onboarded
  1. registered successfully
  2. Tenant authentication information is stored
successfully
  1. correctly
Actual Results
Conclusion (Pass/Fail)
Testing Lab
Tester Name


V00002: WindRiver VMWare OpenStack 2nd VIM Resource OnboardingRegistration

Test Case IdV00002
Test Case Name
WindRiver
VMWare OpenStack
2nd
VIM
Resource Onboarding
Registration
Description
This is to test multi VIM support
VMWare OpenStack VIM Registration
ReleaseAmsterdam
Preconditions
  1. The
first WindRiver OpenStack VIM has been successfully onboardedA new WindRiver
  1. VMWare OpenStack VIM is configured with tenant information and provider networks
  2. The
new WindRiver
  1. VMWare OpenStack VIM services (console, auth, neutron, image, glance, etc) are accessible from ONAP by IP
Testing Steps
  1. Add
the new
  1. VIM from ONAP GUI with VIM location information, tenant authentication information
The new
  1. VIM default service url can be configured based on
the new
  1. real VIM settings
Expected Results
The new
  1. VIM
resource
  1. is
onboarded
  1. registered successfully
The new VIM
  1. Tenant authentication information is stored
successfully
  1. correctly
Actual Results
Conclusion (Pass/Fail)
Testing Lab
Tester Name


V00003: VMWare OpenStack VIM Resource OnboardingData Center Gateway Controller Registration

Test Case IdV00003
Test Case Name
VMWare OpenStack VIM Resource Onboarding
Data Center Gateway Controller Registration
DescriptionThis is part of WAN setup testing. Each data center has its own gateway and controlled by local SDN controller
ReleaseAmsterdam
Preconditions
  1. Each DC gateway has been manually installed and configured correctly
  2. Each DC gateway local controller has been manually installed and configured correctly
Testing Steps
  1. Register each DC gateway local controller from ONAP ESR GUI with the needed information (location, IP, port, authentication information, etc)
Expected Results
  1. The DC local controller is registered successfully
  2. The DC local controller information is stored correctly
DescriptionVMWare OpenStack VIM Resource OnboardingReleasePreconditions
  1. The VMWare OpenStack VIM is configured with tenant information and provider networks
  2. The VMWare OpenStack VIM services (console, auth, neutron, image, glance, etc) are accessible from ONAP by IP
Testing Steps
  1. Add VIM from ONAP GUI with VIM location information, tenant authentication information
  2. VIM default service url can be configured based on real VIM settings
Expected Results
  • VIM resource is onboarded successfully
  • Tenant authentication information is stored successfully
    Actual Results
    Conclusion (Pass/Fail)
    Testing LabCMCC OpenLab
    Tester Name


    V00004: VMWare OpenStack 2nd VIM Resource OnboardingSDN WAN Controller Registration

    Test Case IdV00004
    Test Case Name
    VMWare OpenStack 2nd VIM Resource Onboarding
    SDN WAN Controller Registration
    DescriptionThis is
    to test multi VIM support
    part of WAN setup testing. In this test, all PEs in the network are controlled by the same SDN WAN controller
    ReleaseAmsterdam
    Preconditions
    1. Each PE router has been manually installed and configured correctly
    2. SDN WAN controller has been manually installed and configured correctly
    3. All PEs are managed by SDN WAN Controller
    Testing Steps
    1. Register SDN WAN controller from ONAP ESR GUI with the needed information (IP, port, authentication information, etc)
    Expected Results
    1. The SDN WAN controller is registered successfully
    2. SDN WAN controller information is stored correctly
    ReleasePreconditions
    1. The first VMWare OpenStack VIM has been successfully onboarded
    2. A new WindRiver OpenStack VIM is configured with tenant information and provider networks
    3. The new WindRiver OpenStack VIM services (console, auth, neutron, image, glance, etc) are accessible from ONAP by IP
    Testing Steps
    1. Add the new VIM from ONAP GUI with VIM location information, tenant authentication information
    2. The new VIM default service url can be configured based on the new VIM settings
    Expected Results
  • The new VIM resource is onboarded successfully
  • The new VIM Tenant authentication information is stored successfully
    Actual Results
    Conclusion (Pass/Fail)
    Testing LabCMCC OpenLab
    Tester Name


    V00005: Data Center Gateway Resource Onboarding SVNFM Registration

    Test Case IdV00005
    Test Case Name
    Data Center Gateway Resource OnboardingDescriptionThis is part of WAN setup testing. Each data center has its own gateway and controlled by local SDN controllerEach DC gateway local controller has been manually
    Vendors' SVNFMs Registration
    DescriptionRegister all VNF vendors' SVNFMs
    ReleaseAmsterdam
    Preconditions
  • VIMs from the same vendor have been successfully onboarded
  • Each DC gateway has been manually installed and configured correctly
    1. All VNF vendors' SVNFMs have been installed and configured
    correctly
    Testing Steps
    Onboard each DC gateway local controller from ONAP GUI (need to confirm which ONAP GUI supports SDN local controller - OOM or UsecaseUI?)
    1. Register all VNF vendors' SVNFMs using ONAP ESR GUI with the needed information (IP, port, authentication information, etc)
    Expected Results
  • The DC local controller is onboarded successfully
  • The
    1. All SVNFMs are registered successfully with ONAP
    2. All SVNFM
    DC local controller
    1. information is stored
    successfully
    1. correctly in ONAP
    Actual Results
    Conclusion (Pass/Fail)
    Testing LabCMCC OpenLab
    Tester Name


    V00006: WAN PE Resource Onboarding EMS Registration

    Test Case IdV00006
    Test Case Name
    WAN PE Resource OnboardingDescriptionThis is part of WAN setup testing. There are at least two PEs in the network, controlled by one SDN WAN controllerSDN WAN controller has been manually
    Vendors' EMSs Registration
    DescriptionRegister all VNF vendors' EMSs
    ReleaseAmsterdam
    Preconditions
  • Each PE router has been manually installed and configured correctly
    1. All VNF vendor's EMS have been installed and configured
    correctly
    Testing Steps
    Onboard SDN WAN controller from ONAP GUI (need to confirm which ONAP GUI supports SDN local controller - OOM or UsecaseUI?)
    1. Register all vendors' EMSs using ONAP ESR GUI with the needed information (IP, port, authentication information, etc)
    Expected Results
    The SDN WAN controller is onboarded successfully
    1. All EMSs are registered successfully with ONAP
    2. All EMSs information is stored correctly in ONAP
    Actual Results
    Conclusion (Pass/Fail)
    Testing LabCMCC OpenLab
    Tester Name


    Category: VNF Onboarding and Service Creation


    V00007: VoLTE VNFs Onboarding

    Test Case IdV00007
    Test Case NameVoLTE VNFs Onboarding
    DescriptionVNFD csar files need to be onboarded one by one
    ReleaseAmsterdam
    Preconditions
    1. All vEPC and vIMS VNFD csar files can pass VNFSDK checking
    2. VNF license has been created
    Testing Steps
    1. Use SDC to
    onboard
    1. import, test and certify the following vEPC VNFD csar files one by one:
      1. vSPGW
      2. vEPDG
      3. vPCRF
      4. vHSS
      5. vMME
    2. Use SDC to
    onboard
    1. import, test and certify the following vIMS VNFD csar files one by one:
      1. vPCSCF
      2. vI/SCSCF
      3. vTAS
    Expected Results
    1. vEPC
    csar files
    1. VNFs are onboarded successfully
    2. vIMS
    csar files
    1. VNFs are onboarded successfully
    Actual Results
    Conclusion (Pass/Fail)
    Testing LabCMCC OpenLab
    Testing Date
    Tester Name


    V00008: vEPC Service Creation

    Test Case IdV00008
    Test Case NamevEPC Service Creation
    DescriptionThis creates vEPC service
    ReleaseAmsterdam
    Preconditions
    1. All vEPC VNFD csar files can pass VNFSDK checking
    2. All vEPC VNFs have been onboarded
    3. All VNFs images files have been uploaded manually into
    ONAP
    1. VIM
    Testing Steps
    1. Use SDC to create vEPC service with the following VNFs
      1. vSPGW
      2. vEPDG
      3. vPCRF
      4. vHSS
      5. vMME
    2. As part of vEPC service composition, create Virtual Links (VL) to connect the VNFs together according to vEPC service design
    Expected Results
    1. vEPC service is created successfully
    2. vnfmInfo is correctly pointed to vendor's
    VNFM
    1. SVNFM
    2. vEPC service is successfully stored in catalog
    Actual Results
    Conclusion (Pass/Fail)
    Testing LabCMCC OpenLab
    Testing Date
    Tester Name


    V00009: vIMS Service Creation

    Test Case IdV00009
    Test Case NamevIMS Service Creation
    DescriptionThis test creates vIMS service
    which includes VNFs and Network Service
    ReleaseAmsterdam
    Preconditions
    1. All vIMS
    VNFs
    1. VNFD csar files can pass VNFSDK checking
    2. All vIMS VNFs have been onboarded
    3. All VNFs images files have been uploaded manually into ONAP
    Testing Steps
    1. In SDC create vIMS service with the following VNFs
      1. vPCSCF
      2. vI/SCSCF
      3. vTAS
    2. As part of vIMS service composition, create Virtual Links (VL) to connect the VNFs together according to vIMS service design
    Expected Results
    1. vIMS service is created successfully
    2. vnfmInfo is correctly pointed to vendor's
    VNFM
    1. SVNFM
    2. vIMS service is successfully stored in catalog
    Actual Results
    Conclusion (Pass/Fail)
    Testing LabCMCC OpenLab
    Testing Date
    Tester Name


    V00010: WAN Service CreationResource Onboarding (Underlay and Overlay templates are preloaded into SDC, this test case can be removed if not needed anymore)

    Test Case IdV00010
    Test Case NameWAN
    Service Creation
    Resource Onboarding
    DescriptionThis test
    creates
    onboards WAN
    service
    resource using SDC
    ReleaseAmsterdam
    Preconditions
  • All PNFs (PE, DC gateway) resources have been onboarded
  • All required network connection types (VLAN, MPLS L3 VPN, VXLAN, etc) are defined by SDC
    1. WAN underlay network template (.yaml) has been defined and packaged in a CSAR file
    2. WAN overlay network template (.yaml) has been defined and packaged in a CSAR file
    Testing Steps
    1. In SDC create
    WAN service
  • In service composition, pull in 2 PEs
  • In service composition, pull in 2 data center gateways
  • Create underlay network as following:
    1. DC GW1 → PE1 as vlan
    2. DC GW2 → PE2 as vlan
    3. PE1 → PE2 as MPLS L3 VPN
  • Create overlay network DC GW1 → DC GW2 as VXLAN
    1. a VL resource and upload WAN underlay network CSAR file. Make sure controllerInfo property is set to SDNC in the template
    2. In SDC create a VL resource and upload WAN overlay network CSAR file. Make sure controllerInfo property is set to SDNC
    Expected Results
    1. WAN underlay network resource onboarded successfully and shown in catalog
    2. WAN overlay network resource onboarded successfully and shown in catalog
    Expected Results
  • Underlay WAN network created successfully
  • Overlay WAN network create successfully
    Actual Results
    Conclusion (Pass/Fail)
    Testing LabCMCC OpenLab
    Testing Date
    Tester Name


    V00011: VoLTE Service Creation

    Test Case IdV00011
    Test Case NameVoLTE Service Creation
    DescriptionThis test creates VoLTE service. To workaround SDC limitation on nested service support, in R1 we create separate dummy VNFs in VoLTE service, and link those VNFs to vEPC, vIMS and WAN services
    ReleaseAmsterdam
    Preconditions
    1. vEPC service has been created
    2. vIMS service has been created
    3. WAN
    service has been created
    1. underlay and overlay resources have been onboarded
    2. All required ONAP artifacts (DG, DCAE template, Holmes blueprint, SO workflow, etc) have been designed and uploaded to corresponding components
    Testing Steps
    1. In SDC onboard
    the
    1. a dummy VNF, called EPC, which links its property nsServiceInvariantUUID to vEPC service
    using ID
    1. property invariantUUID. Set the VNF controllerInfo property to VFC
    2. Onboard
    the
    1. a dummy VNF, called IMS, which links its property nsServiceInvariantUUID to vIMS service
    using ID
    1. property invariantUUID. Set the VNF controllerInfo property to VFC
  • Onboard the dummy VNF, called WAN, which links to WAN service using ID property. Set the VNF controllerInfo to SDNC
  • Compose VoLTE service with EPC, IMS and WAN dummy VNFs
  • Upload DG artifact in SDC
  • Upload DCAE template
  • Upload SO workflow
    1. Create a new service called VoLTE service and compose it with EPC VNF, IMS VNF and WAN underlay and overlay VL resources
    2. Test, certify
    Test
    1. and approve VoLTE service
    2. Distribute VoLTE service
    Expected Results
    1. VoLTE service is created successfully
    2. VoLTE service is stored in catalog
    3. Service Creation event is sent via DMaaP
    4. Other modules (SO, VID, Usecase UI, CLAMP, VID, etc) receives the event
    5. VoLTE service
    is
    1. CSAR file can be retrieved from SDC catalog by other modules using SDC API
    Actual Results
    Conclusion (Pass/Fail)
    Testing LabCMCC OpenLab
    Testing Date
    Tester Name


    V00012: Closed Loop Configuration

    Test Case IdV00012
    Test Case NameClosed loop configuration
    Description

    This test uses CLAMP to configure closed loop for VoLTE. There are 4 configurations:

    1. Provisional rules for Policy
    2. Correlation rules for Holmes
    3. Holmes microservice Blueprint for DCAE
    4. VES configuration (in R1, there is no configuration needed)
    ReleaseAmsterdam
    Preconditions
    1. DCAE template has been imported (R1 SDC doesn't support DCAE template design)
    2. Holmes blueprint has been imported
    3. Holmes correlation rules have been defined
    Testing Steps
    1. In CLAMP, choose
    which
    1. VNF
    and event
    1. SPGW and events (VNF Fault and VM Fault) to monitor
    2. Bring up CLAMP configuration policy GUI and copy Holmes correlation rules in it
    3. Bring up CLAMP provisional policy GUI and define
    reboot
    1. healing provisional policy
    2. Save the configuration
    Expected Results
    1. Closed loop template and blueprint for Holmes are distributed to DCAE
    2. Configuration policy has been distributed to DCAE
    3. Provisional policy has been distributed to Policy engine
    Actual Results
    Conclusion (Pass/Fail)
    Testing LabCMCC OpenLab
    Testing Date
    Tester Name


    Category: Service Instantiation and Monitoring


    V00013: VoLTE Service Instantiation (WAN service should be independent of VoLTE and instantiated separately, need to discuss)

    Test Case IdV00013
    Test Case NameVoLTE Service Instantiation
    DescriptionThis test covers the whole VoLTE service instantiation process
    ReleaseAmsterdam
    Preconditions
    1. All external systems (VIM, SDN WAN controller, SDN local controller, vendors SVNFM and EMS) have registered with ESR
    2. VoLTE service has been created, V00011 has passed
    3. Closed loop has been configured, V00012 has passed
    4. PEs have been configured and integrated with corresponding SDN WAN controller
    5. DC GWs have been configured and integrated with corresponding SDN local controller
    6. EVPN VXLAN tunnel has been set up manually between edge and core DC gateway. At this moment, underlay network between PEs has not been set up yet, so EVPN and VXLAN tunnel is configured but may not be working, but it's ok.
    7. Install a web server in core data center
    Testing Steps
    1. ONAP user
    uses
    1. logins Usecase UI GUI
    to start VoLTE service instantiation
    1. Usecase UI will guide the user
    to deploy each of vEPC VNFs. User needs
    1. to
    choose data center location and
    1. fill in
    input parameters defined in TOSCA template
  • Usecase UI will guide the user to deploy each of vIMS VNFs. User needs to choose data center location and fill in input parameters defined in TOSCA template
    1. or verify all needed parameters for vEPC, vIMS, WAN underlay and overlay
    2. User will click a button to deploy VoLTE service
    Usecase UI will guide the user to deploy WAN service. User needs to choose PEs and DC GWs, fill in WAN underlay and overlay parameters
    Expected Results
    1. vEPC service is instantiated successfully, including all vEPC VNFs and VLs
    2. vIMS service is instantiated successfully, including all vIMS VNFs and VLs
    3. WAN service is instantiated successfully, including underlay and overlay networks
    4. Holmes analytics container is brought up correctly by DCAE
    5. Client UE can access network through mobile network, e.g. access the web server
    Actual Results
    Conclusion (Pass/Fail)
    Testing LabCMCC OpenLab
    Testing Date
    Tester Name


    V00014: System Performance and Alarm Monitoring

    Test Case IdV00014
    Test Case NameSystem Performance and Alarm Monitoring
    DescriptionThis test uses Usecase UI GUI to monitor system performance and event
    ReleaseAmsterdam
    Preconditions
    1. VoLTE service has been instantiated
    2. Closed loop for VoLTE has been configured and started
    Testing Steps
    1. ONAP user uses Usecase UI GUI to check system performance, such as CPU, memory and disk usage
    2. ONAP user views the VNF alarms collected by ONAP
    Expected Results
    1. System performance (including system CPU, memory and disk usage) is shown on ONAP Usecase UI GUI
    2. VNF alarms, specified in CLAMP for the closed loop, are shown in Usecase UI GUI
    Actual Results
    Conclusion (Pass/Fail)
    Testing LabCMCC OpenLab
    Testing Date
    Tester Name


    Category: Closed Loop


    V00015: VoLTE Auto-healing

    Test Case Id
    V00014
    V00015
    Test Case NameVoLTE Auto-healing
    DescriptionThis test covers the closed loop with alarm correlation and auto-healing
    ReleaseAmsterdam
    Preconditions
    1. VoLTE service has been instantiated
    2. VoLTE closed loop has been configured and started
    3. SPGW VNF supports event generation
    4. VNF guest OS supports event generation
    5. MultiVIM supports event collection from guest OS and event forwarding
    Testing Steps
    Use Robot test script to
    1. Manually shutdown standby MPU VM inside SPGW VNF. This should trigger the VNF and VIM to generate alarms
    Expected Results
    1. EMS collects standby MPU offline event from SPGW VNF and sends the event to VFC
    2. MultiVIM doesn't receive heartbeat event from standby MPU guest OS for a period of time and triggers alarm which is sent to DCAE VES collector
    3. Holmes receives MPU offline events
    4. Holmes receives MPU guest OS heartbeat missing events
    5. Holmes does correlation of the two types of events and generate one signature event and sends it to Policy
    6. Policy triggers provisional policy and
    reboot
    1. healing action is sent to VFC
    2. VFC uses
    VNFM
    1. SVNFM driver to
    reboot
    1. heal standby MPU VM
    2. Standby MPU is
    reboot
    1. healed
    2. Clear event is sent from SPGW VNF about standby MPU back online
    3. Heartbeat from standby MPU VM is received by MultiVIM and clear event is sent by MultiVIM about heartbeat recovery
    4. Holmes does the event correlation and sends Clear signature event to Policy
    5. Policy stops the
    reboot
    1. healing action
    Actual Results
    Conclusion (Pass/Fail)
    Testing LabCMCC OpenLab
    Testing Date
    Tester Name


    Category: Service Termination


    V00015V00016: VoLTE Service Termination

    Test Case Id
    V00015
    V00016
    Test Case NameVoLTE Service Termination
    DescriptionThis test covers VoLTE service termination process
    ReleaseAmsterdam
    Preconditions
    1. VoLTE service has been created
    2. Closed loop has been configured and started
    3. PEs have been configured and integrated with corresponding SDN WAN controller
    4. DC GWs have been configured and integrated with corresponding SDN local controller
    Testing Steps
    1. ONAP user uses Usecase UI GUI to remove VoLTE service
    Expected Results
    1. Closed loop is stopped
    2. vEPC service is removed successfully, including all vEPC VNFs and VLs
    3. vIMS service is removed successfully, including all vIMS VNFs and VLs
    4. WAN service is removed successfully, including underlay and overlay
    networksClosed loop is stopped
    1. network setup
    2. Holmes analytics container is removed by DCAE
    3. Closed loop related artifacts (including DCAE template, configurational policy and provisional policy, DG, workflows) are removed from ONAP
    Actual Results
    Conclusion (Pass/Fail)
    Testing LabCMCC OpenLab
    Testing Date
    Tester Name