WORK IN PROGRESS - focus will be automated testing so some rows will be deleted from the table
Color Guide
For 2019-08-01 - ONLY Completed items should be green.
COMPLETED
PROGRESSING >= 90%
PROGRESSING <90%
| # | Use Case Functional Requirement Nonfunctional Requirement | Test Cases | Lab | Integration Test Lead | Status (remaining days of effort) | Video URL | Detailed Test Status | Quick Notes | Doc status | CI integration* |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Model Driven Control Loop Design | (vDNS is part of scale out use case below #10) | Intel / Windriver | onboard -PASS instantiate - PASS closedloop - PASS COMPLETED | Retest with Policy FIX Robot fix for automation | POLICY-2109 - Getting issue details... STATUS INT-1272 - Getting issue details... STATUS | ||||
| 2 | Use Case: Residential Broadband vCPE (Approved) | vCPE Integration Test Case | Intel / Windriver | infra-PASS bng - PASS gmux - PASS ResCust - distribute PASS ResCust - tunnel-xconn-assign - PASS tunnel-xconn- create - PASS vnf and vfmodule create PASS brg-tunnel config - PASS Traffic Flows - PASS Closed-Loop - gmux → ves → tca → policy - PASS Policy → AAI, Policy → APPC , APPC → Openstack PASS | vcpe.py tooling issues Looks like Error messages back from APPC despite successful restart of VM | SO-2351 - Getting issue details... STATUS SO-2354 - Getting issue details... STATUS SO-2375 - Getting issue details... STATUS INT-1265 - Getting issue details... STATUS SO-2401 - Getting issue details... STATUS SDNC-939 - Getting issue details... STATUS SDNC-940 - Getting issue details... STATUS INT-1323 - Getting issue details... STATUS INT-1324 - Getting issue details... STATUS | ||||
| 3 | vCPE with TOSCA | vCPE(tosca) with HPA - Integration test cases | Intel / Windriver | All test cases have been passed for supporting ovp tosca vnf validation and the following test is triggered by CLI in windriver lab
PASS | ||||||
| 4 | 5G - Realtime PM and High Volume Stream Data Collection | 5G - Real Time PM and High Volume Stream Data Collection - Integration Test Cases | Intel / Windriver | PASS e2e scenarios passed on sb00 CSIT passed in release-daily | https://docs.onap.org/en/elalto/submodules/integration.git/docs/docs_5g_rtpm.html#docs-realtime-pm Reviewers Minor Edit otherwise looks good. https://gerrit.onap.org/r/#/c/integration/+/97217/ | |||||
| 5 | 5G - PNF PnP | 5G - PNF PnP - Integration Test Cases | Intel / Windriver | PASS e2e scenario passed on sb00 | ||||||
| 6 | 5G - Bulk PM | 5G - Bulk PM - Test Status | Intel / Windriver SB-02 | E/// lab COMPLETED | INT-1296 - Getting issue details... STATUS | |||||
| 7 | 5G - Configuration with NETCONF | 5G - PNF Configuration via Netconf - Test Status | Intel/ Windriver | Manual test COMPLETED CSITs are also passing which test this usecase for SOsim-> Blueprint-processor -> SDNC -> PNFsim. link is https://jenkins.onap.org/job/usecases-master-csit-config-over-netconf/. | SO-2349 - Getting issue details... STATUS Testing completed on windriver SB00 with image versions SO-bpmn : 1.5.3 Blueprint processor : 0.6.4-STAGING-latest sdnc-image:1.7.3 prh-app-server:1.3.1 vescollector:1.4.5 | |||||
| 8 | 5G - Preonboarding/Onboarding - Test Status | Intel / Windriver | PNF Preonboarding (VNF SDK): PNF Onboarding (SDC): | Preonboarding PASS Manual test case using | ||||||
| 9 | 5G - OOF and PCI | 5G - OOF and PCI - Integration Test Cases | Windriver | |||||||
| 10 | Scale Out | Scale Out - Integration Test Cases and Status | Intel / Windriver | Retesting using the latest released images. Manual scale out COMPLETED Automated (closed-loop based) scale out: COMPLETED | Manual scale out: it works with direct call to SO APIs. There is an issue with VID that will be addressed in Frankfurt as it requires to rearchitect the current approach: VID-659 - Getting issue details... STATUS | |||||
| 11 | CCVPN | CCVPN Integration Test Cases | CMCC | Regression test for service creation, service change service creation - PASS service change -PASS service deletion - PASS PASS | ||||||
| 12 | HPA vFW/vDNS | vFW/vDNS HPA Testing(Test Plan 1 inHPA & Cloud Agnostic Intent - R3 Test Plan) | Intel / Windriver | COMPLETED | Service instance creation fails because of error that occurs in homing. This bug is currently blocking further testing. A workaround for the SO bug has been created. See the SO bug for details. Now seeing another blocking error in OOF, also blocking service instance creation. D | SDC-2588 - Getting issue details... STATUS SO-2378 - Getting issue details... STATUS OPTFRA-598 - Getting issue details... STATUS OPTFRA-608 - Getting issue details... STATUS | ||||
| 13 | Change Management - Flexible Designer & Orchestrator | Steps: 1) Designing custom workflows with Workflow Designer 2) Distributing custom workflows to SO 3) Deploying the custom workflow in SO 4) Sending workflow metadata to VID 5) Triggering custom workflow execution from VID 6) Executing custom workflows in SO | Intel / Windriver | EL ALTO TEST COMPLETE Work arounds for some functionality. Remaining functions deferred to Frankfurt 1) Complete 2) 90% Complete 3) Complete 4) 85% Complete 5) Complete 6) 80% Complete In-Progress. 9 Building Blocks successfully tested (Check, Set & Unset Maintenance Flags, Check, Set, and Unset Closed Loop Disabled Flag, Lock & Unlock vNF, and End Workflow.). PauseforManual testing in-progress. | SDC-2541 - Getting issue details... STATUS VID-671 - Getting issue details... STATUS (671 upgraded from medium 10/7) VID-672 - Getting issue details... STATUS (672 upgraded from medium 10/7) SO-2403 - Getting issue details... STATUS SO-2434 - Getting issue details... STATUS SO-2433 - Getting issue details... STATUS SO-2441 - Getting issue details... STATUS VID-675 - Getting issue details... STATUS SO-2482 - Getting issue details... STATUS | |||||
| 14 | Steps: 1) Instantiation of vFWDT 2) Configuration of OOF Policies for vFWDT 3) Execution of OSDF/HAS requests for data collection 4) Configuration of VNF profiles in CDT 5) Configuration of Ansible Server 6) Execution of TD workflow | Intel / Windriver | 1) PASS 2) PASS 3) PASS 4) PASS 5) PASS 6) PASS Retesting using the latest released immages. | 2) Possible only on Chrome Browser | Ansible 0.4.4 APPC 1.6.4 PORTAL-760 - Getting issue details... STATUS | |||||
| 15 | Change Management - 5G PNG Software Upgrade | Change Management - 5G PNF Software Upgrade Integration Test Status | Intel / Windriver | Regression Testing for LCM:upgrade-pre-check/upgrade-software/upgrade-post-check/rollback（No Change in el alto） COMPLETD 100% Enbo Wang yaoguang wang | ||||||
| 16 | Change Management - Schedule Optimization | Intel / Windriver | COMPLETD 100%
| OPTFRA-592 - Getting issue details... STATUS | ||||||
| 17 | BBS Broadband Service Use Case (Dublin) | BBS Use Case Tracking (Dublin Release)#Testing | BBS Swisscom Lab | |||||||
| 18 | VSP Compliance Check within SDC (Dublin) - Phase 1 | VSP Compliance Check within SDC - Integration Test Plan | Vodafone | |||||||
| 19 | K8S based Cloud Region Support - vFW Use case | Deploy vFW Helm Chart using Multicloud K8s Plugin in Kubernetes Cloud | intel / Windriver | COMPLETED | Retest distribution with SO 1.5.3 (includes SO-2357 fix) passed. Create and delete verified with fixes provided by two jira's - as noted Closed Loop test worked. | SO-2357 - Getting issue details... STATUS MULTICLOUD-835 - Getting issue details... STATUS OOM-2117 - Getting issue details... STATUS | ||||
| 20 | K8S based Cloud Region Support - EdgeXFoundry Use case | Deploy EdgeXFoundry Helm Chart using Multicloud K8s Plugin in Kubernetes Cloud | Intel/ Windriver | COMPLETED | Onboarding of Edgex chart is ongoing. The tar.gz has been created for the chart. Installation was completed and pods deployed. | |||||
| 24 | ETSI Alignment SO plugin to support SOL003 to connect to an external VNFM | COMPLETED new CSIT is used for automated testing | CSIT testing simulators have been implemented, and SO and SOL003 Adapter have been tested by leveraging simulators and CSIT test scripts | SO-1638 - Getting issue details... STATUS | ||||||
| 25 | Improve our E2E Process Automation | COMPLETED OOM verify Job Deployer COMPLETED – Yuriy Malakov Abdelmuhaimen Seaudi E2E vLB testing:
Current failure status: [closed] - JIRA https://gerrit.onap.org/r/#/c/so/+/96621/ Fails at the SO to CDS step for config assign. "statusMessage": "Exception in org.onap.so.bpmn.infrastructure.flowspecific.tasks.ConfigAssignVnf.preProcessAbstractCDSProcessing java.util.LinkedHashMap cannot be cast to java.lang.String", | ||||||||
| 26 | S3P - Footprint Optimization (Integration with Alpine) | EL ALTO | ||||||||
| 27 | Component/memory optimization: Ongoing Assessment | EL ALTO | ||||||||
| 28 | Global optimization (e.g. shared DB) - stretch goal=shared PostgreSQL; shared MariaDB | DUBLIN aai + sdc: Cassandra sdnc + so : MariaDB | ||||||||
| 29 | S3P- Geo-Redundancy | NO CHANGE | ||||||||
| 30 | SOL 004 (VNF Package Security) | 5G - Preonboarding/Onboarding - Test Status | Ericsson, Intel / Windriver | Preonboarding PASS Manual test case using | ||||||
| 31 | xNF communication security enhancements. | xNF communication security enhancements - Tests Description and Status | NO CHANGE | |||||||
| 32 | Locale/Internationalization language support | NO CHANGE | ||||||||
| 33 | ||||||||||
| 34 | HPA vIPsec | vIPsec HPA testing | Intel / Windriver | COMPLETED | ||||||
| 35 | Health Checks for PODS | Percentage coverage of Health Checks for Running PODS | Integration | |||||||
| 36 | New Installation Tests in 3 labs | |||||||||
| Intel/Windriver | ||||||||||
| Orange (Rocky) | Morgan Richomme | |||||||||
| Ericsson Lab (Rocky) | ||||||||||
| Robot Keyword Testing | ||||||||||
| init_robot | PASS | |||||||||
| init | PASS | |||||||||
| health | PARTIAL | 59*/61 Orange Lab 58*/61 E// Lab (*SO tooling issue) OOF-CMSO SDNC
CDS blueprint (PASS in Orange) | ||||||||
| eteHelm-k8.sh | PARTIAL | 48/49 Intel/Windriver LabOOF-CMSO Tweaks: M4 SDNC and remove log/pomba | ||||||||
| healthdist | PASS | |||||||||
| instantiateVFWCL | PASS | |||||||||
| vfwclosedloop <png_ip> | PASS | |||||||||
| instantiateVFWDT | PASS | |||||||||
| distributeVCPEResCust | PASS | |||||||||
| sdc-dcae-d | ||||||||||
| vcpe.py | init: PASS infra: PASS custservice: PASS loop: PASS noloss: PASS | |||||||||
| postinstall | PASS | |||||||||
| 72 Hour Stability Run | onboard/instantiate | PASS | ||||||||
| closedloop | PASS | |||||||||
HTTP Certificate Expiry Test (verify_cert.sh) | clamp 3/2020 dmaap 3/2020 | root@dev-robot-robot-5f67d595b6-cl6bc:/var/opt/ONAP# ./verify_cert.sh host:aaf-service.onap Not After : Oct 11 01:09:04 2020 GMT host:aai.onap Not After : Sep 27 19:34:54 2020 GMT host:appc.onap Not After : Jul 15 19:49:01 2029 GMT host:clamp-backend.onap Not After : Mar 21 16:16:58 2020 GMT host:clamp-external.onap Not After : Mar 21 16:16:58 2020 GMT host:clamp.onap Not After : Mar 21 16:16:58 2020 GMT host:config-binding-service.onap Not After : Apr 10 09:58:41 2020 GMT host:dcae-cloudify-manager.onap Not After : Apr 10 09:58:41 2020 GMT host:deployment-handler.onap Not After : Apr 10 09:58:41 2020 GMT host:dmaap-bc.onap Not After : Mar 22 17:37:39 2020 GMT host:dmaap-dr-node.onap Not After : Apr 1 09:24:11 2020 GMT host:dmaap-dr-prov.onap Not After : Apr 1 09:49:34 2020 GMT host:msb-eag.onap Not After : Mar 15 02:08:14 2027 GMT host:msb-iag.onap Not After : Mar 15 02:08:14 2027 GMT host:pap.onap Not After : Aug 22 23:43:27 2020 GMT host:portal-app.onap Not After : Apr 15 18:38:36 2020 GMT host:sdc-be.onap Not After : Sep 9 11:33:20 2020 GMT host:sdc-fe.onap Not After : Sep 9 11:33:20 2020 GMT host:sdc-wfd-fe.onap Not After : Aug 18 11:38:03 2015 GMT host:sdnc-portal.onap Not After : Apr 26 17:48:01 2020 GMT host:sdnc.onap Not After : Apr 26 17:48:01 2020 GMT host:vid.onap Not After : Oct 3 11:52:42 2020 GMT | ||||||||
DCAEGEN2-1539 - Getting issue details... STATUS
*: CI integration: the goal is to introduce the use case in CI daily chains for Frankfurt. Ths table could be completed with a CI dashboard where the use cases will be run every day.
As it should be runnable in any lab, some constraints need to be considered:
- no proprietary VNFs
- use cases must be self-content , third party equiments may not be present in all labs.
- infrastructure setup/teardown shall be clear: the use case must be runnable from a clean ONAP installation. These resources must be cleaned at the end of the test ,
- results shall be pushed to a common DB. It is recommended ot use xtesting to run the tests (robotframework, python, bash, unit test drivers)
1 Comment
Brian Freeman
Please update your testing status as you make progress.