Versions Compared

Old Version 237

changes.mady.by.user Morgan Richomme

Saved on

compared with

New Version 238

changes.mady.by.user Morgan Richomme

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

  1. Action point follow-up
    1. Morgan Richomme action morgan initiate Illia Halych committer promotion procedure
      1. done
    2. Morgan RichommeMichał Jagiełło develop basic_vm_macro and basic_cbf_macro
      1. basic_vm ready to be tested, basic_cnf shall be very closed
    3. Morgan Richommecontact Lasse Kaihlavirta to see if CDS regression could be integrated in CSIT
      1. done, no bandwidth available, Jira created
    4. Morgan RichommeAndreas Geisslerlist UI and create a healthcheck test to check the availability of the UI
      1. done, see next section
  2. Admin
    1. Committer promotion: formal vote initiated Committer Promotion Request for Integration Illia Halych
      1. vote in progress, please committer vote before the end of the week
    2. testing docker discussion (formating, unexpected removal of daily build dockers,..)
      1. 2 tickets created on LF
        1. "fresh" xtesting dockers disappear time to time on the nexus https://jira.linuxfoundation.org/plugins/servlet/theme/portal/2/IT-21812
        2. "impossible to release because the tag format of the snapshot docker does not respect some rules" => https://jira.linuxfoundation.org/plugins/servlet/theme/portal/2/IT-21710
        3. xtesting dockers are rebuilt daily based on the branches (guilin/master end of branches of all the components embedded in the docker), CI or end users cannot guess the format imposed by the LF rules (used for components e.g. 6.0-STAGING-20210315T185806Z)
  1. Honolulu
    1. status: Image AddedINT-1883 - Reccurent various errors in smoke tests In Progress
      1. better results on daily master
      2. last patch for the release was the SDNC patch, currently in gate - shall be merged before the TSC
      3. dockers version (on Monday Master) - SO integrated meanwhile, wait for SDNC but otherwise it looks good: https://docs.google.com/spreadsheets/d/1t3GNRtabdkVxG4ZAxqBJ-X7OO-Zv6xRwd8KrR52V9zU/edit?usp=sharing
      4. weekly versions test show also some progress but we still have bad versions: https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-03/29_08-09/security/versions/versions.html, results transfered to seccom (Amy Zwarico )
    2.  lab
      1. azure staging resinstallation initiated
        1. done, keys pushed to the lab, ready to use: https://logs.onap.org/onap-integration/daily/onap_oom_staging_azure_1/2021-03/31_07-06/
      2. windriver: feedback from Intel => recontact them after Honolulu for Hardware upgrade
    3. Automated tests
      1. pnf_macro
        1. issue with the start of the simulator, wait 30s but it takes more time to start in master, a mechanism to check that everything shall be implemented #action Michał Jagiełło add mechanism in simu step to verify that everything is well started before sending a VES event.
      2. basic_clamp
        1. some regression had been introduced in master. Cleanup in progress, tests to be done after the meeting # action morgan re-run a clean basic_clamp on daily
      3. basic_vm_macro
        1. test almost ready, a config-deploy file was requested to complete the deployment, Michał Jagiełło wil include an empty file (nothing to do..) and re-test #action Michał Jagiełło complete basic_vm_macro tests
      4. tern integration in CI
        1. long duration test seen in https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-03/29_08-09/security/tern/index.html but not sure the processing has been executed, no ternenv seen in the master-weekly lab. troubleshooting to be done (same mechanism will be reused for the stability tests.)
      5. Krzysztof Kuzmicki indicates that there are issues on cmpv2 and hv-ves on IPv4/Ipv6 lab due to proxy issues in Nokia lab, errors can be ignored
    4. stability and resiliency testing
  2. Istanbul
    1. first tests to execute the robotframework based tests in external ns
      1. several current robotframework, k8s job healthcheck
    2. url checker for GUI
      1. mail sent, no feedback #action Morgan Richomme create a wiki page to share the link before building the test (could be introduced in Honolulu)
      2. #action all review and complete this patch
    3. requests from new project (OOF, AAI) "on how to bring new tests to integration" => discussion on the best way..
      1. Q&A + official exist but are probably not enough
      2. #action Morgan Richomme create a wiki page to collect integration contributer view
      3. #action all review and complete the page
  3. AoB
    1. Do we have/should we have some Best Practices laid down for various simulators? (Lasse)
      1. the answer is no, we do not have, and yes we should have
      2. #action Morgan Richomme initiate a official doc page on simulator
      3. #action Krzysztof Kuzmicki use NS refactoring to complete this page
  •  Michał Jagiełło pnf_macro add processing for pnf_macro to wait for a clean startup of the simulator (currently time based, and it seems that it takes more time than expected)
  •  Morgan Richomme cleanup basic_clamp and re-run it on a daily master
  •  Michał Jagiełło complete basic_vm_macro
  •  Morgan Richomme create wiki page to refeence the GUI endpoints to be tested
  •  all review the wiki page and add the links of the GUI you are using
  •  Morgan Richomme create a wiki page to improve the process on test integration in CSIT and/or CI from third patries
  •  all review/complete the wiki on process to integrate new external tests in CSIT or CI
  •  Morgan Richomme initiate an official page on simulator
  •  Krzysztof Kuzmicki complete the official simulator page with NS illustration
  •  Bartek Grzybowski review the official simu page (list of usable simu)
  •  Illia Halych review the official simu page (pythonsdk wrapper)

@

  1. Action point follow-up
    1. AP1: Morgan Richomme  amend pythonsdk_tests to include the workaround
      1. new patch submitted yesterday (first one not efficient as no exception was raised by the SDK at this stage, just a 404 but no exception ResourceNotFound)
      2. https://gerrit.onap.org/r/c/testsuite/pythonsdk-tests/+/119655
    2. AP2: Santosh bayas  identify inputs to see if parameters are missing + publish robot code in testsuite for review
    3. AP3: Morgan Richomme  Krzysztof Kuzmicki  check why we have a path difference between execution in the robot pod and in the xtesting docker
      1. not done
    4. AP4: Morgan Richomme  review MR from user-98b79 for the integration of tern in ci-weekly
      1. first attempt done last friday but faield, new attempts to be done, see next section
    5. AP5: Morgan Richomme  contact use case owner to collect the possible needs and remind the need to update the doc
      1. done see next section
    6. AP6: Morgan Richomme  Krzysztof Kuzmicki  create Epic and tasks to detail expectations on robot pod refactoring (alpine, split web/robot, python3, execution from outside the cluster, use python baseline image,....)
      1. done
        Jira
        serverONAP JIRA
        serverId425b2b0a-557c-3c0c-b515-579789cceedb
        keyTEST-318
  2. Admin
    1. update on the use cases (mail)
      1. Use Case for ONAP-based SON for 5G networks: "We do the testing in Winlab (Rutger’s University) within our use case team. So there is no requirement from Integration team for resources (human as well as lab infra). Some automation has been done – see https://gerrit.onap.org/r/gitweb?p=testsuite.git;a=tree;f=robot/testsuites/usecases;h=1690b9d4b7222f3748211df3a539e2a7d48e2a88;hb=HEAD."

      2. E2E Network Slicing use case requirements for Honolulu release: "We do the testing in CMCC lab, Winlab (Rutger’s University), and at Windriver (OOF tenant) within our use case team. So there is no other requirement from Integration team for resources (human as well as lab infra). Automation – we expect to make some progress in Istanbul release."

      3. CCVPN - Transport Slicing use case requirements for Honolulu release  Henry/Lin (some tests automated, dedicated repo created) "We are using our local lab in Ottawa to test the CCVPN use-case.  And in parallel, the integration test of Network Slicing and Transport Slicing is done together with the Network Slicing use case team (i.e., using CMCC lab and Win lab)."
    2. doc updated
    3. Time to think to renew committers (some are no more very active) => suggest to promote Illia Halych
      1. #action morgan initiate promotion procedure
  1. Honolulu
    1. Troubleshooting campaign status: INT-1883 - Reccurent various errors in smoke tests In Progress
      1. good progress on Master, still lots of patches in gate
      2. on daily Master still regular timeouts on basic_vm|cnf|network
        1. these 2E2 basic tests are leveraging A lacarte bpmn, it seems that most of the Service Providers priviledge the macro bpmn
        2. pnf_macro under integration in CI
        3. it would make sense to keep 1 test with a la carte workflow (basic_network) and move basic_vm and basic_cnf in macro Michal Jagiello Michał Jagiełło and Morgan Richomme started working on this task
      3. Daily Guilin very stable - no so many timeouts for the basic_tests...so for sure the change of DB had an impact..we are not very good in evaluating the DB performance...important to address before B&R tests
      1. VID issue fixed
      2. Certificates at large fixed (openstack, AAI,..)
      1. Daily master of yesterday looks fine: https://logs.onap.org/onap-integration/daily/onap_daily_pod4_master/2021-03/23_11-52/
      2. Majority of the issues due to timeout on SO requests
        1. Issues could be due to SO or any other components SO is discussing with including DB
        2. On Guilin it is much more stable and for a full workflow it take usually 2 times less average (1100 versus 500s)
        3. Code did not changes that much => could be due to the DB upgrade
        4. #action morgan/michal: write basic_vm_macro, basic_cnf_macro
        5. Lukasz Rajewski indicates that we must keep the test of a la carte bpmn because they are usefull to specify vnf order and complete macro mode
        6. the goal is not to exclude these tests but to create new one on macro mode which is the main used bpmn
        7. Krzysztof Kuzmicki indicates that some SO slow down are also affecting the pf-registrate, wich is using te macro mode
    3. Automated tests
      1. update on CDS regression test (session with Jozsef Csongva Morgan Richomme
        1. session done last week, it was possible to launch the mock and start running the tests
        2. some issues due to the fact that the test code was not fully up to date regarding master version
        3. #action morgan contact Lasse Kaihlavirta and see if integration in CSIT will nto make more sense
      2. DCAEmod integrated in CI Krzysztof Kuzmicki
        1. integrated in CI
        2. false negative (icon red but test green) due to the fact that the test was not declared in the test DB. Test added, so at teh end of xtesting the status is pushed ot the DB http://testresults.opnfv.org/onap/api/v1/results?case_name=dcaemod
        3. From a dashboard perspective => test in healthcheck = 1 project / Smoke = test involved several projects BUT test still executed as part of smoke test in CI (avoid race condition with DCAE healthcheck)
      3. tern integration in CI failed in last weekly (gitlab-ci under review)
        1. use of a bad env var for the weekly rules change merged to be retested
  2. Istambul
    1. Move Robot pod out of ONAP cluster (including robot)
      1. certificate and robot pod to be executed in onap-testing cluster
        1. deals only with xtesting (not the robot pod)
        2. the idea is to keep the onap namespace cleaned
        3. to be tested, only open question = cm shall be recreated in teh new onap-testing namespace? url seems OK in cm to allow x-namespace exchanges
    2.  PythonSDK
      1. which tests? what is poorly covered?
        1. vFWCL? scaleout?
        2. NBI
        3. basic loop (basic_clamp extended)
        4. ETSI bpmn?
        5. use cases?
    3. Robot
      1. Bell/apex (if not done in Honolulu)
    4. Add a retry capability in CI? other rules (if onap-helm or onap-k8s Fail "too much" => do not execute E2E tests to save CI time?)
      1. Open discussion on the missing tests
        1. Bartek Grzybowski Illia Halych  Backup & Restore
        2. Michał Jagiełło Krzysztof Kuzmicki Service Sanity check. Healthcheck are not enough, they may be PASS when the system is not working at all..Problem reported already (lots of Healthcheck tests do not provide more information that the onap-k8s checking that the pods are up&running). We have several examples when the pods are up&running but it does not wrok. Exception shall not caught to hide the issues. We would need to formalize a little bit to discuss with the PTL.
        3. Andreas Geissler we are missing a simple test checking the UI endpoint
          1. #action andreas morgan send a mail to PTL to collect admin and user UI and create a simple test checking that the UI are available
        4. user-98b79 add clair scan, even if it shall be done at LF, usually showing it in weekly is the fastest way for adoption
    5. CI dashboard: improved dashboard to identify regression over time (see xls graphs) - multi platforms?
  3. AoB

...