...
...
- fabian rouzaut Security tests
Jira server ONAP Jira serverId 425b2b0a-557c-3c0c-b515-579789cceedb key INT-1995
- Lukasz Rajewski Multicloud azure issues
- Lab status
- Failuers on both DT and Nokia labs
- Due to Vexxhost migration a lot of tests failing - there were no docker images
- Istanbul release
- Robot gating
- Plan to create a gating for robot tests - discussion with Sylvain Desbureaux needed if we could do that - gating is already heavy loaded
- Lab status
- several issues on both Orange and DT - now it looks stable
- Changes in ONAP release process Release Process
- Needs to be clarified with David McBride about "Complete E2E testing"
- Integrtion can be resposible only for the automated tests
- We can only rely on the assurance of those responsible for the use case that they have carried out the tests and they are working correctly if they have not automated the process and shared it with the Integration team
- Needs to be clarified with David McBride about "Complete E2E testing"
- DT tests not performed due to change in helm push plugin - https://github.com/chartmuseum/helm-push/
- there is no more `helm push` command
- OOM Makefile uses that command https://gerrit.onap.org/r/gitweb?p=oom.git;a=blob;f=kubernetes/Makefile;h=d994397269d8cdcdb7837f6d8c0c81fc7e714ba1;hb=refs/heads/master#l84
- Get rid of VID:
- VID healtcheck removed and pnf-registrate change also made - wait for DT tests results
- Python ONAP SDK instantiation requires VID objects - it has to be refactored and mark VID module as deprecated → Draft: Resolve "Mark VID as deprecated" (!275) · Merge requests · Orange-OpenSource / lfn / onap / python-onapsdk · GitLab
- Python ONAP SDK:
- new maintainer candidate Michal Chabiera !!!
- before new version release we should remove VID package I think Draft: Resolve "Mark VID as deprecated" (!275) · Merge requests · Orange-OpenSource / lfn / onap / python-onapsdk · GitLab
- new branch - Istanbul
- Repositories archived:
- Update on Pythonsdk_tests gating Morgan Richomme
- Update on helm based components in tests Krzysztof Kuzmicki
- NSA/CISA kubernetes hardening guidance https://kubernetes.io/blog/2021/10/05/nsa-cisa-kubernetes-hardening-guidance/ - there is a tool which can be used to check the cluster security
- We will have a guest next week - fabian rouzaut will talk about the solutions we could use to automate security tests
- Kenny Paul will do a research about a tool to share the knowledge about issues on ONAP. We all agreed that the use of mailing lists is not convenient and it would be good to have a tool to report bugs in ONAP and present possible solutions
- It's needed to get the response from Seshu Kumar Mudiganti if we should change the number of replicas in SO's maria -
Jira server ONAP Jira serverId 425b2b0a-557c-3c0c-b515-579789cceedb key INT-1925
- Lab status
- All daily labs are OK
- ONAP codebase license scan:
- Integration
- Testsuite
- No issue reported on repositories under Integration responsability
- VID is going to be marked as unmaintained and remove from release:
- https://gerrit.onap.org/r/c/testsuite/pythonsdk-tests/+/124507 - pythonsdk-tests change to not use VID API.
- in Python ONAP SDK instantiation requires VID objects - it has to be refactored and mark VID module as deprecated
- Test done on DT daily (after patch in pythonsdk) with no portal/vid
- Portal and VID Healthcheck are FAIl (normal)
- pnf-registrate is fail (call to VID)
- VID is used in the init.sh script that is still used by ONAP end users
- As a consequence removing VID would need adaptation in pnf-registrate and probably in other robot based tests leveraging the init script.
- Action Michał Jagiełło Krzysztof Kuzmicki evaluate the risk...
- No feedback on the archiving of the following repositories:
- integration/simulators/masspnf-simulator
- integration/simulators/dc-simulator
- integration/usecases/mdons
- integration/usecases/bbs
- integration/terraform
- integration/terragrunt
- testsuite/heatbridge
- No feedback
- Note that one repo is finally empty..as the initial code was not passing the verification job and was never amended by the author. It is always difficult to be sure when we create the repo, but we could evantually use a public gitlab repo as a first step to evaluate the code before creating resources for ..nothing in ONAP?
- Action Michał Jagiełło create a ticket to LF to arhive these repositories
- Action Morgan Richomme once officially archive, remove the reference in the integration documentaiton
- Morgan Richomme ran successfully CPS healthcheck test!
- Official docker generated: nexus3.onap.org:10003/onap/xtesting-smoke-usecases-robot-py3:master
- Action Morgan integrate case in CI (already tested in the weekly run this weekend) then in dashboard
- Update on Pythonsdk_tests gating Morgan Richomme
- mechanism seems OK now https://gitlab.com/Orange-OpenSource/lfn/ci_cd/chained-ci/-/pipelines/379154288
- watch changes intestsuite/pythonsdk_tests
- build of pythonsdk docker on the fly
- use this docker during testing part
- First results with 123869-1 but lots of red...patch or still issues, to be investigated https://gitlab.com/Orange-OpenSource/lfn/onap/xtesting-onap/-/pipelines/379369426
- doc
- refactoring of the dependencies as a LF IT changes broke all the doc build chain (ODL , ONAP,..) not the first time....
- Cedric Ollivier (OPNFV/Anuket) fixed similar issues in OPNFV 3 years ago, he suggested the way to follow to be as clean as possible => 2 patches merged in Integration to validate
- Misc
- Julien is leaving bell, thanks for the short but great contributions. He initiated the handover to Bell's colleagues. We hope we could verify his last patch through the new gate...
- Admin
- Lab status
- Data provider repo creation - under review https://gerrit.linuxfoundation.org/infra/c/releng/info-master/+/68853
- Istanbul
- Openstack/Kubernetes supported version, shall we precise, shall we plan tests (with which resources?)
- Openstack - need to be precised
- MSB K8S - helm packages <= 3.5, could be higher version if not any 3.5+ featue is used
- Integration of CPS test in CI Morgan Richomme
- CPS tests require newer version of Robot library (Python 3 based, where we currently use Python 2 based one)
- Istanbul
- Morgan Richomme provide a change https://gerrit.onap.org/r/c/integration/xtesting/+/124428 where CPS tests are going to be executed using separate image (with Python 3.X)
- Jakarta
- robot tests should be migrated to use newer Robot library (Python 3 based) so CPS tests could be executed with the rest of the robot tests
- Openstack/Kubernetes supported version, shall we precise, shall we plan tests (with which resources?)
- Jakarta
- Vijay Venkatesh Kumar DCAE plan to drop cloudify packages support at the begging of Jakarta
- Krzysztof Kuzmicki already working on the tests migration to use helm charts
- there is a risk that tests start failing because of that. We need:
- migrate tests to use helm charts before cloudify support ends
- remove from testsuite tests that use cloudify
- the second opiton can be chosen only if the migration cannot take place
- the team has an idea to create a place where users can share problems they encountered while working with onap and solutions
- using mailing lists is inconvenient
- users could also share an ideas and solutions which are already used by them, that could help sharing a knowledge
- needs to be discussed with doc team Thomas Kulik Eric Debeau / other PTLs
- Bartek Grzybowski mention about https://gerrit.onap.org/r/admin/repos/oom/offline-installer repo where many useful tools are
- Vijay Venkatesh Kumar DCAE plan to drop cloudify packages support at the begging of Jakarta
- Admin
- archiving of integration repositories...if no activity since ...1 year? - report created by Morgan Richomme
- Repositories which are going to be archived short list - need to be discussed via mailing list
- archiving of integration repositories...if no activity since ...1 year? - report created by Morgan Richomme
- Istanbul
- CPS: demo on staging Azure by Aditya Puthuparambil (evaluation complexity for CI integration in Istanbul) Morgan Richomme
- looks it could be used as an integration test "healthcheck" category if it won't use docker-compose but real CPS instance
- review David's JIRA
- many of tasks can be closed (some of them were already closed by David)
- Data provider demo Piotr Stanior
View file name data_provider.pptx height 250 - DT is going to release that tool to ONAP's gerrit
- Robot pod deprecation
- DT tests are executed with disabled robot - let's check the results in one week
- CPS: demo on staging Azure by Aditya Puthuparambil (evaluation complexity for CI integration in Istanbul) Morgan Richomme
pre meeting on CI introduction: CI-CD_introduction.pdf video: https://logs.onap.org/onap-integration/presentations/integration_introductin_CI_09082021.mp4
- Action points
- Question on ESR removal (side effects on smoke tests)
- gating OK + verif on DT daily => green light to remove this component, oom waiting for Michał Jagiełło green light on the gerrit https://gerrit.onap.org/r/c/oom/+/123812
- Question on ESR removal (side effects on smoke tests)
- Admin
- new PTL congratulations Michał Jagiełło all INFO.yaml shall be good
- archiving of integration repositories...if no activity since ...1 year? Morgan Richomme
Jira server ONAP Jira serverId 425b2b0a-557c-3c0c-b515-579789cceedb key INT-1975 - pythonsdk gating chain pythonsdk_tests_gating.pdf
- Seshu Kumar Mudiganti asked for rerun basic_vm stability tests both with 1 and 3 replicas of mariadb_galera
- demo scancode.io? user-98b79
- The curious case of basic_vm_macro Andreas Geissler
- Andreas found an issue with basic_vm_macro test - looks like parameters used during the service instantiation are not used -> fixed by https://gerrit.onap.org/r/c/testsuite/pythonsdk-tests/+/123890
- Istanbul
- review David's JIRA
- weather board???
- doc....
- any news from the use case?
- most of the projects released first version of new dockers..even so
- Data creation project (Michal Chabiera python code / Piotr Stanior DT code / ..) - official doc clearly no more accurate - moved into next week agenda
- Openstack/Kubernetes supported version, shall we precise, shall we plan tests (with which resources?)
- Operation document ?
- component update?
- backup and restore ?
- troubleshooting
- ...
- review David's JIRA
- jakarta
- Lab status
- Orange lab tests results are not stored/tests are not execute for a couple of days Morgan Richomme
- DT lab also
but the lastest results looks promising Looks healthchecks and pnf issues are resolved
- DT tests results are not visble on https://logs.onap.org/ Andreas Geissler Michał Jagiełło
Jira server ONAP Jira serverId 425b2b0a-557c-3c0c-b515-579789cceedb key INT-1956
- Acceptance criterias for Istanbul (TSC Objectives)
- Infrastructure healthcheck: 100%
- Healthcheck: 100%
- Smoke use-cases: 100%
- Security: 100%
- Some errors may be considered as exceptions:
- PODs used for tests - we agree that all tests resources should be created in separate namespace
Jira server ONAP Jira serverId 425b2b0a-557c-3c0c-b515-579789cceedb key TEST-356 - PODs which are no more maintained (e.g. ESR)
- PODs used for tests - we agree that all tests resources should be created in separate namespace
- For all other cases JIRA tickets should be created
- Some errors may be considered as exceptions:
- Stability: 85%
- Istanbul integration blocking points
- Two more open tickets:
Jira server ONAP Jira serverId 425b2b0a-557c-3c0c-b515-579789cceedb key INT-1925 Jira server ONAP Jira serverId 425b2b0a-557c-3c0c-b515-579789cceedb key OOM-2285
- Two more open tickets:
- More advanced SO instantiation test use cases
- Julien Fontaine presented the PythonSDK-tests changes he would like to provide:
- Separate the SDC model and SO instance definitions file - currently keeped in one YAML (like https://gerrit.onap.org/r/gitweb?p=testsuite/pythonsdk-tests.git;a=blob;f=src/onaptests/templates/vnf-services/basic_vm-service.yaml;h=4609bae52edbb8d6e524a170f60e5a9026c4794f;hb=HEAD)
- The instance definition file is going to be rendered using Jinja2 before the test execution
- Change: https://gerrit.onap.org/r/c/testsuite/pythonsdk-tests/+/123136
- Julien Fontaine presented the PythonSDK-tests changes he would like to provide:
...
Jira | ||||||
---|---|---|---|---|---|---|
|
- we should run cleanup for steps which were already executed and passed
- we should create additional K8S resources on separate namespaces and delete them after test exection
...
Jira | ||||||
---|---|---|---|---|---|---|
|
...
...
Jira | ||||||
---|---|---|---|---|---|---|
|
- reproduction is not possible
- documentation is outdated
- there are some steps in description which looks "hacky" (image recreation, restart pods manually)
- What we can do:
- find anyone who can help to reproduce that use case and prepare a nice documentation
- prepare a doc/wiki page with the clear use case acceptance process to avoid that kind of issues in the future
...
- Catherine Lefevre Where the smoke usecases code is stored?
- Lukasz Rajewski Is it possible to run gating tests with custom version of PythonSDK tests?
- Is it possible to execute gating on custom pythonsdk-tests version (branch, gerrit change)? Morgan Richomme Sylvain Desbureaux
Jira server ONAP Jira serverId 425b2b0a-557c-3c0c-b515-579789cceedb key INT-1952
- Is it possible to execute gating on custom pythonsdk-tests version (branch, gerrit change)? Morgan Richomme Sylvain Desbureaux
- Labs status
- Orange daily back to live https://logs.onap.org/onap-integration/daily/onap_daily_pod4_master/2021-07/20_12-44/
- Still PNF macro and pnf-register tests failing
Jira server ONAP Jira serverId 425b2b0a-557c-3c0c-b515-579789cceedb key TEST-351
- Still PNF macro and pnf-register tests failing
- Orange daily back to live https://logs.onap.org/onap-integration/daily/onap_daily_pod4_master/2021-07/20_12-44/
- Lukasz Rajewski did a change for Python ONAP SDK which resolve an issue with basic_vm and basic_cnf tests on SO gating - new version of Python ONAP SDK should be released. Michał Jagiełło checks how to use the newest ONAP SDK for the SO Gating tests
- Kenny Paul will work on the bridge extension (Done)
- Istanbul Integration Blocking points
- Michał Jagiełło - will follow up about pfn registration & pfn macro
- Labs Status - ONAP reporting
- Orange Daily (Christophe Closset , Xue Gao) - Master view
- DT (Andreas Geissler will check the jobs)
- Nokia (Need to follow up with Damian Nowak)
- Ms Azure - https://logs.onap.org/onap-integration/daily/onap_oom_staging_azure_1/2021-04/09_12-18/
installation done on the 9th of April (94 days of uptime)
For the results of the gate it is in the gerrit, look for Azure3 or Azure 4 in the gerrit logs (as it is a pool of gating labs, you have 50% of chance today as we have 2 orange and 2 Azure gating labs up and running)
e.g. Azure 4 result for patchset https://gerrit.onap.org/r/c/oom/+/120213
https://logs.onap.org/onap-integration/gating//120213-7/index.html
- Need to check with Eric Debeau, Sylvain Desbureaux about the expectations
- Healthcheck Infrastructure Results - Xtesting ONAP Results
- Andreas Geissler will look at where the traces are for infrastructure-healthcheck/k8s-teardown
- Security Checks - Istanbul Acceptance Criteria (Eric Debeau is following up)
- Main Integration Tasks - Agile Board - ONAP JIRA - None of us can access to the Board - to be followed up with Morgan Richomme, Sylvain Desbureaux
- Michał Jagiełło - Need to follow-up with the OOM regarding services that are not initialized (no pod assignment, etc.) - helm chart maybe not up-to-date?
- Can we establish the list ?
- Admin
- Istanbul
- python baseline image (patch in gate) user-98b79
- Logs
- 13h30-14h00 UTC: Joined session with SO team to see how to move forward on instantiation tests Seshu Kumar Mudiganti
- Reminder on the tests/test env/... stability_tests_troubleshooting.pdf
- JIRA: INT-1925 - Issues during stability instantiation tests on weekly master Open
- Last release
- 20/06: https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-06/19_22-22/stability/results_instantiation_10_24h//onaptests_bench.html
- 11/06: errors but still some PASS after 24h https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-06/11_22-55/stability/results_instantiation_10_24h//onaptests_bench.html
- 5/06: https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-06/05_11-51/stability/results_instantiation_10_24h//onaptests_bench.html
- 28/05: https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-05/28_22-58/stability/results_instantiation_10_24h//onaptests_bench.html
- AoB
- Introduction of Carriers Integration Leadership - define role/responsabilities based on Min. TSC scope + Team's wishes
#1 Ensure that the Gating is up and running and daily results are available and identify any blocker by raising JIRA ticket
#2 Perform 72H stability test run on any ONAP Release package (final containers), provide report and identify any blocker by raising JIRA ticket
#3 Help to include in CI Chains any new automated test developed by the ONAP Use Case/Requirement Integration Lead and/or any project team
- Admin
- INFO.yaml updated
- Istanbul
- Update on DCAE migration
- CDS tests in Istanbul
- update stability/resiliency
- AoB
- Action point follow-up
- Michał Jagiełło : send a mail to indicate main topic for onapsdk in istanbul
- all cleanup the repo (integration/testsuite/Demo) and remove in master what is not used for istanbul/honolulu
First attempt on security versions but good question rasied regarding the repo. At the moment the code of versions has been externalized in gitlab but not under the LFN umbrella for a PoC. as for xtesting, it shall be brought back either in ONAP repo or in gitlab under LFN organisation. #actin morgan send a mail to LF to see if there are any gitlab space endorsed by LFN that we coudl use otherwise use the old way - which is fine but not as flexible as a gitlab repo.
- Krzysztof Kuzmicki and Michał Jagiełło specify the service to be unique and linked to the use case → [TEST-345] Refactor ves client and mongo db host name usage in CMPv2 test - ONAP JIRA
change done for mongo but still conflict on pfn-simulator ..pnf-macro curse...JIRA created
- Michał Jagiełło create the 2 tags (7.6.0=8.0.0) for pythonsdk
- Istanbul
- Update tern (scan of last baseline images) / Presentation scancode.io user-98b79: 2021_05_02_INT_ScanCodeIO_for_ONAP.pdf
- #action Alexander send a mail to LF IT + S.Winslow + Jessica + TSC, maybe plan a demo for TSC?
- Demo Litmus for resiliency testing (drain node scenario) Natacha Chéreau: chaos_english.pdf
- Lived demo with the eviction of a compute node - Fail as APPC DB was not properly evicted, all the other pods seem to have been evicted properly
- Goal is to automate drain experiment on weekly and test other experiments (pod, kafka, cassandra,;..)
- Demo simu Krzysztof Kuzmicki
- quick demo, the NS-Simulator provides lots of features, it has been documented in https://docs.onap.org/projects/onap-integration/en/latest/simulators/nf_simulator.html#nf-simulator
- note the ves part is used in pnf-macro
- Results last stability tests / DB tuning
- results of weekly honolulu (without tuning) and weekly master with tuning are available
- both chains were able to perform the 2 stability tests during 24h
- SDC
- weekly honolulu: https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-05/28_22-58/stability/results_sdc_5_24h/onaptests_bench.html.gz
- weekly master: https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_honolulu/2021-05/27_10-47/stability/results_sdc_5_24h/onaptests_bench.html.gz
- Cassandra tuning improves results...to be applied
- Instantiation
- weekly honolulu: https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_honolulu/2021-05/27_10-47/stability/results_instantiation_10_24h/onaptests_bench.html.gz
- weekly master: https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-05/28_22-58/stability/results_instantiation_10_24h/onaptests_bench.html.gz
- mariadb tuning not good, better performance (duration at the beginning) then unrecovered crash...is it due to the tining or any divergence between master and honolulu..?
- until it is clearer => do not apply the change
- Update tern (scan of last baseline images) / Presentation scancode.io user-98b79: 2021_05_02_INT_ScanCodeIO_for_ONAP.pdf
- AoB
- DDF meeting next week, meeting canceled
- Action point follow-up
- Morgan Richomme dig a little bit to understand if CSIt use cases it corresponds to active use cases > not done
- Lukasz Rajewski slide deck vFW CNF Use Case Demo > done
- Konrad Bańka Lukasz Rajewski Morgan Richomme estimate what is missing for integration in CI > not done
- Morgan Richomme send a mail to the community to get feedback on this xNF status page > not done
- Admin
- SECCOM reco / baseline images
- Patch created based on SECCOM reco
- Baseline images created (currently in gate) https://gerrit.onap.org/r/c/integration/docker/onap-java11/+/121545, https://gerrit.onap.org/r/c/integration/docker/onap-python/+/121544
- repo cleanup
- #action all cleanup the repo (integration/testsuite/Demo) and remove in master what is not used for istanbul/honolulu
- versioning
- xtesting images can be created now - 1.7.0 created for guilin..bad version 7.0.0 shall be created as well as 8.0.0 for honolulu (shall be possible now..)
- in progress....
- onapsdk => new version for guilin (7.6.0) and for honolulu (8.0.0)
- agreed => 2 versions needed #action michal create the 2 tags (7.6.0=8.0.0)
- xtesting images can be created now - 1.7.0 created for guilin..bad version 7.0.0 shall be created as well as 8.0.0 for honolulu (shall be possible now..)
- Future PTL: how to move forward: houston.pdf
- SECCOM reco / baseline images
- Honolulu
- integration of pnf_macro
- issue with mongo service when run in // with pnf-registrate, #action Krzysztof and Michal specify the service to be unique and linked to the use case
- stability integration in CI
- now included in weekly master chain
- currently limited to 24 because the system is breaking before 72h
- DB tuning in progress
- no more Maria DB issue, issues on ONAP side (SO or subsequent component for instantiation tests and data middleware for SDC)
- now included in weekly master chain
- move back to pythonsdk xtesting version corresponding to the chain version (was master for all) => guilin backporting required
- that is why we need a "good" version for onapsdk..guilin is consuming guilin xtesting, honolulu/honlulu, master/master
- resiliency follow-up -teasing for next week presentation by Natacha Chéreau
- see you next week for a short presentation and a demo
- integration of pnf_macro
- Istanbul
- discussions on CI/CD category:
- shall we split smoke into smoke (E2E test to validate the release) and use cases
- stability category added in honolulu, shall we add a resiliency category or put them all together
- postponed to next week
- onap-k8s update
- add the generation of a onap_versions.json to automate the container tracking
- onap_versions.json generated => shall be able to track the docker version changes on daily
- onapsdk main dev for Istanbul Michał Jagiełło
- #action michal send a mail to indicate main topic for onapsdk in istanbul
- dcae test update Krzysztof Kuzmicki
- postponed
- Epic for istanbul to be created
- discussions on CI/CD category:
- AoB
- Thanks Lasse Kaihlavirta for the great work done since more than 2 years for integration project.
Action point follow-up
- Lukasz Rajewski prepare a demo on test automation managed by Lukasz..why it could not be integrated in CI for the moment (reflexion for Istanbul)
- Morgan Richomme sent a mail to PTL on good practice. Reminder that no patch in gate oldes than x months is a criteria for milestones..it is clearly not respected.
- Lasse Kaihlavirta plan a 20' section for a full session
- Admin
- committers
- baseline images
- repo cleanup
- Status on CSIT Lasse Kaihlavirta : CSIT status after Honolulu.pptx
- CPS = Showcase example, everything is fine
- CCSDK previous showcase, still requires distributtion of docker builds, not possible to rebuild locally
- DMaaP (dmaap/buscontroller moved), OPTF (optf/odf and optf/has already moved), SDNC (almost finished but still some failures) transition in progress
- AAI mumtitenancy done outside ONAP planned to be incoprorated into aai/resources and aai/traversal => use of scala+gatling (examples based on robot so far)
- Policy: CSIT moved to reir repositories but tests do not follow reco (test after merge, not project specific, jjb fully specific
- Still some active projects with old design (DCAEGEN2, Modeling, Multicloud, OOM-platform-cert-service, SO)
- Usecase #action Morgan Richomme dig a little bit to understand if it corresponds to active use cases
- VFC and VID do not seem really maintained
- VNFSDK: only images for frankfurt => not maintained
- CSIT tests disabled (aaf, appc, clamp, music)
- clamp moved to SDC => what does it mean from a CSIT perspective
- Demo vFW CNF Lukasz Rajewski Konrad Bańka
- slide deck: #action Lukasz Rajewski vFW CNF Use Case Demo
- use case partially automated but would require a KUD (test cannot use a ns of the existing k8s hosting ONAP)
- As the use case is maintained and includes regularly feature changes, it woudl be interesting to include it in CI..to show that it is red/green as part of the use cases (more than smoke tests)
- #action Morgan Richomme Konrad Bańka Lukasz Rajewski estimate what is missing for integration in CI
- Open question and where the status of the xNF shall be available for ONAP admin..not clear status of the service/vnf/module was available in VID, some test status was available in SDC but not sure to know where the info on the status of the xNF would be the more accurate #action Morgan Richomme send a mail to the community to get feedback on this xNF status page
- AoB
- Action point follow-up
- AP1: Morgan Richomme review Integration simu and release versions if possible + update doc accordingly
- not done yet wait for release of the xtesting dockers
- AP2: all move or closed the JIRA in honolulu
- done
- AP3: report your feedback on honolulu lesson learned
- to be done
- AP4: participants indicate their top 3 priorities for honolulu (https://etherpad.anuket.io/p/onap_integration_istanbul)
- done
- AP1: Morgan Richomme review Integration simu and release versions if possible + update doc accordingly
- Admin
- Admin status (PTL, election, INFO.yaml)
- congratulations Bartosz Gardziejewski elected as integration committer
- congratulations Bartosz Gardziejewski elected as integration committer
- Admin status (PTL, election, INFO.yaml)
- Morgan Richomme send a reminder for Alexander currently 4/4 but more than for 4 votes needed
- all the INFO.yaml actions will be performed once
- PTL no candidate => escalation done to TSC
- xtesting docker release
- new way, need to twist the scripts but could be OK. Need to fix guilin CI chain first (issue due to new pbr value)
- baseline images: wait for seccom reco for istanbul
- Morgan Richomme contact Seccom to prepare baseline images. user-98b79 indicates that we need to discuss offline especially for the python image.
- Honolulu
- regression on multicloud-k8s / pnf-macro
issue due to helm format, no error with the helm charts sued for Lukasz's use case. Fix done, docker images must be regeneratedJira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key MULTICLOUD-1348 - action Lukasz Rajewski prepare a demo on test automation managed by Lukasz..why it could not be integrated in CI for the moment (reflexion for Istanbul)
- master-weekly
- tern
- should be fully OK next week
- last results : https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-05/07_22-21/security/tern/index.html
- user-98b79 indicates that anothe resource from Saùmsung is working on the topic to improve the images. No feedback from the PTL.Morgan Richomme indicates that he reports the issue to PTL during PTL meeting. Not normal to have no feedback (e.g. CLI improvement patch in gate for a long time), David McBride sent a mail to user-67d6f #action morgan sent a mail to PTL on good practice. Reminder that no patch in gate oldes than x months is a criteria for milestones..it is clearly not respected.
- stability tests
- update on MariaDB tuning => reduce the replica of non core components/removal of appc/... see stability_follow-up 2-2.pdf
- redimensioning is enough to avoid DB crash but recommendation to TSC to reduce replica for non core components 15 instances of Maria DB => 15x30 Go of additional RAM needed to avoid crash in load conditions...9 nodes cassandra...
- stability tests are now performed on each weekly => results are here: https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-05/07_22-21/stability_tests/
- choice of the figures for weekly
- start with instantiation to avoid lots of models in SDC
- 10 instantiation in // during 24h (basic_vm / a la carte bpmn)
- then 5 onboarding in // during 24h
- needs to add basic_vm_macro if stability reached on first instantiation
- DB tuning
- suggestions from Samuel Liard to modify Maria Db and Cassandra DB default configuration: https://gerrit.onap.org/r/c/oom/+/121229/10
- all integration committers are using teh default DB values on their lab, no specific tuning, only the nb of compoentns may be reduced (because not used)
- update on MariaDB tuning => reduce the replica of non core components/removal of appc/... see stability_follow-up 2-2.pdf
- tern
- CSIT status after Honolulu (we may have to reserve some time for this so not necessarily today): Integration
- introduction to the status of CSIT at the end of Honolulu
- #action Lasse Kaihlavirta plan a 20' section for a full session
- regression on multicloud-k8s / pnf-macro
- Istanbul
- New functional tests
- Do we have resources / Shall we consider the existing list and select some to automate?
- scale out
- vfw CL
- Software upgrade
- MDONS
- 5G slicing
- ...
- pythonsdk
- Current Open issues: https://gitlab.com/Orange-OpenSource/lfn/onap/python-onapsdk/-/issues
- onaptests priorities
- update pnf_macro with Eli's simulator wrapper?
- to be reprogrammed whith Michał Jagiełło Andreas Geissler Illia Halych
- Do we have resources / Shall we consider the existing list and select some to automate?
- Backup and Restore
- tooling: Velero? other solutions? Any feedback
- Scenario? what can we automate?
- #action Morgan Richomme send a mail to the community/task force to converge on a scenario of tests that coudl be used for istanbul
- Operations
- Upgrade of a pod / chart
- Stability
- include a test with basic_vm_macro
- tuning of the DB
- Resiliency
- Integration of a k8s chaos framework?
- New functional tests
- AoB
- if ready Krzysztof Kuzmicki could also make a demo of the simulator
- Action point follow-up
- Morgan Richomme review Integration simu and release versions if possible + update doc accordingly
- not done yet..several issues on LF (SSO, Nexus, gerrit,..)
- Morgan Richomme create a JIRA for basic_vm_macro
- not needed last results with CCSDK fix look good
- user-98b79 finalize push to LF back end
- test in progress on weekly master..config is good
- Morgan Richomme create resiliency chapter in doc and initiate Jira for confirmed issue (appears at least 2 times), restart of a controller still to be done to complete the test.
- done see next section
- Krzysztof Kuzmicki create a wiki page in Istanbul page for robot refactoring follow-up
- Krzysztof Kuzmicki create a wiki page in Istanbul page for DCAE migration
- Morgan Richomme review Integration simu and release versions if possible + update doc accordingly
- Admin
- INFO.yaml updated needed (NS repo have no yaml + status of the project)
- Commiter Promotion Request for Bartosz Gardziejewski: results 10 Yes on 10
- Commiter Promotion Request for Alexander Mazuruk: results 4 Yes on 4
- PTL election: call for self nomination sent
- all operations on INFO.yaml will be done at the same time
- note Lasse Kaihlavirta indicated that he also will step down in 1 month
- Honolulu
- status:board review https://jira.onap.org/secure/RapidBoard.jspa?rapidView=229
- few JIRA still open # action all move or closed the JIRA in honolulu
- release testsuite 1.8.0 done
- just merged
- doc updated
- done
- Remaining work on tests
- macro issues Lukasz Rajewski
stability/resiliency tests
Maria DB galera troubleshooting
- honolulu_sign-off.pdf
- stability_follow-up.pdf
- Honolulu: Lessons Learned
- #action report your feedback on honolulu lesson learned
- status:board review https://jira.onap.org/secure/RapidBoard.jspa?rapidView=229
- Istanbul
- New Jira board created: https://jira.onap.org/secure/RapidBoard.jspa?rapidView=233
- brainstorm on new use cases for Istanbul open => https://etherpad.anuket.io/p/onap_integration_istanbul
- participants indicate their top 3 priorities for honolulu
- AoB
- Morgan Richomme review Integration simu and release versions if possible + update doc accordingly
- all move or closed the JIRA in honolulu
- report your feedback on honolulu lesson learned
- Action point follow-up
- Morgan Richomme review Integration simu and release versions if possible + update doc accordingly
- not done, will be done after doc, stability and resiliency testing #reaction
- Lukasz Rajewski properly tag SO bugs to reference them as blocking in Integration blicking table
- SO bugs do not appear in Honolulu Integration Blocking Points => to be crosscheck with Lukasz Rajewski
- if SDNC-1515 merged and pnf-simu OK in honoluluMorgan Richomme add pnf-macro to CI
- honolulu daily shows that the basic_vm_macro issue is not really fully fixed => #action Morgan Richomme create a JIRA for basoc_vm_macro
- Morgan Richomme grant access to Christophe Closset to check the status of SDC/DB
- done see section on SDC stability test
- Morgan Richomme retry and wait for the 5 minutes (Eviction timeout) then investigate on the networking issue
- done, Jira completed
- Morgan Richomme check ansible image availability for chained-ci
- doneAndreas Geissler gave a new try, still issue but not related to teh image. open question: shall we also reference the image in the CI page of the release?
- Morgan Richomme review Integration simu and release versions if possible + update doc accordingly
- Admin
- INFO.yaml updated needed (NS repo have no yaml + status of the project)
- Commiter Promotion Request for Bartosz Gardziejewski: please vote before end of the week. BTW if other committer proposal => feel free to suggest (to replace Pawel, Marcin and partly Thierry)
- Honolulu
- status:board review https://jira.onap.org/secure/RapidBoard.jspa?rapidView=229
- release testsuite 1.8.0 under built
- in progress https://gerrit.onap.org/r/c/testsuite/+/120960, once merged, submit a patch to OOM
- Remaining work on tests
- macro issues Lukasz Rajewski
- pnf-macro test to be done on master before integration in CI (working fine in guilin)
- basic_vm_macro good results on guilin but not so good on master/honolulu => JIRA created
Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key SDNC-1529
- tern: test run in CI but results not pushed yet to LF Backend => WIP to manage properly the push to lf backend. On last honolulu and master-weekly, artifacts were produced and pushed manually #action user-98b79 finalize push to LF back end
stability/resiliency tests
Resiliency test
- worker node restart (thanks Bartek Grzybowski for the full analysis) => how do we follow, shall we create Jiras?
- test done, eviction looks good. some issues detected on some pods but not always trivial to reproduce may depend on the associated evicted pods. sometimes Init error, sometimes pod Running but exception..sometimes first eviction look good, but second seems to be fail... different cases referenced in Honolulu Resiliency and Backup and Restore test
- for SDC issue, the restart may be due to teh fact that the SDC is trying to recreate tables that already exist in cassandra
- #action Morgan Richomme create resiliency chapter in doc and initiate Jira for confirmed issue (appears at least 2 times), restart of a controller still to be done to complete the test.
stability tests
- SDC tests: 72h 5 parallel on boarding INT-1912 - SDC Stability test: success rate drops after ~ 500 onboarding Open
- Christophe Closset reports that it was useful to detect new issues, especially the continuous time increasing that my be lead to teh DB and/or a middle ware layer (explicit exception seen)
- plan to integrate such test in weekly as part of benchmark test
- instantiation tests
- 48h 5 parallel basic_vm (initial issue on SDNC (fixed by restart) then regular timeout value then internal openstack issues INT-1918 - Stability tests: // instantiation Open
- more timeout than in guilin, need to be confirmed with a 1 test (no parallelization), mariadbgalera issue observed on some replica
- 72h 1 test basic_vm => problem with mariadb galera, results will not be as good as in guilin run
- issue with mariadbgalera after 24h (but tests run after the 48h tests..) once finished (thurday morning => reinstallation of honolulu weekly to reproduce the tests with the last dockers
- mariadb galera workaround reported by Bell to OOM, seems to improve the overall quality of gate but trigger an new issue on CDS. tradefoff between redundancy and efficiency not clear
- macro issues Lukasz Rajewski
- documentation update started but not finished => https://gerrit.onap.org/r/c/integration/+/120236
- estimation 2-3 days still needed to compelte the doc / release note + testsuite integration
- Istanbul
- New Jira board created: https://jira.onap.org/secure/RapidBoard.jspa?rapidView=233
- robot refacroting..it eems we have a plan (Krzysztof Kuzmicki )
- discussion by mail, #action Krzysztof Kuzmicki crerate a wiki page in Istanbul page
- migration of DCAE from cloudify to helm chart..first analysis by Krzysztof Kuzmicki , impacts to be planned on several tests
- discussion by mail, #action Krzysztof Kuzmicki crerate a wiki page in Istanbul page
- New CSIT (AAI, CDS,..)
- AAI is folowwing the SDC way (Maven based), for CDS it shoudl be possible to reuse what Lasse Kaihlavirta suggested. AAI team contacted SDC
- brainstorm on new use cases for Istanbul open => https://etherpad.anuket.io/p/onap_integration_istanbul
- all review the page, it is an etherpad..so do not refrain yourself..put everything you have in mind
- AoB
- Lasse Kaihlavirta closed the CSIT refactoring EPIC
- Lasse Kaihlavirta also disabled useless SO CSIT tests (with very old images) - the Jira was closed without action by Seshu Kumar Mudiganti so we remove some jobs as they are meaningless (test with casablanca images)
- Lasse Kaihlavirta add a browser cleanup on robot healthcheck (vid) to save resource, it will be integrated in 1.8.0
- Action point follow-up
- Illia Halych review the official simu page (pythonsdk wrapper)
- done, no example for Honolulu, to be completed in Istanbul
- Morgan Richomme refine the information about the conditions of the resiliency tests of worker restart -
- See Honolulu topic
- Lasse Kaihlavirta Offline discussions with Lasse about AAI CSIT
- Done
- Krzysztof Kuzmicki Next week proposal about what we can do with robot
- Postponed to next week
- Krzysztof Kuzmicki INT-1907 - browser_setup.robot does not provide proper teardown Open no teardown for browser-based checks - high consumption of resources at the end
- used by VID test, workaround to be confirmed
- Maybe we need more information about DMaaP simulator use and to write it better in the simulator doc → Reported by Lasse Kaihlavirta
Globally we are not well structured on the simulator, on the release note of version X we should list all the versions of the simulators we used for the validation. It shall be done by the use case teams that developed a simu and:or the integration team. The difficulty ois to get the full view on the simulators, some are hidden in repositories, some are unmaintained,...# action Morgan Richomme review Integration simu and release versions if possible + update doc accordingly
- Need of consolidation of versioning of images of simulators used in CSIT → Reported by Lasse Kaihlavirta
- sure it is a bit messy...and impossible to maintain, integration can cleanup time to time..but the best way for functional tests => bring functional tests including simu in their own repo. And if the simu can be used more widely create a dedicate repo under integration/simulators..
- Illia Halych review the official simu page (pythonsdk wrapper)
- Admin
- INFO.yaml updated needed (NS repo have no yaml + status of the project)
- strange INFO.yaml is needed to create the repo => to be xchecked. Morgan Richomme indicates to Thomas Kulik that it will be done with the next INFO update (needed as at least 2 committers announced they will stop after Honolulu)
- thanks to Pawel and Marcin for their contributions and all the best for the next challenges
- INFO.yaml updated needed (NS repo have no yaml + status of the project)
- Honolulu
- status:board review https://jira.onap.org/secure/RapidBoard.jspa?rapidView=229
- Remaining work on tests
- pnf_macro => OK on daily guilin, what do we do regarding CI integration
kudos to Michał Jagiełło - first pythonsdk tests with simulator - let's wait for the merge of Dan fix for SDNC-1515, if OK #action Morgan Richomme add pnf-macro to CI
- stil SO bugs preventing Lukasz Rajewski to complete his test .. deja-vu?
- #action Lukasz Rajewski properly tag SO bugs to reference them as blocking in Integration blicking table
- tern: test run in CI
- test results available (launched manually) => https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-04/19_18-36/security/tern/index.html
- next CI run shall be good..wait and see
stability/resiliency tests
I started the first resiliency tests: TEST-308 - [Resiliency] Evaluate ONAP behavior on k8s worker node restart In Progress , replay done, mail sent to the community (network issue?)
- behavor different on Nokia (pod stuck for ever in Terminating state when stopping a working) and Samsung RKE2 cluster, continue discussion on the ticket Bartek Grzybowski indicates that we shall be careful especially with statefulset, it coudl explain some issues
- #action morgan retry and wait for the 5 minutes (Eviction timeout) then investigate on the networking issue
stability tests
- SDC tests: 72h 5 parallel on boarding started on the 20th of April..wait and see...
- First tests were all OK but duration ontinuously increases. No error rate is high. Wait for the end of the tests to get the graphs.
- #action Morgan Richomme grant access to Christophe Closset to check the status of SDC/DB
- instantiation tests to be planned after onboarding tests
- SDC tests: 72h 5 parallel on boarding started on the 20th of April..wait and see...
consolidation of versions
better exception catching + integration of the GUI => still action Morgan
removal of the submodule: done
- pnf_macro => OK on daily guilin, what do we do regarding CI integration
- documentation update started but not finished => https://gerrit.onap.org/r/c/integration/+/120236
- release testsuite 1.8.0 to be done?
- next week if no objection
- simulators: whuch dockers are available in nexus today? could we clarify the build chain?
- see previous topic
- Istanbul
- New Jira board created: https://jira.onap.org/secure/RapidBoard.jspa?rapidView=233
- discussion on future tests
- follow up integration AAI tests in CSIT
- CDS in CSIT?
- AoB
- Andreas Geissler mentioned that his CI chains were affected by the non availability of an ansible image used in the runner => could be a problem, need to release also such images somewhere to avoid such issues #action Morgan Richomme check ansible image availability for chained-ci (2.7.13)
- issue also on Orange chains last week (build chain of ansible image was broken, chain fixed)
- Christophe Closset raises a question on python2.7 in jenkins CI and in robot. Partly addressed by the initiative of robot refactoring. note xtesting dockers also still using python2.7 due to dependency to python-utils that has not evolved for a long time.
- Andreas Geissler mentioned that his CI chains were affected by the non availability of an ansible image used in the runner => could be a problem, need to release also such images somewhere to avoid such issues #action Morgan Richomme check ansible image availability for chained-ci (2.7.13)
- Morgan Richomme review Integration simu and release versions if possible + update doc accordingly
- Lukasz Rajewski properly tag SO bugs to reference them as blocking in Integration blicking table
- if SDNC-1515 merged and pnf-simu OK in honoluluMorgan Richomme add pnf-macro to CI
- Morgan Richomme grant access to Christophe Closset to check the status of SDC/DB
- Morgan Richomme retry and wait for the 5 minutes (Eviction timeout) then investigate on the networking issue
- Morgan Richomme check ansible image availability for chained-ci
- Action point follow-up
- Michał Jagiełło pnf_macro add processing for pnf_macro to wait for a clean startup of the simulator (currently time based, and it seems that it takes more time than expected)
- in progress
- Michał Jagiełło complete basic_vm_macro
- done
- all review the wiki page and add the links of the GUI you are using
- all review/complete the wiki on process to integrate new external tests in CSIT or CI
- Krzysztof Kuzmicki complete the official simulator page with NS illustration
- documentation is done
- Illia Halych review the official simu page (pythonsdk wrapper)
- no feedback
- all review your Jira ticket move to instanbul or close
- Morgan Richomme review the patch dealing with sumbmodule removal + cherry + adapt xtesting-onap
- done
- Morgan Richomme complete wiki instanbul page to track this AAI scenario
- done, thanks to Lasse Kaihlavirta to ahve provided more examples.
- Michał Jagiełło pnf_macro add processing for pnf_macro to wait for a clean startup of the simulator (currently time based, and it seems that it takes more time than expected)
- Admin
- good occasion to discuss on the future organization and the future PTL..
- instanbul renaming (instead of istambul) done in seccom and oparent repo
- Honolulu
- status:board review https://jira.onap.org/secure/RapidBoard.jspa?rapidView=229&selectedIssue=INT-1842
- Daily Honolulu CI/CD chain created
- still sometimes some issues at SDNC startup (
, Dan Timoney already added check mechanism but it seems that there are still issue in mariadbgalera, workaround (restart of SDNC seems to be OK)Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key SDNC-1512 - basic_vm_macro does not work due to SDNC (
)Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key SDNC-1515 - basic_vm and basic_cnf fail randomly - mostly due to SDC problem or timeout in SO - this looks less stable than in guilin
- observed problem with config-assign/config-deploy in vFW-CNF integration test
- we need to cover config-assign/config-deploy in our smoke testsJira server ONAP JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 425b2b0a-557c-3c0c-b515-579789cceedb key SO-3626 - to sum up: we have two major problems with service-macro-create
- still sometimes some issues at SDNC startup (
- Remaining work on tests
basic_clamp: OK
basic_vm_macro: OK show a regression in SDNC (
)Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key SDNC-1515 pnf_macro => Michał Jagiełłois working on the latest comments and needs further tests
refactoring of 5gbulkpm => OK (additional patch required submitted by Krzysztof Kuzmicki and cherry-picked)
tern: test declared in CI => we went further on the weekly-honolulu (https://gitlab.com/Orange-OpenSource/lfn/onap/xtesting-onap/-/jobs/1165577708), ansible-playbook was missing on the runner...fix merged (we need to use image with already ansible installed) ...to be continued but at least it is properly triggered
stability tests
I started the first resiliency tests:
, replay today - we need to add remark this is only for stateless services - what about DB?Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key TEST-308 stability I planned to regive a try to Natacha's suite as soon as the other tests are OK
consolidation of versions
better exception catching
integration of the GUI part to generate an html reporting (for the moment I usually replay manually the tests, get the results and apply the GUI processing manually)
removal of the submodule: test in progress (seems there was an import in the simu step), once done we could review our docker build (to remove the developer mode) but we shall not forget to adapt xtesting-onap according ly (/Src/onaptests/src/onaptests => /usr/lib/python3.8/sites-available/onaptests => action morgan for the week end..to avoid breaking everything before my PTO..
- documentation update started => https://gerrit.onap.org/r/c/integration/+/120236
- release testsuite 1.8.0 to be done
- Istanbul
- discussion on future tests
- follow up integration AAI tests in CSIT
- AOB
- Illia Halych review the official simu page (pythonsdk wrapper)
- Morgan Richommerefine the information about the conditions of the resiliency tests of worker restart -
- Lasse Kaihlavirta Offline discussions with Lasse about AAI CSIT
- Krzysztof Kuzmicki Next week proposal about what we can do with robot
- Krzysztof Kuzmicki
no teardown for browser-based checks - high consumption of resources at the endJira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key INT-1907 - Maybe we need more information about DMaaP simulator use and to write it better in the simulator doc → Reported by Lasse Kaihlavirta
- Need of consolidation of versioning of images of simulators used in CSIT → Reported by Lasse Kaihlavirta
...
- Action point follow-up
- Michał Jagiełło pnf_macro add processing for pnf_macro to wait for a clean startup of the simulator (currently time based, and it seems that it takes more time than expected)
- Not done
- Morgan Richomme cleanup basic_clamp and re-run it on a daily master
- Done
- Michał Jagiełło complete basic_vm_macro
- In progress, issue in SDNC:
Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key SDNC-1515
- In progress, issue in SDNC:
- Morgan Richomme create wiki page to reference the GUI endpoints to be tested
- Done Honolulu UIs
- all review the wiki page and add the links of the GUI you are using
- Morgan Richomme create a wiki page to improve the process on test integration in CSIT and/or CI from third parties
- all review/complete the wiki on process to integrate new external tests in CSIT or CI
- Morgan Richomme initiate an official page on simulator
- Krzysztof Kuzmicki complete the official simulator page with NS illustration
- To be Done
- Bartek Grzybowski review the official simu page (list of usable simu)
- Done
- Illia Halych review the official simu page (pythonsdk wrapper)
- To be Done
- Michał Jagiełło pnf_macro add processing for pnf_macro to wait for a clean startup of the simulator (currently time based, and it seems that it takes more time than expected)
- Admin
- Illia Halych officially new Integration commiter, welcome !
- No update on xtesting docker release
- LF IT backend , not possible to move logs, no cleanup mechanism (manual cleanup needed)
- Honolulu
- status:board review https://jira.onap.org/secure/RapidBoard.jspa?rapidView=229&selectedIssue=INT-1842
- action #all review your Jira ticket move to instanbul or close
- Daily Honolulu CI/CD chain created
- Remaining work on tests
basic_clamp: integrated in CI, several successful runs, but improvements needed to manage exceptions (Policy/DCAE/CLamp) => https://gerrit.onap.org/r/c/testsuite/pythonsdk-tests/+/120234
basic_vm_macro (replace vfw_macro): in progress still some issues on the daily honolulu
pnf_macro => add mechanism to have a better control of the pnf simu launch (timer is not enough) => # action Michal
refactoring of 5gbulkpm => #action Krzysztof => test done this afternoon => merged
tern: test declared in CI, but issue => # action morgan & Alexander
stability tests
I started the first resiliency tests: TEST-308 - [Resiliency] Evaluate ONAP behavior on k8s worker node restart
- Other labs are experiencing different issues
- Nokia => pod remain in stuck
- #action Daniel, Andreas create a JIRA ticket for each resiliency test
- #action morgan create a wiki page to consolidate resiliencey tests
- Other labs are experiencing different issues
stability I planned to regive a try to Natacha's suite as soon as the other tests are OK
- to be tried asap: if enough resources => weekly Honolulu to be launched on Wednesday
consolidation of versions: I was waiting for the come back of Pawel but as far as I understood, you will move to othe r challenges soon.. :) so I need to review that
better exception catching
integration of the GUI part to generate an html reporting (for the moment I usually replay manually the tests, get the results and apply the GUI processing manually)
removal of the submodule: test in progress (seems there was an import in the simu step), once done we could review our docker build (to remove the developer mode) but we shall not forget to adapt xtesting-onap according ly (/Src/onaptests/src/onaptests => /usr/lib/python3.8/sites-available/onaptests
- #action morgan review the patch dealing with sumbmodule removal + cherry + adapt xtesting-onap
CDS / CSIT => will not have the resource to do it for Honolulu => move to Istanbul
GUI tests: wait for completion of the Wiki page and see if enough bandwidth => move to instanbul
- documentation update started
- honolulu branching done
- release testsuite 1.8.0 to be done
- status:board review https://jira.onap.org/secure/RapidBoard.jspa?rapidView=229&selectedIssue=INT-1842
- Istanbul
- requests from new project (OOF, AAI) "on how to bring new tests to integration" => How to help projects to integrate their tests in Integration?
- Discussion on the support for AAI gatling tests. By default the multitenancy support is disabled and they require a keycloak third party
- first issue is the test env => we coudl imagine to have a daily scenario daily_master_multitenancy but we need to take care of the resources. Until we cannot rely on the windriver lab, it will be hard to create new scenarios...
- installation of keycloak shall be done in a tooling namespace through a helm chart => OOM
- Mohammad Hosnidokht indicated that gatling part is almost automated (docker verifying the roles)
- William Reehil indicated that it will probably break the other tests
- it is probably more relevant to start by creating a CSIT test (fonctional test) with AAI + keycloak rather than a daily..
- #action Morgan Richomme complete wiki instanbul page to track this scenario
- run robot tests out of onap namespace: How to run the tests out of the ONAP cluster?
- requests from new project (OOF, AAI) "on how to bring new tests to integration" => How to help projects to integrate their tests in Integration?
- AoB
- Michał Jagiełło pnf_macro add processing for pnf_macro to wait for a clean startup of the simulator (currently time based, and it seems that it takes more time than expected)
- Michał Jagiełło complete basic_vm_macro
- all review the wiki page and add the links of the GUI you are using
- all review/complete the wiki on process to integrate new external tests in CSIT or CI
- Krzysztof Kuzmicki complete the official simulator page with NS illustration
- Illia Halych review the official simu page (pythonsdk wrapper)
- all review your Jira ticket move to instanbul or close
- Morgan Richomme review the patch dealing with sumbmodule removal + cherry + adapt xtesting-onap
- Morgan Richomme complete wiki instanbul page to track this scenario
- Action point follow-up
- Morgan Richomme action morgan initiate Illia Halych committer promotion procedure
- done
- Morgan RichommeMichał Jagiełło develop basic_vm_macro and basic_cbf_macro
- basic_vm ready to be tested, basic_cnf shall be very closed
- Morgan Richommecontact Lasse Kaihlavirta to see if CDS regression could be integrated in CSIT
- done, no bandwidth available, Jira created
- Morgan RichommeAndreas Geisslerlist UI and create a healthcheck test to check the availability of the UI
- done, see next section
- Morgan Richomme action morgan initiate Illia Halych committer promotion procedure
- Admin
- Committer promotion: formal vote initiated Committer Promotion Request for Integration Illia Halych
- vote in progress, please committer vote before the end of the week
- testing docker discussion (formating, unexpected removal of daily build dockers,..)
- 2 tickets created on LF
- "fresh" xtesting dockers disappear time to time on the nexus https://jira.linuxfoundation.org/plugins/servlet/theme/portal/2/IT-21812
- "impossible to release because the tag format of the snapshot docker does not respect some rules" => https://jira.linuxfoundation.org/plugins/servlet/theme/portal/2/IT-21710
- xtesting dockers are rebuilt daily based on the branches (guilin/master end of branches of all the components embedded in the docker), CI or end users cannot guess the format imposed by the LF rules (used for components e.g. 6.0-STAGING-20210315T185806Z)
- 2 tickets created on LF
- Committer promotion: formal vote initiated Committer Promotion Request for Integration Illia Halych
- Honolulu
- status: INT-1883 - Reccurent various errors in smoke tests In Progress
- better results on daily master
- last patch for the release was the SDNC patch, currently in gate - shall be merged before the TSC
- dockers version (on Monday Master) - SO integrated meanwhile, wait for SDNC but otherwise it looks good: https://docs.google.com/spreadsheets/d/1t3GNRtabdkVxG4ZAxqBJ-X7OO-Zv6xRwd8KrR52V9zU/edit?usp=sharing
- weekly versions test show also some progress but we still have bad versions: https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-03/29_08-09/security/versions/versions.html, results transfered to seccom (Amy Zwarico )
- lab
- azure staging resinstallation initiated
- done, keys pushed to the lab, ready to use: https://logs.onap.org/onap-integration/daily/onap_oom_staging_azure_1/2021-03/31_07-06/
- windriver: feedback from Intel => recontact them after Honolulu for Hardware upgrade
- azure staging resinstallation initiated
- Automated tests
- pnf_macro
- issue with the start of the simulator, wait 30s but it takes more time to start in master, a mechanism to check that everything shall be implemented #action Michał Jagiełło add mechanism in simu step to verify that everything is well started before sending a VES event.
- basic_clamp
- some regression had been introduced in master. Cleanup in progress, tests to be done after the meeting # action morgan re-run a clean basic_clamp on daily
- basic_vm_macro
- test almost ready, a config-deploy file was requested to complete the deployment, Michał Jagiełło wil include an empty file (nothing to do..) and re-test #action Michał Jagiełło complete basic_vm_macro tests
- tern integration in CI
- long duration test seen in https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-03/29_08-09/security/tern/index.html but not sure the processing has been executed, no ternenv seen in the master-weekly lab. troubleshooting to be done (same mechanism will be reused for the stability tests.)
- Krzysztof Kuzmicki indicates that there are issues on cmpv2 and hv-ves on IPv4/Ipv6 lab due to proxy issues in Nokia lab, errors can be ignored
- pnf_macro
- stability and resiliency testing
- status: INT-1883 - Reccurent various errors in smoke tests In Progress
- Istanbul
- first tests to execute the robotframework based tests in external ns
- several current robotframework, k8s job healthcheck
- url checker for GUI
- mail sent, no feedback #action Morgan Richomme create a wiki page to share the link before building the test (could be introduced in Honolulu)
- #action all review and complete this patch
- requests from new project (OOF, AAI) "on how to bring new tests to integration" => discussion on the best way..
- Q&A + official exist but are probably not enough
- #action Morgan Richomme create a wiki page to collect integration contributer view
- #action all review and complete the page
- first tests to execute the robotframework based tests in external ns
- AoB
- Do we have/should we have some Best Practices laid down for various simulators? (Lasse)
- the answer is no, we do not have, and yes we should have
- #action Morgan Richomme initiate a official doc page on simulator
- #action Krzysztof Kuzmicki use NS refactoring to complete this page
- Do we have/should we have some Best Practices laid down for various simulators? (Lasse)
- Michał Jagiełło pnf_macro add processing for pnf_macro to wait for a clean startup of the simulator (currently time based, and it seems that it takes more time than expected)
- Morgan Richomme cleanup basic_clamp and re-run it on a daily master
- Michał Jagiełło complete basic_vm_macro
- Morgan Richomme create wiki page to refeence the GUI endpoints to be tested
- all review the wiki page and add the links of the GUI you are using
- Morgan Richomme create a wiki page to improve the process on test integration in CSIT and/or CI from third patries
- all review/complete the wiki on process to integrate new external tests in CSIT or CI
- Morgan Richomme initiate an official page on simulator
- Krzysztof Kuzmicki complete the official simulator page with NS illustration
- Bartek Grzybowski review the official simu page (list of usable simu)
- Illia Halych review the official simu page (pythonsdk wrapper)
- Action point follow-up
- AP1: Morgan Richomme amend pythonsdk_tests to include the workaround
- new patch submitted yesterday (first one not efficient as no exception was raised by the SDK at this stage, just a 404 but no exception ResourceNotFound)
- https://gerrit.onap.org/r/c/testsuite/pythonsdk-tests/+/119655
- AP2: Santosh bayas identify inputs to see if parameters are missing + publish robot code in testsuite for review
- AP3: Morgan Richomme Krzysztof Kuzmicki check why we have a path difference between execution in the robot pod and in the xtesting docker
- not done
- AP4: Morgan Richomme review MR from user-98b79 for the integration of tern in ci-weekly
- first attempt done last friday but faield, new attempts to be done, see next section
- AP5: Morgan Richomme contact use case owner to collect the possible needs and remind the need to update the doc
- done see next section
- AP6: Morgan Richomme Krzysztof Kuzmicki create Epic and tasks to detail expectations on robot pod refactoring (alpine, split web/robot, python3, execution from outside the cluster, use python baseline image,....)
- done
Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key TEST-318
- done
- AP1: Morgan Richomme amend pythonsdk_tests to include the workaround
- Admin
- update on the use cases (mail)
- Use Case for ONAP-based SON for 5G networks: "We do the testing in Winlab (Rutger’s University) within our use case team. So there is no requirement from Integration team for resources (human as well as lab infra). Some automation has been done – see https://gerrit.onap.org/r/gitweb?p=testsuite.git;a=tree;f=robot/testsuites/usecases;h=1690b9d4b7222f3748211df3a539e2a7d48e2a88;hb=HEAD."
E2E Network Slicing use case requirements for Honolulu release: "We do the testing in CMCC lab, Winlab (Rutger’s University), and at Windriver (OOF tenant) within our use case team. So there is no other requirement from Integration team for resources (human as well as lab infra). Automation – we expect to make some progress in Istanbul release."
- CCVPN - Transport Slicing use case requirements for Honolulu release Henry/Lin (some tests automated, dedicated repo created) "We are using our local lab in Ottawa to test the CCVPN use-case. And in parallel, the integration test of Network Slicing and Transport Slicing is done together with the Network Slicing use case team (i.e., using CMCC lab and Win lab)."
- doc updated
- Time to think to renew committers (some are no more very active) => suggest to promote Illia Halych
- #action morgan initiate promotion procedure
- update on the use cases (mail)
- Honolulu
- Troubleshooting campaign status: INT-1883 - Reccurent various errors in smoke tests In Progress
- good progress on Master, still lots of patches in gate
- on daily Master still regular timeouts on basic_vm|cnf|network
- these 2E2 basic tests are leveraging A lacarte bpmn, it seems that most of the Service Providers priviledge the macro bpmn
- pnf_macro under integration in CI
- it would make sense to keep 1 test with a la carte workflow (basic_network) and move basic_vm and basic_cnf in macro Michał Jagiełło and Morgan Richomme started working on this task
- Daily Guilin very stable - no so many timeouts for the basic_tests...so for sure the change of DB had an impact..we are not very good in evaluating the DB performance...important to address before B&R tests
- VID issue fixed
- Certificates at large fixed (openstack, AAI,..)
- Daily master of yesterday looks fine: https://logs.onap.org/onap-integration/daily/onap_daily_pod4_master/2021-03/23_11-52/
- Majority of the issues due to timeout on SO requests
- Issues could be due to SO or any other components SO is discussing with including DB
- On Guilin it is much more stable and for a full workflow it take usually 2 times less average (1100 versus 500s)
- Code did not changes that much => could be due to the DB upgrade
- #action morgan/michal: write basic_vm_macro, basic_cnf_macro
- Lukasz Rajewski indicates that we must keep the test of a la carte bpmn because they are usefull to specify vnf order and complete macro mode
- the goal is not to exclude these tests but to create new one on macro mode which is the main used bpmn
- Krzysztof Kuzmicki indicates that some SO slow down are also affecting the pf-registrate, wich is using te macro mode
- Automated tests
- update on CDS regression test (session with Jozsef Csongva Morgan Richomme
- session done last week, it was possible to launch the mock and start running the tests
- some issues due to the fact that the test code was not fully up to date regarding master version
- #action morgan contact Lasse Kaihlavirta and see if integration in CSIT will nto make more sense
- DCAEmod integrated in CI Krzysztof Kuzmicki
- integrated in CI
- false negative (icon red but test green) due to the fact that the test was not declared in the test DB. Test added, so at teh end of xtesting the status is pushed ot the DB http://testresults.opnfv.org/onap/api/v1/results?case_name=dcaemod
- From a dashboard perspective => test in healthcheck = 1 project / Smoke = test involved several projects BUT test still executed as part of smoke test in CI (avoid race condition with DCAE healthcheck)
- tern integration in CI failed in last weekly (gitlab-ci under review)
- use of a bad env var for the weekly rules change merged to be retested
- update on CDS regression test (session with Jozsef Csongva Morgan Richomme
- Troubleshooting campaign status: INT-1883 - Reccurent various errors in smoke tests In Progress
- Istambul
- Move Robot pod out of ONAP cluster (including robot)
- certificate and robot pod to be executed in onap-testing cluster
- deals only with xtesting (not the robot pod)
- the idea is to keep the onap namespace cleaned
- to be tested, only open question = cm shall be recreated in teh new onap-testing namespace? url seems OK in cm to allow x-namespace exchanges
- certificate and robot pod to be executed in onap-testing cluster
- PythonSDK
- which tests? what is poorly covered?
- vFWCL? scaleout?
- NBI
- basic loop (basic_clamp extended)
- ETSI bpmn?
- use cases?
- which tests? what is poorly covered?
- Robot
- Bell/apex (if not done in Honolulu)
- Add a retry capability in CI? other rules (if onap-helm or onap-k8s Fail "too much" => do not execute E2E tests to save CI time?)
- Open discussion on the missing tests
- Bartek Grzybowski Illia Halych Backup & Restore
- Michał Jagiełło Krzysztof Kuzmicki Service Sanity check. Healthcheck are not enough, they may be PASS when the system is not working at all..Problem reported already (lots of Healthcheck tests do not provide more information that the onap-k8s checking that the pods are up&running). We have several examples when the pods are up&running but it does not wrok. Exception shall not caught to hide the issues. We would need to formalize a little bit to discuss with the PTL.
- Andreas Geissler we are missing a simple test checking the UI endpoint
- #action andreas morgan send a mail to PTL to collect admin and user UI and create a simple test checking that the UI are available
- user-98b79 add clair scan, even if it shall be done at LF, usually showing it in weekly is the fastest way for adoption
- Open discussion on the missing tests
- CI dashboard: improved dashboard to identify regression over time (see xls graphs) - multi platforms?
- Move Robot pod out of ONAP cluster (including robot)
- AoB
- Morgan Richomme action morgan initiate Illia Halych committer promotion procedure
- Morgan Richomme Michał Jagiełło develop basic_vm_macro and basic_cbf_macro
- Morgan Richomme contact Lasse Kaihlavirta to see if CDS regression coudl be integrated in CSIT
- Morgan Richomme Andreas Geissler list UI and create a healthcheck test to check the availability of the UI
- Action point follow-up
- AP1 Illia Halych : ask OOM for the right place to host helm chart
- unformal exchange. For CDS helm charts are stored intestsuite/cds/mock-* until we coudl move the helm charts to a helm registry
- AP2 Andreas Geissler add warning in doc for backup and restore
- patch done
- AP3 Morgan Richomme contact TSC regarding B&R task force and possible scenario for Honolulu
- done, mail sent, Catherine Lefevre answered and agreed on shortmid term plans
- AP4 Morgan Richomme test basic-pnf (macro) on daily master before integrating thsi use case in CI
- done, test issues reported to Michał Jagiełło and added in
Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key INT-1822
- done, test issues reported to Michał Jagiełło and added in
- AP1 Illia Halych : ask OOM for the right place to host helm chart
- Admin
- congratulation to the team for the S3P..now it will be cool to fix the issues in Master/Gating
- windriver re-installation: to be planned after honolulu => answer sent to Intel team
- Azure Staging lab reinstalled
- #acrion Morgan Richomme send a mail to the community to indicate that staging azure is back (guilin maintenance release as master is broken)
- congratulation to the team for the S3P..now it will be cool to fix the issues in Master/Gating
- Honolulu
- Troubleshooting campaign status: INT-1883 - Reccurent various errors in smoke tests In Progress
- several issues found
- SDC races => including offset to move to pseudo sequential seems to reduces the number of SDC issues. Still one where ceritification seems to take to much time => possibility to add a workaround in the test to retry and/or delay the creation/certification
- #action Morgan Richomme amend pythonsdk_tests to include the workaround
- VID 500 due to the presence of a "U" char in random password (which explains why we did not have always the error...à - deobuscated by VID even if not obfuscated using jetty library interprating U as unicode char breaking everything. Problem found and fixed by Krzysztof Opasiak => last gating seems reasonably better
- SO error (so-bpmn stuck) was alsready observed time to time in guilin, so it is not a new problem...it explains why sometimes an instantiation takes 3 minutes sometimes 20, a jIRA had been created on this aspect
Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key SO-3419 - rebase/remerge in gate in progress...gates will burn over the next days...but it means that master chains should be back and usable
- SDC races => including offset to move to pseudo sequential seems to reduces the number of SDC issues. Still one where ceritification seems to take to much time => possibility to add a workaround in the test to retry and/or delay the creation/certification
- several issues found
- Automated tests
- demo Apex pdp Policy Santosh bayas Prudence Au
- test done on Bell labs, adaptation needed to be intgrated in CI (config in a dedicated file not leveraging CM information)..
- #Action Santosh bayas identify inputs to see if parameters are missing + publish robot code in testsuite for review
- test done on Bell labs, adaptation needed to be intgrated in CI (config in a dedicated file not leveraging CM information)..
- CDS regression test (session with Jozsef Csongvai planned on thursday to give a try on Orange lab and see how we can integrate in CI - helm chart finally with docker file in mock repositories)
- mock in gates
- troubleshooting planned on thursday
- DCAEmod, 5Gbulkpm evolution (Krzysztof Kuzmicki ) → Morgan Richomme
- Could we move this point to next week when bulkpm over HTTPS server will be finished and DCAEMOD in daily CI ?
- #action Morgan Richomme Krzysztof Kuzmicki check why we have a path difference between execution in the robot pod and in the xtesting docker
- pythonsdk (pnf, clamp, .., removal of the submodule, patch to allow test replay)
- tern integration in CI user-98b79 Morgan Richomme - how to test/troubleshoot ansible par
- user-98b79 provides follow up in the MR https://gitlab.com/Orange-OpenSource/lfn/onap/xtesting-onap/-/merge_requests/37
- #action Morgan Richomme reriew MR for the integration of tern in ci-weekly
- #agreed moved weekly reinstallation on Friday next week
- user-98b79 provides follow up in the MR https://gitlab.com/Orange-OpenSource/lfn/onap/xtesting-onap/-/merge_requests/37
- demo Apex pdp Policy Santosh bayas Prudence Au
- S3P tests
- back on B&R mail to the community => only resilience tests to be planned for Honolulu
- stability => target?
- wait for master stabilization
- Figures to be decided: 5 // onboarding during 48, 20 // instantiation during 48?
- feed back on first test executed with litmus
- k8s chaos monkey framework
- it was possible to anote onap component and generate chaos..coudl be used for resiliency testing to check that components are recovering
- use cases
- who knows the list of Honolulu use cases?..
- Lukasz Rajewski shared the link with the Honolulu Release Requirements
- no requests received so far
- no documentation changes seen
- #action Morgan Richomme contact use case owner to collect the possible needs and remind the need to update teh doc
- who knows the list of Honolulu use cases?..
- Troubleshooting campaign status: INT-1883 - Reccurent various errors in smoke tests In Progress
- Istambul => for information to be discussed next week
- Move Robot pod out of ONAP cluster (including robot)
- certificate and robot pod to be executed in onap-testing cluster
- CI dashboard: improved dashboard to identify regression over time (see xls graphs) - muti platforms?
- PythonSDK
- which tests? what is poorly covered?
- vFWCL? scaleout?
- NBI
- basic loop (basic_clamp extended)
- which tests? what is poorly covered?
- Add a retry capability in CI? other rules (if onap-helm or onap-k8s Fail "too much" => do not execute E2E tests to save CI time?)
- Robot refactoring: target (slot planned after the meeting if we are short in time)
- #action Morgan Richomme Krzysztof Kuzmicki create Epic and tasks to detail expectations on robot pod refactoring (alpine, split web/robot, python3, execution from outside the cluster, use python baseline image,....)
- Move Robot pod out of ONAP cluster (including robot)
- AoB
- AP1: Morgan Richomme amend pythonsdk_tests to include the workaround
- AP2: Santosh bayas identify inputs to see if parameters are missing + publish robot code in testsuite for review
- AP3: Morgan Richomme Krzysztof Kuzmicki check why we have a path difference between execution in the robot pod and in the xtesting docker
- AP4: Morgan Richomme review MR from user-98b79 for the integration of tern in ci-weekly
- AP5: Morgan Richomme contact use case owner to collect the possible needs and remind the need to update the doc
- AP6: Morgan Richomme Krzysztof Kuzmicki create Epic and tasks to detail expectations on robot pod refactoring (alpine, split web/robot, python3, execution from outside the cluster, use python baseline image,....)
...
- AP1: Andreas Geissler check if doc of the official use cases are excplictely referencing the robot pod
- not done yet
- AP2: Morgan Richomme include Nokia pod in onap-integration web site
- AP3: all help reviewing the OOM patches to give Integration feedback
- in progress see next section
...
last master daily results
...
Issues
- VID: VID-960 - VID exception on E2E tests In Progress
- 500 on SDC (multi retry on SDC component get versions)
- gitlab.com issues (runner problems)
...
- daily master: https://logs.onap.org/onap-integration/daily/onap_daily_pod4_master/2021-03/10_06-03/
- daily guilin: https://logs.onap.org/onap-integration/daily/onap_daily_pod4_guilin/2021-03/10_05-03/
...
something seems broken since the 19/2...only 3 patches have been merged around this date: 117743, 117936, 117753
...
- sequential versus parallel
- tests planned to move to E2E sequential to avoid side effect due to parallelization. Parallel tests could be done on weekly...need to fist master and gating chains first.
- sequential versus parallel
- tests update (Michał Jagiełło 15' )
- pnf (macro flow)
- ready to be tested (logger fixed)
- #action morgan test basic-pnf on daily
- exceptions not caught
- patch provided
- basic_clamp
- almost there: last issue due to DCAE TCA, seems tehre is a regression on dcae due to python2.7 / Python3 =>
. Wait for fix before retrying.Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key INT-1822
- almost there: last issue due to DCAE TCA, seems tehre is a regression on dcae due to python2.7 / Python3 =>
- wrapper
- OK, question on where to host the helm charts for the simulator (pnf and future one)
- #action contact OOM team => similar question than the for CDS mock objects
- it is possible to remove the submodule - just be carefull, once removed => possible to deploy onapsdk as a "standard module" no more as developer module => impact on xtesting-onap as the paces will be different
- pnf (macro flow)
- Backup/restore Morgan Richomme 15' Backup_and_restore_discussion.pdf
- discussion on a realistic use case...
- backup and restore seems too optimistic
- consensus to work on the resiliency of the core component
- #action Andreas Geissler warning shall be added in the documentation, as aarna network is not really usable for a real backup/restore (more some examples to use velero)
- Companies that tried velero has issues due to teh fact that ONAP is not really cloud native
- discussion on a realistic use case...
- AoB
- cmpv2 about to be moved from healthcheck to smoke
- Krzysztof Kuzmicki indicates that 5Gbulkpm will be updated ..may have some turbulences during the update
Actions
- AP1 Illia Halych : ask OOM for the rright place to host helm chart
- AP2 Andreas Geissler add warning in doc for backup and restore
- AP3 Morgan Richomme contact TSC regarding B&R task force and possible scenario for Honolulu
- AP4 Morgan Richomme test basic-pnf (macro) on daily master before integrating thsi use case in CI
- Action point follow-up
- no action point from last week (several docs)
- action point from the 17/02
- AP1: Morgan Richomme : contact vcpe tosca team to get their position in Honolulu (
)Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key TEST-239 - done no feedback on the JIra
- AP2: user-98b79 : get the list of projects using the baseline images
- done
- AP3: Illia Halych amend https://gerrit.onap.org/r/c/testsuite/pythonsdk-tests/+/117192 for helm3 support
- done
- AP4: Morgan Richomme test basic_pnf on Orange daily master with onaptests:master
- not done PTO..but I can reaction myself;.
- AP5: Michał Jagiełło Morgan Richomme test carefully before submitting until we got a gate to do it...
- not really an action point but the advise is wise...be carefull after removing the submodules + change the xtesting installation (using standard python module not developer way...xtesting-onap shall follow)
- AP6: Krzysztof Kuzmicki prepare a 10 minutes demo on pnf-simu to nf-simu (explaning the graph shared by mail)
- done
- AP7: Marcin Przybysz plan a demo of VID new UI, how to launch it, how it is used in PNF macromode + see with VID how we can manage tests that are using legacy VID..
- done
- AP8: Morgan Richomme contact Kenny Paul for the status of the slack chan
- done it is a free chan, no idea how LF decides to pay for a non free chan..if community thinks that we shall be able to keep the history and then take a non free chan, it is possible to report it to the TSC
- AP1: Morgan Richomme : contact vcpe tosca team to get their position in Honolulu (
- Admin
- A1 policy enforcement created
- nf repo created (Krzysztof Kuzmicki )
- M3 is over..JIRA review needed to know what is possible (support/test cases/) what shall be postponed...
- release cadence is new, some projects were surprised. No new code will be accepted in OOM gate. Lots of patches submitted...gating system under pressure..wait for merge to reach the RC0
- Honolulu (M3 is over..)
- CSIT issue with python2.7/Python3 (Lasse Kaihlavirta Morgan Richomme )
- fix submitted by user-98b79
- list of jenkins images known in CSIT repo but not sure it is documented. Some issues due to python 2.7/3.5/3.6/3.7/3.8/3.9...lib dependency issues NB the seccom recommended version is the 3.9
- Refactoring of testsuite
- python2.7/micro service/root
- cert
- helm chart
- Lukasz Rajewski and Andreas Geissler suggest to put the effort on migrating to pythonsdk rather than maintaining the pod
- Morgan Richomme reminds that robot tests are run through pythonsdk, the question is linked to the robot pod not the ability to run robotframework test cases, which is always possible. Who is using the e2e scripts that trigger processing on the robot pod. Lasse Kaihlavirta indicates that some usecases (under upstreaming (A1)) are in this case. For Honolulu we may ask for a waiver and remove the pod in Istambul (is nobody is using it). Morgan Richomme sent a mail to the community but no feedback so far. Scaleout and vFWCL still probably requires the pod, other usercases requires also the init offered by the bash scripts #action Andreas Geissler look in the doc which use cases explicitely make reference to the scripts in the official documentation
- status on CI/CD (delta guilin) (Morgan Richomme )
- New Nokia dualstack lab added in the pool of lab reporting results http://testresults.opnfv.org/onap/api/v1/results?last=5&case_name=full&pod=ci-dualstack-master-daily
- IPv4/IPv6 server, ONAP Master daily => results shared with the community in LF backend + Test DB
- shall we update http://testresults.opnfv.org/onap-integration/ ?
- Daniel Milaszkiewicz OK #action morgan add Nokia in http://testresults.opnfv.org/onap-integration/
- Master is currently not that bad: https://logs.onap.org/onap-integration/daily/onap_daily_pod4_master/03-01-2021_06-03/
- error due to internal cert executed in onap ns (shall be moved to an external ns)
- 1 error on enhanced policy => policy anticipated the OOM merge and already adapted the test, it shall be back OK once the new policy dockers woudl be merged
- 1 error in E2E due to SDC (exception not caught)
- lots of patch in OOM gate..if lots of errors it is not normal as the master is reasonably stable...
- #action all help reviewing the OOM patches to give Integration feedback
- Guilin is not good
- same CI errors in DT and Orange
- runnign the tests directly is usually good => issue in the way to launch the test in guilin
- "old" ansible version in the xtesting-onap branch
- tests in progress to use xtesting-onap master (assuming that the test version is provided by a env variable already) => no need of specific branch
- delta guilin/current master shared with the community
- https://docs.google.com/spreadsheets/d/1t3GNRtabdkVxG4ZAxqBJ-X7OO-Zv6xRwd8KrR52V9zU
- as usual some projects anticipate and provided features continuously (Policy, DCAE, NBI, OOF, OOM), some did not...which explains the gating bottlneck due to the M3
- New Nokia dualstack lab added in the pool of lab reporting results http://testresults.opnfv.org/onap/api/v1/results?last=5&case_name=full&pod=ci-dualstack-master-daily
- security test update
- tern (user-98b79 )
- last results: https://logs.onap.org/onap-integration/daily/onap_weekly_pod4_master/02-25-2021_02-02/security/tern/
- patch submitted to fix some projects ..but they have no gating so manual test needed
- first patch submitted
- advantage, it will be easier to chase CVE if we can rely on baseline images
- issues with Python (and lib dependencies) but few components dealing with python
- next steps work on teh integration as a weekly job in xtesting-onap to run automatically tern everyweek
- versions still some issue in weekly chain + reporting to be integrated
- patch merged for the bug
- results of last week shared => https://logs.onap.org/onap-integration/daily/onap_weekly_pod4_master/02-25-2021_02-02/security/versions/versions/index.html
- core/internal nodeports tests can be long ..any idea how to parallelize
- wait for Paweł Wieczorek come back as same issue than for versions
- tern (user-98b79 )
- Backup & restore
- postponed to next week
- simu question on the availability of the E// pnf simu availability in nexus AJAY SINGH
- answer sent by mai, changes done broke the built chain => after 40 days the snampshot images are removed, as the image was not released and not rebuilt, expiration date was reached => need to fix the build or to release an old version (pretty sure that jenkins history is also cleaned..so probably not possible)
- CSIT issue with python2.7/Python3 (Lasse Kaihlavirta Morgan Richomme )
- AoB
- Action points for next meeting
- AP1: Andreas Geissler check if doc of the official use cases are excplictely referencing the robot pod
- AP2: Morgan Richomme include Nokia pod in onap-integration web site
- AP3: all help reviewing the OOM patches to give Integration feedback
- Action point follow-up
- Admin
- Honolulu
- demo simulator Core NSSMF simulator (Zhang Min ) 20'
- demo CSIT of VNF/NFVO package management and LCM Gareth Roper , Andrew Lamb Byung-Woo Jun 10'
- nf-simulator (pnf-simu evolution) Krzysztof Kuzmicki 10'
- VID new UI Marcin Przybysz VID_MODERN_UI.pptx
- AoB
- Action point follow-up
- AP1: morgan_orange publish video and slideck on the how to
- done: Youtube video link, discussions in progress at LFN level to suggest how to publish such videos per community channel
- AP2: morgan_orange plan a new slot with praveen
- no feedback from PRAVEEN VELUGUBANTLA , need to replan a slot
- AP3: morgan_orange send mail to suggest repo, and if OK => create teh repo
- mail sent to Konrad Bańka , wait for green light before creating the new repo
- AP4: morgan_orange anser to the mail for pnf-simu evolution and plan a formal agreement from committer next week
- see next section
- AP5: natacha add the _v to manage the settings path in order to retrieve the pythonsdk logs
- done but side effect due to the fact that the settings in master are in /Src/onaptests/src/onaptests and not under /usr/lib/python3.8 in xtesting
- AP1: morgan_orange publish video and slideck on the how to
- Admin
- new ietf repo created, Bartek Grzybowski added JS linter, first upload in progress
- Guilin CI chain
- ~ OK http://testresults.opnfv.org/onap-integration/results/index-onap_daily_pod4_guilin-ONAP-oom.html
- Certificates CDS-UI expires in 3 days and VID in 29 days
- Error due to 500 from SDC
- Master chain
- pythonsdk tests broken during 2-3 days due to the introduction of onapsdk 7.4 (onapsdk custom exceptions were no more caught in onaptests)
- problem affecting all the gating chains
- Comments from Lukasz Rajewski on the slack channel => quid about the history, free account does not provide long retention period #action 8
- Honolulu
- Discussion on a recovery pipeline / resiliency tests (Marek Szwałkiewicz Morgan Richomme
- Epic created on Honolulu Resiliency and Backup and Restore test
- several tasks to perform tests
- Please do not hesitate to use the JIRA if you are performing such tests in your lab
- Epic created on Honolulu Resiliency and Backup and Restore test
- Discussion on CI/CD chain evolution (Morgan Richomme )honolulu_ci_evolution_08022021.pdf
- Krzysztof Kuzmicki indicates that dcae-mod shall be tested also through a new healthcheck test
- FWCL not possible to integrate in CI straightforward - preprovisionned resources needed, dependencies on installation (documented),
- Not sure if FW (macromode) will bring more than basic_VM + PNF and note sure Andreas Geissler will have resource to deal with it
- VCPE TOSCA (CLI) was not possible to run it on daily labs (running onwly on tester lab), improvements were planned in Guilin (seems to be done). #action 1
- #agreed No objection from the new categories and new tests for Honolulu branch
- for tern/scancode: watch out the versions of the OS, Debian9 is too old. #action 2
- Security
- versions: last update
- test created by Paweł Wieczorek
- waiver list introduced in Honolulu (patch merged)
- New GUI suggested more explicit that the json file: https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/02-11-2021_22-02/security/versions/versions/
- still lots of java 8 and python 2.7...
- very few projects seem to use the baseline images (that are up to date..)
- versions: last update
- Pythonsdk
- removal of the submodule?
- #agree remove the pnf submodule as soon as possible, be careful with the side effects on xtesting dockers / xtesting-onap ansible roles as we coudl move back from src/onaptests to /usr/lib/python3.8 path
- status on the wrapper (helm2/helm3)
- Illia Halych indicates that there is a client supporting helm2 and one supporting helm3 but not both
- #agree consider only helm3
- Add the statement in the README (as for the python min version 3.7 (same as pythonsdk)
- #action 3
- status on the tests basic_pnf, basic_clamp
- problem for basic_pnf seems to be fixed, patch merged on SDC
- see if the basic_pnf could be introduced in CI (test it first manually on daily weekly then add teh job xtesting-onap project that defines the CI chain) #action 4
- Improve reporting
- reporting may be misleading as it indictaes only the results of resource creation. We recently got an issue when deleting the resources => all the reporting is green even if there was an error during vfModule deletion. Michał Jagiełło submitted a patch to consider deletion in reporting #action 5
- Solution to limit issues on master gate (gating on pythonsdk?, versioning,..)
- pnf-simu > nf-simu: formal review
- #agreed
- demo to be planned next week #action6
- Discussion on a recovery pipeline / resiliency tests (Marek Szwałkiewicz Morgan Richomme
- AoB
- Marcin Przybysz indicates that a discussion is needed on VID. Vid new UI is needed for PNF macromode tests but old UI is needed for vFWCL..how do we manage that
- Michał Jagiełło tried to setup the new VID uI (VID 7.0.2) but has issue, Marcin Przybysz indicates that some flags have to be tuned #action 7
- actions for next meeting
- AP1: Morgan Richomme : contact vcpe tosca team to get their position in Honolulu (
)Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key TEST-239 - AP2: user-98b79 : get the list of projects using the baseline images
- AP3: Illia Halych amend https://gerrit.onap.org/r/c/testsuite/pythonsdk-tests/+/117192 for helm3 support
- AP4: Morgan Richomme test basic_pnf on Orange daily master with onaptests:master
- AP5: Michał Jagiełło Morgan Richomme test carefully before submitting until we got a gate to do it...
- AP6: Krzysztof Kuzmicki prepare a 10 minutes demo on pnf-simu to nf-simu (explaning the graph shared by mail)
- AP7: Marcin Przybysz plan a demo of VID new UI, how to launch it, how it is used in PNF macromode + see with VID how we can manage tests that are using legacy VID..
- AP8: Morgan Richomme contact Kenny Paul for the status of the slack chan
- AP1: Morgan Richomme : contact vcpe tosca team to get their position in Honolulu (
- Action point follow-up
- Admin
- Honolulu
- demo 5g-core-nf simu (PRAVEEN VELUGUBANTLA ) 20'
- demo ORAN A1-based Closed Loop (Konrad Bańka ) 20' A1 Policy Enforcement PoCv1.pptx
- demo stability test launcher (Natacha Chéreau )10'
- AoB
(18:02:16) collab-meetbot: Meeting ended Wed Feb 10 17:02:15 2021 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
(18:02:16) collab-meetbot: Minutes: http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_10_2_2021/onap-int-integration_weekly_meeting_10_2_2021.2021-02-10-16.44.html
(18:02:16) collab-meetbot: Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_10_2_2021/onap-int-integration_weekly_meeting_10_2_2021.2021-02-10-16.44.txt
(18:02:16) collab-meetbot: Log: http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_10_2_2021/onap-int-integration_weekly_meeting_10_2_2021.2021-02-10-16.44.log.html
- Action point follow-up
- Admin
- new repo follow-up
- Honolulu
- demo PNF (Michał Jagiełło )
- discussion on simulators
- simulator wrapper in pythonsdk-tests (Illia Halych)
- Preparation vEvent
- AoB
(18:57:32) collab-meetbot: Minutes: http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_27_01_2021/onap-int-integration_weekly_meeting_27_01_2021.2021-01-27-17.29.html
(18:57:32) collab-meetbot: Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_27_01_2021/onap-int-integration_weekly_meeting_27_01_2021.2021-01-27-17.29.txt
(18:57:32) collab-meetbot: Log: http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_27_01_2021/onap-int-integration_weekly_meeting_27_01_2021.2021-01-27-17.29.log.html
- Action point follow-up
- Admin
- new repo follow-up
- new Real time support channel: slack (already exists), publick rocket chat, mattermost, ....
- Honolulu
- demo PNF (Michał Jagiełło )
- discussion on simulators
- simulator wrapper in pythonsdk-tests (Illia Halych)
- CSIT redesign updates (Lasse Kaihlavirta)
- stability/resiliency testing (Natacha Chéreau )
- Update on Tern tests (user-98b79 )
- Preparation vEvent
- AoB
(17:40:10) collab-meetbot: Minutes: http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_20_01_2021/onap-int-integration_weekly_meeting_20_01_2021.2021-01-20-16.06.html
(17:40:10) collab-meetbot: Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_20_01_2021/onap-int-integration_weekly_meeting_20_01_2021.2021-01-20-16.06.txt
(17:40:10) collab-meetbot: Log: http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_20_01_2021/onap-int-integration_weekly_meeting_20_01_2021.2021-01-20-16.06.log.html
- Action point follow-up
- Honolulu
- demo CDS / how it is used in Bell and how to upstream (Jozsef Csongvai ) 10' +10' discussions
View file | ||||
---|---|---|---|---|
|
- demo tests DataFileCollector (Krzysztof Gajewski ) 10'
- Guilin Maintenance Release
- Preparation vEvent
- AoB
(16:26:02) collab-meetbot: Minutes: http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_13_1_2021/onap-int-integration_weekly_meeting_13_1_2021.2021-01-13-14.51.html
(16:26:03) collab-meetbot: Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_13_1_2021/onap-int-integration_weekly_meeting_13_1_2021.2021-01-13-14.51.txt
(16:26:03) collab-meetbot: Log: http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_13_1_2021/onap-int-integration_weekly_meeting_13_1_2021.2021-01-13-14.51.log.html
...