Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migration of unmigrated content due to installation of a new plugin

Table of Contents

Meeting Info

When: Wednesdays  (weekly) 13:00-14:00 Irish Standard Time

Where: https://zoom.us/j/836561560?pwd=TTZNcFhXTWYxMmZ4SlgzcVZZQXluUT09

OrganizerToine Siebelink

Recurring Agenda Items

Software Status

Archived: Configuration Persistence Service Kohn Meeting Notes & Recordings

Minutes

DateAttendeesAgenda / Notes

 

Please continue to Configuration Persistence Service Montreal Meeting Notes & Recordings

 

  • Lukasz Rajewski Discuss use of cpsPath for get operations (instead of absolute xpath support only)
    • Toine Siebelink explain the core reasons for NOT combining these 
      • Architecture design to separate 'get' from 'query' functionality: Client should be aware what they use when
      • Although xPath and cpsPath can look very similar the cost and complexity of processing a cpsPath is order of magnitude higher
      • xpath cannot easily be distinguished from  valid cpsPath unless the model is parsed too which again would lead to additional cost to support both on the same interface. 
      • Operation that modify the data will only support xPath too. This is again an architecture decision that is based on best practice, scalability and transparency.  If the get operations would support cpsPath the same would be expected for the operations that modify data too and would violate the aforementioned principles.
  • Lukasz Rajewski  asked about CPS → CPS Temporal notifications , performance effects
    • Delta interesting but prefer to use dataspace/datatores in future for this instead of CPS Temporal
    • Support with disabling notifications dataspace, details to be forwarded to Arne Chrestin 
  • Ahila P write operations performance was not prioritized in London release instead the focus was on read operations and some queries.  In Montreal we wil shift toward optimizing write operations too. But the team is short on resources and have a large backlog. David Terwilliger  will try to get additional resourcing for the CPS team.  Ahila will create requirement Jira for optimizing write operations. 
  • Ahila P  Pagination wil be progressed by DT/Tech Mahindra (also need tracking JIra for R13/Montreal)
  • SoftwareStatus
    • London Branches have been created for CPS Repos, teams can start working on Montreal (master branch)  
    • CPS-1582 and CPS-1580 need to be dropped back to London

 

 

 

  • rajesh kumar present study on pagination (https://wiki.onap.org/display/DW/CPS-1567%3A+Investigate+Implementation+of+pagination+for+Query+AP
    • Pagination of fragments not feasible as it stil needs to be recombined into a complete tree
    • Pagination by anchor seems feasible but wil only be usable for bulk operations
  • SoftwareStatus
      • Jira
        serverONAP Jira
        serverId425b2b0a-557c-3c0c-b515-579789cceedb
        keyCPS-1568
        Toine Siebelink discussed this with Tony Finnerty  and agreed CPS should be abel to handle DB down and have a retry mechanism too. So apart from ETH making deployment changes CPS team should also provide a build in robustness on DB connection...
      • Halil Cakal and Priyank Maheshwari are build new release for CPS, will try to incorporate this bugfix too:
        Jira
        serverONAP Jira
        serverId425b2b0a-557c-3c0c-b515-579789cceedb
        keyCPS-1563
  • Milestone Status
    • CPS TEam completed all M3 tasks. Code free might start tomorrow, decision will be taken during TSC meeting!
  • A.O.B
    • Raviteja Karumuri propose small improvement regarding docker log handling after CSIT test
      • Logs truncated because of print-buffer limitation
      • just affected local run
      • increase buffer size as workaround; Raviteja Karumuri will document

 

  • SoftwareStatus
  • Introducing Anirban Bhattacharjee Product Owner for CPS
  • CI (docs) Build issue status
    • Ongoing CI Team aware troubleshooting ongoing
    • CPS Team can IGNORE this (new) job for now as long as all other jobs pass any Comitter can delete the -1 from CI Build and set Verify to +1 instead if need to merge pending commits

 

  • SoftwareStatus
    • Jira
      serverONAP Jira
      serverId425b2b0a-557c-3c0c-b515-579789cceedb
      keyCPS-1497
      Discussed model proposal with team and recorded  in Jira description
  • Luke Gleeson demo the forwarding of subscription create events to DMI Plugins
    Jira
    serverONAP Jira
    columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
    columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
    serverId425b2b0a-557c-3c0c-b515-579789cceedb
    keyCPS-1431
    • Happy flow looks good (smile)
    • Kafka configured to create non-existing topic on the fly so could not demo required behavior as per AC
    • Need update to handle non-existing cm-handles (possible  separate commit)
    • user story wil be demoed again when all negative scenarios are covered too

 

 

  • Arpit Singh will present study: Support CPS Core Patch operation for multiple data nodes
    1. Two top element trees, patch using xpath /, error reported. Although support would be 'nice' there is no urgency. Need to check with DT stakeholder re. priority
    2. Two top elements but no children just leafs, works partially (only one tree would be updated,a s expected).  Expecting such simple model is unlikely and fixing 1. above should fix this too 
    3. List example problem is unclear, needs to be updated. It seems unrelated to multiple root scenario and possible legacy issue(bug). Arpit Singh will separate this into a bug on its own. 
  • SoftwareStatus
  • Rudrangi Anupriya presented on some early suggestion for supporting 'contains' operations, see CPS-1272 Support for contains in cps-path
    • Anupryia will expand example in a table to make it more clear. 
    • Before any further investigation Postgress JsonB query support for this should be investigated

 

 

  • SoftwareStatus
    • Jira
      serverONAP Jira
      serverId425b2b0a-557c-3c0c-b515-579789cceedb
      keyCPS-1433
      model change required. Lee Anjella Macabuhay has tried to organize meeting with Ahilla and Robert Vagra but Robert is hard to get. In any case Andras has been able to use the given model with the recommended fix in the latest version of CPS to do further performance testing as part of
      Jira
      serverONAP Jira
      serverId425b2b0a-557c-3c0c-b515-579789cceedb
      keyCPS-1475
    • Toine Siebelink demonstrated in a bit mode detail the instrumentation introduced by
      Jira
      serverONAP Jira
      serverId425b2b0a-557c-3c0c-b515-579789cceedb
      keyCPS-1457
      sample results can be seen here: NCMP De-Register API Performance#Instrumentationsample
    • Jira
      serverONAP Jira
      serverId425b2b0a-557c-3c0c-b515-579789cceedb
      keyCPS-1477
      Arne Chrestin has provide small fix and CPS team wil review and merge later today if possible.
  • Ahila P reported some progress on performance testing with some possible changes. Toine Siebelink will call meeting early next week to coordinate her and CPS team's efforts

@  

  • Rajesh kumar  demoed
    Jira
    serverONAP Jira
    serverId425b2b0a-557c-3c0c-b515-579789cceedb
    keyCPS-1381
    • Depth functionality works well, some updates required to OpenAPI documentation: spelling, use " quotes for text values, explain 0 and positive values, only mention -1 as a note. 
  • Ahila Performance Discussion
  • SoftwareStatus
    • Jira
      serverONAP Jira
      serverId425b2b0a-557c-3c0c-b515-579789cceedb
      keyCPS-1433
      model change required. Lee Anjella Macabuhay will call meeting with to discuss and clarify with parties involved
    • DMI_Plugin build blocked because of relative paths used by init container. Lee Anjella Macabuhay  wil create a fix for this


 

Please note new schedule (moved to Wednesday), see onap-meetings calendar

  • SoftwareStatus
    • Sourabh Sourabh is fixing self reported bug
      Jira
      serverONAP Jira
      serverId425b2b0a-557c-3c0c-b515-579789cceedb
      keyCPS-1463
  • Backlog Grooming (only develop team present, took the opportunity to groom some upcoming stories)
    • Jira
      serverONAP Jira
      columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
      columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
      serverId425b2b0a-557c-3c0c-b515-579789cceedb
      keyCPS-1443
       handed over to Andras Zoltan Kiss from Levente Csanyi 
    • Jira
      serverONAP Jira
      columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
      columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
      serverId425b2b0a-557c-3c0c-b515-579789cceedb
      keyCPS-1394
      assigned to Sourabh Sourabh 
  • Sourabh Sourabh  and Daniel Hanrahan  presented delete Performance, preliminary results
    • See NCMP De-Register API Performance 
      • Native Query 10-20x improvement but not meeting requirement
      • Delete performance not linear: more nodes slower
      • Tested up to about 2,300 cm handles which is the highest feasible given the current performance
    • Delete-by-batch is in progress and it is hope that will give the required boost. Hopefully can be demonstrated/reported next week

 

Meeting cancelled, no attendance bar development team 


 

  • SoftwareStatus
    • New bug 
      Jira
      serverONAP Jira
      serverId425b2b0a-557c-3c0c-b515-579789cceedb
      keyCPS-1446
       Toine will triage 
    • Lee Anjella Macabuhay  still confusion about bug
      Jira
      serverONAP Jira
      serverId425b2b0a-557c-3c0c-b515-579789cceedb
      keyCPS-1433
      Ahila P believes it is a ODL parser issue and that the model used to parse with teh previous version of ODL. Lee-Anjelle will double check and then organise a follow up meeting
  • Sourabh Sourabh demo
    Jira
    serverONAP Jira
    serverId425b2b0a-557c-3c0c-b515-579789cceedb
    keyCPS-1443
    • Currently used different 3pp for add 'timer' Might change to promethuse.io client instead and use 'Histogram'
    • User story  
      Jira
      serverONAP Jira
      serverId425b2b0a-557c-3c0c-b515-579789cceedb
      keyCPS-1445
      will introduce some critical times and counters
  • Arpit Singh demo
    Jira
    serverONAP Jira
    serverId425b2b0a-557c-3c0c-b515-579789cceedb
    keyCPS-341
    • looks good
    • get in progress
    • more impacts are planned
    • add test plan for unforseen impacts

 

  • Milestone update: Release Planning: London
  • SoftwareStatus
    • CSP-1409 to be confirmed. 
    • Meeting next week re. performance
  • Discussion on payload for get node APIs Arpit Singh 
  • Ahila P presented more performance test findings:
    • see CPS-Core Read & Write (large load) Test Results 
    • Increasing Heap Memory from 30 GB to 100MB dus allow to achieve more nodes to be stores (from 500 to 1000)
      • Threaddumps show many Kafka (?!) and Hazelcast thread waiting
    • Increasing threads to 60, decreased performance.
    • Suggestions for further testing:
      • Disable CPS-Temporal notifications by configuring dataspace filter:
        config.additional.notification.data-updated.filters.enabled-dataspaces:"NON-EXISTING-DATASPACE"
      • 5 parallel client threads instead of 60

 

  • rajesh kumar to present/discuss CPS-1377: Investigate CPS Path search over multiple anchors under one dataspace
    • Proposal look good but Toine would like to see
      • Comparison with existing anchor based query endpoint
      • Response structure proposal using high level structure model beside a details (separate example)
      • 3rd and prefered option seems not correct missing key value pairs and array definition to hold anchor details
  • SoftwareStatus
    • Ahila P will call meeting to discuss
      Jira
      serverONAP Jira
      serverId425b2b0a-557c-3c0c-b515-579789cceedb
      keyCPS-1409
    • Jira
      serverONAP Jira
      columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
      columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
      serverId425b2b0a-557c-3c0c-b515-579789cceedb
      keyCPS-1408
      Toine Siebelink  will contact SecCom (Tony) to be able to close
  • Ahila P discuss performance issues
    • goal: 50,000 x 1920 Fragments!
    • max about 500 nodes (1 request with 1node at the time (no delays at the moment)
    • crashes during read & write operations
    • 1 instance using 10CPUs  (max is 12) 16Gb memory
    • Ahila P will schedule meeting next week to discuss in more detail and how to proceed
  • N.K. Shankaranarayanan  discuss VES Event handling and updates for primary and secondary  model. 
    • ONAP DMI PLugin can handle Primary updates listen to VES events
    • But NCMP nor DMI can decide make secondary model changes. At best send event about primary model changes
  • Future meetings this Meeting
    • NO meeting on Friday Dec 23rd
    • Toine can no longer host this meeting on Fridays and proposed in 2023 this meeting will move to Thursdays same time, Toine wil send e-mail to confirm

 

  • SoftwareStatus
    • Lee Anjella Macabuhay  ODL 8 upgraded completed. ODL 9 requires Java 17 (are on 11) Need to investigate...
    • RTDv3 CI Jobs are failing this is a know issue . CSIT (in CI) should be OK. If for a given commit only the RTDv jobs fails and there are no RTD changes that commit can be approved (Toine Siebelink can set the verify flag  +1 if needed). Toint to check teh status of the related fix with LNF support: Matthew Watkins 
    • Levente Csanyi reported issues with the performance tests in Nordix CI. It might just be the time limits are to close and teh variation is too large. Levent will check
  • rajesh kumar updates on
    • API Version handling proposal  OpenApi / Swagger UI
      • apiVersion for any method that support both versions

      • hardcode v1 or v2 when needed 

    • Output imitation using 'depth' parameter
      • agreed on v2 version of method (document affected methods) with just 1 parameter: 'descendants' (type string). Suggested acceptable values 'true',' false', 'all', 'nono' (case insensitive)  number '0' and any positive integer value


 

  • Status External Commits: https://gerrit.onap.org/r/q/projects:cps+status:open
  • London Milestone Status
    • M1 Due CPS Requirements (jira) need to exist by then (not committed)
  • Update on OpenApi multiple version support Luke Gleeson 
    • Still not working reached out to community for help
  • N.K. Shankaranarayanan  discuss state of CPS and revisit CM Notification Handling (by CPS?)
    • RAN CM Changes VES Message (Virtual-function Event Stream) → VES Collector, Ves Processor ? VES Notification has Yang payload
    • Status NCMP: Data can be cached, Performance, Event Consumption on Backlog (via DMI)
    • RAN VES Event needs to have some id that can be translated to a CM Handle ID
    • ONAP DMI PLugin would take role of CM-VES Parser (high Volume!)
    • TBDMT Not needed if Yang daat in NCMP uses same model as device
    • Freshness of data is relevant too, Tony Finnerty and Lukasz Rajewski like to explore this further
    • Shankar will write requirement on ONAP DMI Plugin

 

  • Discuss PreFix proposal Ahila P 
    Postponed to dedicated meeting next week
  • Discuss API Versioning issues/possible strategies
    • Not quite ready to discuss API version strategy
    • Re. Open API Swagger UI rendering issue following approaches wil be tried and reported on next week
      • Small completely new Open API 3.0 yaml file test with Swagger UI testing one unchanged URL and one URI which has 2 versions
      • Spit v1 and v2 API into 2 files with some duplication
  • Michał Jagiełło will demonstrate
    Jira
    serverONAP Jira
    serverId425b2b0a-557c-3c0c-b515-579789cceedb
    keyCPS-1257
    • Create use-case using XML
    • Insert use-case XML (add to list)
    • Validation of model demonstrated by changing top level node name
    • Some concern about ':'  in front of shops, prefix blank? Michał Jagiełło wil check
    • What about read, put? Not really needed
  • SoftwareStatus

 

  • SoftwareStatus
  • Jira
    serverONAP Jira
    columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
    columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
    serverId425b2b0a-557c-3c0c-b515-579789cceedb
    keyCPS-1002
    Luke Gleeson  to demo 
    • Luke Gleeson  presented the demo and there was no question raised.

 

  • SoftwareStatus
  • CI Build Issue Update
    • Jira
      serverLinux Foundation Jira
      serverId786ecce4-c7f8-3725-b80d-ceab920b9b14
      keyRELENG-4523
    • Toine has applied workaround that makes Robot CSIT tets working again, see https://gerrit.onap.org/r/q/topic:%22RELENG-4523%22 but his has to be undone when RELENG tiket is properly resolved. But the team is unblocked now
  • A.O.B. 
    • rajesh kumar looking for update on requirements  from Ahila P : Long term yes but nothing concrete, but would like to join efforts. Consider just using Depth-parameter for now as already internally supported buy CPS service. Rajesh wil start investigating and create ticket.
    • Update on Prefix handling: Notes in CPS-1380. Ahila P  will move analysis to CPS-1301: Insert Module Prefixes wiki instead. Arne Chrestin can help to some level and analysis. CSP To provide.  Toine Siebelink will review 

 

 

  • SoftwareStatus
  • Progress Update CPS-1301: Insert Module Prefixes
  • Project Milestones
    • Kohn RC passed (big grin)
    • Release Planning: London No M1 tasks yet,. But please do start adding CPS requirements in Jira to help planning for London
  • Michał Jagiełło updates 
    • XML Issue: data-wrapper required by XML. Will try to Poc Wrap/Unwrap with data as needed, communicate with ODL community about ODL support to automatically wrap it as part of parsing
    • Jakarta fix, build issue Luke Gleeson ssuspec CSIT depend on a now deleted image. Will contact Michal next week.

 

  • Reviewed CPS-1301: Insert Module Prefixes
    • agreed on issue #3 insert prefixes only when module changes
    • agreed on issue #4 always insert prefix for top level node in output
    • agreed on issue #5 insert 'prefix' attribute
    • new issue #9: is prefix mandatory? Ahila P to check
    • Toine Siebelink  to call follow up meeting (called for )
  • SoftwareStatus
  • A.O.B.
    • Ahila P asked for plans re. 'Pagination of large output'. Toine Siebelink explained there are no plans and expressed concerns because of the nature of Graph Data. Support of 'depth' parameter already available in Java API. Support could easily be extended to REST API. Tony Finnerty mentioned that Netflix might have solution in the open source domain. Michał Jagiełło suggested to check with A&AI since they have graph data and some form of pagenination implemented

  

  • Approaches to
    Jira
    serverONAP Jira
    serverId425b2b0a-557c-3c0c-b515-579789cceedb
    keyCPS-1301
    • Ahila P explained their requirements for prefixes and showed some preliminary code changes. Also indicated they need a 'short-term' fix before a a ore costly 'long term' solution
    • Agreed the following
      • Sourabh Sourabh will just fix the related performance issue 
        Jira
        serverONAP Jira
        serverId425b2b0a-557c-3c0c-b515-579789cceedb
        keyCPS-1326
         without changing any of the functionality issue
      • Jira
        serverONAP Jira
        serverId425b2b0a-557c-3c0c-b515-579789cceedb
        keyCPS-1344
        is created to ensure the first prefix is correct
      • Long term solution needs proper DB backed implementation to allow for quick acces to module information from any fragment
      • quite a few more details were discussed and issues raised, Toine Siebelink will create an Proposal Page with a list of 'open issues': 
        CPS-1301: Insert Module Prefixes
      • Toine Siebelink will call meeting with interested parties next week to discuss the issues, ensure agreement in communities and see how best to proceed implement a short term and a long term solution
  • Kohn Milestone Status (RC)
    • No immediate action required all task assigned or already done
  • SoftwareStatus
    • Andras Zoltan Kiss discovered  new bug for OOm chart issue:
      Jira
      serverONAP Jira
      serverId425b2b0a-557c-3c0c-b515-579789cceedb
      keyCPS-1343

Recordings

Date

Zoom Recording