Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Michael started parallel full time ONAP related DevOps position - should discuss impact on logging/pomba project
    • Notice to onap: https://lists.onap.org/g/onap-discuss/topic/michael_reduced_availability/29918628?p=,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,0,29918628

      Team,

         I have taken on a full time DevOps kubernetes based role last week directly related to ONAP that may cause less focus on public in the short term and includes 20% travel.

         We can discuss this in the affected meetings.

         I have more than a couple pending mails to answer – sorry for the de-focus the last 2 weeks.

         I am still working out the details of working privately and publicly – as I was previously 100% public – the team I work with on DevOps is very open to the idea of continuing the LOG and CD work – as there is also opportunity for up-sourcing as both sides are ONAP focused and the role is in support of production ONAP deployment.

         So just a heads up


    • Any committer can run for the PTL role - it is a short 2 phase process - 3 days for who runs - on onap-discuss - then 3 days for committers to vote
  • Prep for M2 this thursday
  • Continue coding changes for spec
  • Answer pending questions/mails

...

  • Logging Deliverables for Functionality Freeze Milestone Checklist Template
  • TSC-25 ramping down - CD work
  • Continue coding changes for spec
  • Answer pending questions/mails
  • Review opentracing/zipkin 
    Jira
    serverONAP JIRA
    serverId425b2b0a-557c-3c0c-b515-579789cceedb
    keyLOG-104
  • Lorraine Welch System.out standardout - review in terms of not using a hardcoded file appender - in terms of syslogs
  • Discussion: DCAE logs
    • log formatting questions - the 4 types of logs per microservice - 
    • Discuss: Acumos single log discussion - MARKERs to replace log file name key - optional for ONAP for now - TODO verify our spec

Attendees 

Prudence Au

Lorraine Welch

Luke Parker

Items

  • Plan for 
    • current implementation
    • Future spec for el-alto for VNFs below
  • this week...
    • logging work
      • Dev environment back up - merging existing library - using vid-app-common as a template for usage of org.onap.portal.sdk
        • <epsdk.version>2.4.0</epsdk.version>
      • prepping for splitting repos - 1 per component - will need 8+
      • working with dmaap on charts and filebeat
      • pylog issues for vfc are transient LF issues - posting response (with multicloud)
      • release notes
      • scorecard for S3P
      • dcaegen2 work in 
        Jira
        serverONAP JIRA
        serverId425b2b0a-557c-3c0c-b515-579789cceedb
        keyDCAEGEN2-1166
         under https://gerrit.onap.org/r/#/c/77910/
      • questions on logging format onap-discuss including hv-ves I need to address https://lists.onap.org/g/onap-discuss/message/14997?p=,,,20,0,0,0::Created,,log,20,2,20,29162034
        • Discussion on VNF logs (CLAMP) @Sanjay - with Alok Gupta
        • vnf behaviors on top of vnf events
        • dmaap TCA like events - look into capturing these
        • ?add our own log tracing when VNFs react to events - another tracing EPIC we should look at
          • both for VES and non-VES format
          • There is a gap in tracing VNF behaviour - via 5G RAN - 
          • cloud infrastructure logs - vm, k8s and cloud service logs (beyond the vm level) - CNI cloud-native plugins example
          • need to think more about combining the logs - at the same time as we currently just capture them.
          • Provide log requirements to VNF onboarding team.
          • Spondon Dey - feeding in to CL, policy - scaling behaviour - onap to drive more
    • infrastructure work
      • Helm ownership - 
      • CI/CD - going with Orange MQ robot for oom merges
      • Perf and mostly crashloop avoidance 
      • Deploy changes for RHEL7.6
      • Deploy order work
      • ARM A1 testing of new containers from dockerhub on AWS
        • 80g/vm images - reducing footprint, standard alpine java image, ARM/i64 compat
      • Nodeports for dmaap
      • Datalake (now part of DCAE) does not yet affect us but it will - 
    • ONS April conference prep work
    • The rest of our backlog is still in progress - M2 is coming up on the 14th
  • last week....
  • get committed resources for the next 2 months M2 to RC0 - so we can state what is in and out of the Dublin release
    9 weeks to april 4th
    • M2 - functionality freeze - 21 Feb
      M4 - April 4th 
      I have taken the liberty of adding some names - feel free to add your availability or edit this section - we will paraphrase it in the M1 report - Logging Dublin M1 Release Planning
    • Michael O'Brien - 50% direct Logging work - really 40% dev/devops + 10% PTL/TSC/Project - the rest = related ONAP, CD, Doc, OOM, conference/customer,
      • Todo lowered?
    • Prudence Au - doing half of the PTL work, template, meets, reviews - especially POMBA with James MacNider on reviews - representing on most Thu POMBA meets
    • Avdhut Kholkar - thank you for all the commit reviews
    • Luke Parker - co-PTL and reference code
    • Sanjay - TODO: % of work on the project
  • Meeting at 1200 EST today on ARM docker images (affecting LOG images as we need to get the ARM layer into the image - wrap the dockerhub versions)
    Jira
    serverONAP JIRA
    serverId425b2b0a-557c-3c0c-b515-579789cceedb
    keyLOG-331
  • Stop using "latest" for any image - lock down the version tag for testing stability - see our use of busybox

    Jira
    serverONAP JIRA
    serverId425b2b0a-557c-3c0c-b515-579789cceedb
    keyLOG-949
  • Good news: We passed M1 last Thu
  • Dublin scope finalized for M1
    Release Planning#DublinReleaseCalendar
    Logging Dublin Scope
  • New work for dublin
    • Assist in 5G edge work via OOM/AWS work - meet is at 1100 EST Wed with ramki krishnan's team
    • plus metric capture via Prometheus -
      Jira
      serverONAP JIRA
      serverId425b2b0a-557c-3c0c-b515-579789cceedb
      keyLOG-911

      Jira
      serverONAP JIRA
      serverId425b2b0a-557c-3c0c-b515-579789cceedb
      keyLOG-707
  • Review/consolidate JIRAs
  • opentrace - will try to get in by april - an LF project 
  • priority list
    • infrastructure - filebeat sidecars (before DaemonSet refactor) - see Log Streaming Compliance and API
    • format - via library - portal/sdk - minimal retrofit for markers/mdcs - 
      Jira
      serverONAP JIRA
      serverId425b2b0a-557c-3c0c-b515-579789cceedb
      keyLOG-600
    • all s3p - security, perf (aai-log-3**) - scaling - run with 1 logstash
    • Logstash used to be a Daemonset  - however the filebeat needs to a daemonset - instead of each sidecar - 1 container per vm - get story
    • Additional tools - get POC for each - determine which goes to production level
      • prometheus - requires coordination with oom and multicloud
      • log checker - pending
      • opentracing - us - 
        Jira
        serverONAP JIRA
        serverId425b2b0a-557c-3c0c-b515-579789cceedb
        keyLOG-104
      • search guard - us

...