...
- Michael started parallel full time ONAP related DevOps position - should discuss impact on logging/pomba project
Notice to onap: https://lists.onap.org/g/onap-discuss/topic/michael_reduced_availability/29918628?p=,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,0,29918628
Team,
I have taken on a full time DevOps kubernetes based role last week directly related to ONAP that may cause less focus on public in the short term and includes 20% travel.
We can discuss this in the affected meetings.
I have more than a couple pending mails to answer – sorry for the de-focus the last 2 weeks.
I am still working out the details of working privately and publicly – as I was previously 100% public – the team I work with on DevOps is very open to the idea of continuing the LOG and CD work – as there is also opportunity for up-sourcing as both sides are ONAP focused and the role is in support of production ONAP deployment.
So just a heads up
- Any committer can run for the PTL role - it is a short 2 phase process - 3 days for who runs - on onap-discuss - then 3 days for committers to vote
- Prep for M2 this thursday
- Continue coding changes for spec
- Answer pending questions/mails
...
- - Logging Deliverables for Functionality Freeze Milestone Checklist Template
- TSC-25 ramping down - CD work
- Continue coding changes for spec
- Answer pending questions/mails
- Review opentracing/zipkin
Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key LOG-104 - Lorraine Welch System.out standardout - review in terms of not using a hardcoded file appender - in terms of syslogs
- Discussion: DCAE logs
- log formatting questions - the 4 types of logs per microservice -
- Discuss: Acumos single log discussion - MARKERs to replace log file name key - optional for ONAP for now - TODO verify our spec
Attendees
Items
- Plan for
- current implementation
- Future spec for el-alto for VNFs below
- this week...
- logging work
- Dev environment back up - merging existing library - using vid-app-common as a template for usage of org.onap.portal.sdk
<epsdk.version>2.4.0</epsdk.version>
- prepping for splitting repos - 1 per component - will need 8+
- working with dmaap on charts and filebeat
- pylog issues for vfc are transient LF issues - posting response (with multicloud)
- release notes
- scorecard for S3P
- dcaegen2 work in
under https://gerrit.onap.org/r/#/c/77910/Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key DCAEGEN2-1166 - questions on logging format onap-discuss including hv-ves I need to address https://lists.onap.org/g/onap-discuss/message/14997?p=,,,20,0,0,0::Created,,log,20,2,20,29162034
- Discussion on VNF logs (CLAMP) @Sanjay - with Alok Gupta
- vnf behaviors on top of vnf events
- dmaap TCA like events - look into capturing these
- ?add our own log tracing when VNFs react to events - another tracing EPIC we should look at
- both for VES and non-VES format
- There is a gap in tracing VNF behaviour - via 5G RAN -
- cloud infrastructure logs - vm, k8s and cloud service logs (beyond the vm level) - CNI cloud-native plugins example
- need to think more about combining the logs - at the same time as we currently just capture them.
- Provide log requirements to VNF onboarding team.
- Spondon Dey - feeding in to CL, policy - scaling behaviour - onap to drive more
- Dev environment back up - merging existing library - using vid-app-common as a template for usage of org.onap.portal.sdk
- infrastructure work
- Helm ownership -
- CI/CD - going with Orange MQ robot for oom merges
- Perf and mostly crashloop avoidance
- Deploy changes for RHEL7.6
- Deploy order work
- ARM A1 testing of new containers from dockerhub on AWS
- 80g/vm images - reducing footprint, standard alpine java image, ARM/i64 compat
- Nodeports for dmaap
- Datalake (now part of DCAE) does not yet affect us but it will -
- ONS April conference prep work
- The rest of our backlog is still in progress - M2 is coming up on the 14th
- logging work
- last week....
- get committed resources for the next 2 months M2 to RC0 - so we can state what is in and out of the Dublin release
9 weeks to april 4th- M2 - functionality freeze - 21 Feb
M4 - April 4th
I have taken the liberty of adding some names - feel free to add your availability or edit this section - we will paraphrase it in the M1 report - Logging Dublin M1 Release Planning - Michael O'Brien - 50% direct Logging work - really 40% dev/devops + 10% PTL/TSC/Project - the rest = related ONAP, CD, Doc, OOM, conference/customer,
- Todo lowered?
- Prudence Au - doing half of the PTL work, template, meets, reviews - especially POMBA with James MacNider on reviews - representing on most Thu POMBA meets
- Avdhut Kholkar - thank you for all the commit reviews
- Luke Parker - co-PTL and reference code
- Sanjay - TODO: % of work on the project
- M2 - functionality freeze - 21 Feb
- Meeting at 1200 EST today on ARM docker images (affecting LOG images as we need to get the ARM layer into the image - wrap the dockerhub versions)
Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key LOG-331 - Stop using "latest" for any image - lock down the version tag for testing stability - see our use of busybox
Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key LOG-949 - Good news: We passed M1 last Thu
- Dublin scope finalized for M1
Release Planning#DublinReleaseCalendar
Logging Dublin Scope - New work for dublin
- Assist in 5G edge work via OOM/AWS work - meet is at 1100 EST Wed with ramki krishnan's team
- plus metric capture via Prometheus -
Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key LOG-911 Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key LOG-707
- Review/consolidate JIRAs
- opentrace - will try to get in by april - an LF project
- priority list
- infrastructure - filebeat sidecars (before DaemonSet refactor) - see Log Streaming Compliance and API
- format - via library - portal/sdk - minimal retrofit for markers/mdcs -
Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key LOG-600 - all s3p - security, perf (aai-log-3**) - scaling - run with 1 logstash
- Logstash used to be a Daemonset - however the filebeat needs to a daemonset - instead of each sidecar - 1 container per vm - get story
- Additional tools - get POC for each - determine which goes to production level
- prometheus - requires coordination with oom and multicloud
- log checker - pending
- opentracing - us -
Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key LOG-104 - search guard - us
...