Agenda

Refactoring the patterns in logback.xml - (ONAP and Acumos) - plus implications for teams

Lorraine and I started aligning the patterns in logback.xml last thursday in the acumos meeting - however we all need to discuss this and the implications
https://gerrit.onap.org/r/#/c/62405/

currently

acumos - https://wiki.acumos.org/display/OAM/Log+Standards

onap - ONAP Application Logging Specification v1.2 (Casablanca)

 <property name="pattern" value="%nopexception${p_tim}|%X{EntryTimestamp}|%X{InvokeTimestamp}|%X{RequestId}|
%X{InvocationId}|%X{InstanceUUID}|
%X{ServiceInstanceId}|%thread|%X{ServiceName}|%X{PartnerName}|%X{StatusCode}|%X{ResponseCode}|%X{ResponseDesc}|%.-5level|
%X{Severity}|
%X{ServerIPAddress}|%X{ElapsedTime}|
%X{ServerFQDN}|%X{ClientIPAddress}|%X{VirtualServerName}|%X{ContextName}|%X{TargetEntity}|%X{TargetServiceName}|
%X{TargetElement}|%X{User}|
${p_log}|${p_mdc}
|${p_msg}%n" />


|2018-08-27T14:08:47.233Z|||1ff4ba60-4b23-4e81-a5fc-0b8055bbf1de||http-apr-8080-exec-4|/logging-demo/rest/health/health|||||
INFO||||dev.onap.info|10.42.0.1|||||||
org.onap.demo.logging.ApplicationService|
InstanceUUID=1ff4ba60-4b23-4e81-a5fc-0b8055bbf1de, RequestID=7ddbb889-0c6d-418e-b59a-df0450e57d76, 
ServiceName=/logging-demo/rest/health/health,
 InvocationID=e16e8b3e-01c6-4578-a3fc-d802ab31d91a, InvokeTimestamp=2018-08-27T14:08:47.233Z, 
PartnerName=, ClientIPAddress=10.42.0.1, ServerFQDN=dev.onap.info
|Running /health

onap and acumos teams are looking for a standard logback.xml - see https://gerrit.onap.org/r/#/c/63235/
- However we have the following considerations
- Moving MDC attributes to standard attributes (defined in logback.xml)
- (Yes pipe) pipe/tab delimited - portal and a lot of onap is still pipe (align with acumos where pipes are used)
  - plan for migration needs pipeline change
- implications for the pipeline yml in elasticsearch
- implications for our and other logging libraries in use - setting these field as mdcs or standard attributes
  - https://git.onap.org/logging-analytics/tree/reference/logging-slf4j/src/main/java/org/onap/logging/ref/slf4j/ONAPLogAdapter.java#n320
- (Yes for single) A single pattern for all 4 log files (debug, error, metrics, audit)
- Acumos reference https://wiki.acumos.org/display/OAM/Logging#Logging-Logback.xml
- !) Log levels are still as defined in the spec (INFO/TRACE/DEBUG/WARN/ERROR/FATAL) no fine/finer/finest
- What to do with application.log (it is the default for a lot of components - but is being migrated to audit.log)
  - plan: for code migration - ie portal/sdk application logs that would break if application appender is removed
    - move app to audit? yes
  - Do we make audit.log the default (info logs) - yes
  - Adjust msg to Message
migration story
default log level - still info (yes)
Note p_log (logger field (one of debug,audit etc) - not needed once markers is implemented - markers only available via Luke's slf4j lib currently
- for now hardcode the name in the 4 pattern strings
default debug state true/false (default should be false)
- need to test if changing a onap/values.yaml helm variable requires a bounce of the container (–set component.enabled=false/true)

single logback.xml for all of OOM - is this feasible (yes going ahead with it) - - lowers config map size issue

- parameterized values for logDirectory, debug, level
JIRA board - https://jira.onap.org/secure/RapidBoard.jspa?rapidView=143
review state of Logging Casablanca Scope
need to fix ciritical CLM issues (library versions) to remove instability flag on CLM job - -
- versions approved being fixed https://lists.onap.org/g/onap-discuss/topic/logging_m3_template_ready/24828468?p=,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,0,24828468
- some fixes (guava) are in oparent now -
pending commits (none 2 merged)
- https://gerrit.onap.org/r/#/q/status:open+project:+logging-analytics
- https://gerrit.onap.org/r/#/c/63235/
Collaborations
- Portal/sdk library usage (EELF based) -
- Acumos log standardization - single logback.xml https://jira.acumos.org/browse/ACUMOS-1584
- SDC logging issue (fixed?) - and https://gerrit.onap.org/r/#/c/58267/
Still need to incorporate templating
review changes in ONAP Application Logging Specification v1.2 (Casablanca)#StandardAttributes (standard and required) - expected as in
fix spec table for mdc to standard attributes changes ONAP Application Logging Specification v1.2 (Casablanca)
fix RI code log/portal for MDC changes
finish 4 logback.xml pattern examples for 4 logs
Start enumerating uses cases and verify MDC propagation flows inside and outside a microservice - distributed transactions inside/outside
- between microservices
- inside a microservice

Notes

oparent

clamp

logback.xml rework

Answer serverFQDN question from Dave - ONAP Application Logging Specification v1.2 (Casablanca)#Logback - testing on clustered system to see if dev.onap.info becomes dev0/dev1