WIP
This page is the design and reference site for the Logging RI. The following topics are being provided to the rest of the ONAP community.
- How to use the EELF Logging Library
- How to adhere to the logging guidelines
- How to implement the filebeat sidecar container to push logs to the ELK stack pod
- How to use the EELF AOP wrapping library
- How to ensure requestID tracking during distributed transactions
Questions on onap-discuss from the teams
https://lists.onap.org/pipermail/onap-discuss/2018-January/007415.html
LOG-118 - Getting issue details... STATUS
Purpose
The goals of the logging project are the following
provide for standardized logs across all ONAP components
provide ELK framework for log capture, indexing and presentation/search
provide framework for requestID tracking of distributed transactions (correlation ID's in p.162 of Building Microservices 2015 by Sam Newman) via HTTP ETags
Quickstart
WIP (as of 20180121) - will notify when complete
Goto your IDE workspace
Clone the logging repo
https://gerrit.onap.org/r/#/admin/projects/logging-analytics
https://git.onap.org/logging-analytics/tree/
goto the reference folder and load the pom.xml project
obrienbiometrics:wse_sts392b michaelobrien$ git clone ssh://michaelobrien@gerrit.onap.org:29418/logging-analytics Cloning into 'logging-analytics'... Warning: Permanently added '[gerrit.onap.org]:29418,[198.145.29.92]:29418' (RSA) to the list of known hosts. remote: Total 374 (delta 0), reused 374 (delta 0) Receiving objects: 100% (374/374), 1001.91 KiB | 1.22 MiB/s, done. Resolving deltas: 100% (120/120), done. obrienbiometrics:wse_sts392b michaelobrien$ cd logging-analytics/reference/ obrienbiometrics:reference michaelobrien$ ls\ logging-demo logging-library logging-mock-service pom.xml # build the project to verify obrienbiometrics:reference michaelobrien$ mvn clean install -U -DskipTests=true -Dmaven.test.skip=true -Dmaven.javadoc.skip=true -Dadditionalparam=-Xdoclint:none [INFO] logging-reference .................................. SUCCESS [ 3.069 s] [INFO] logging-library .................................... SUCCESS [ 1.313 s] [INFO] logging-mock-service ............................... SUCCESS [ 0.395 s] [INFO] logging-demo ....................................... SUCCESS [ 2.329 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 9.232 s [INFO] Finished at: 2018-01-21T09:29:15-05:00
Audits
Existing EELF implementations
Please use the latest 1.0.0 from https://github.com/att/EELF - nexus artifacts pending
| app | project | repo | version |
|---|---|---|---|
| aai | aai-common | 1.0.0 | |
| aai | aai-core | 1.0.0 | |
| aai | aai-service (deprecated) | 0.0.1 (not in use) | |
| aai | aai-traversal | 1.0.0 | |
| aai | ajsc-aai | 1.0.0 | |
| aai | eelf-logging | logging-service | 0.0.1 (not in use) |
| appc | (pax.logging = wrapper on log4j, slf4j, commons-logging 1.2 and logback 1.1.1) needs this RI - provide a demo to the appc team | 0.0.1 (upgrade?) | |
| ccsdk | nbapi | ccsdk/platform/nbapi | 1.0.0 |
| dmaap | buscontroller | dmaap/buscontroller | 0.0.1 |
| ui | dcae_dmaap_webapp | dmaapbc | 1.0.0 |
| ui | dmaap-bc-app-common | 1.0.0 | |
| ui | dmaap-bc-app-parent | 1.0.0 | |
| policy | common-modules/ | common | 1.0.0 |
| policy | drools-pdp | 1.0.0 | |
| portal | ecompportal-be-common | portal | <eelf.version>1.0.0 |
| policy | feature-eelf | drools-pdp | 1.0.0 |
| policy | PolicyEngineSuite/ONAP-logging | common | 0.0.1 (not in use) |
| policy | ONAP-SDK-APP | engine | 0.0.1 (not in use) |
| sdc | ? | ||
| so | common | so | 0.0.1 (upgrade ?) |
| vid | epsdk-app-onap | vid | 1.0.0 |
| vid | vid-app-common | vid | 1.0.0 |
Projects with a filebeat container as of 20170114
ubuntu@ip-172-31-49-200:~$ kubectl get pods --all-namespaces | grep 2/2 onap-aai aai-resources-1039856271-3ph1d 2/2 Running 0 37m onap-aai aai-traversal-143081090-f4cls 2/2 Running 0 37m onap-aai model-loader-service-4144225433-0jr21 2/2 Running 0 37m onap-aai search-data-service-3842430948-53j1w 2/2 Running 0 37m onap-aai sparky-be-4222608366-966p4 2/2 Running 0 37m onap-appc appc-1828810488-mhsf7 2/2 Running 0 37m onap-mso mso-736294529-c2p8v 2/2 Running 0 37m onap-policy drools-2600956298-dp7bq 2/2 Running 0 37m onap-policy pap-466625067-n23bp 2/2 Running 0 37m onap-policy pdp-2354817903-kmxv5 2/2 Running 0 37m onap-portal portalapps-1783099045-79lcq 2/2 Running 0 37m onap-sdc sdc-be-2336519847-p6p5l 2/2 Running 0 37m onap-sdc sdc-fe-2862673798-xcpnk 2/2 Running 0 37m onap-sdnc sdnc-0 2/2 Running 0 37m onap-sdnc sdnc-dbhost-0 2/2 Running 0 37m onap-vid vid-server-3026751708-rpmq4 2/2 Running 0 37m
Current Logging Libraries
Current Framework usage impacting Logging
Logging in VNFs
Deploying the RI
WIP
Kubernetes deployment of the RI
Design Issues
DI 1: 20171231: Initial Maven Project Structure
https://gerrit.onap.org/r/#/admin/projects/logging-analytics
git clone ssh://michaelobrien@gerrit.onap.org:29418/logging-analytics
see
obrienbiometrics:logging-analytics michaelobrien$ git status On branch master Your branch is up-to-date with 'origin/master'. Changes to be committed: (use "git reset HEAD <file>..." to unstage) new file: reference/logging-demo/.gitignore new file: reference/logging-demo/pom.xml new file: reference/logging-demo/src/main/java/org/onap/logging/LoggingReference.java new file: reference/logging-library/.gitignore new file: reference/logging-library/pom.xml new file: reference/logging-library/src/main/java/org/onap/logging/LoggingReference.java new file: reference/pom.xml Untracked files: (use "git add <file>..." to include in what will be committed) reference/.project reference/.settings/ reference/logging-demo/.checkstyle reference/logging-demo/.classpath reference/logging-demo/.project reference/logging-demo/.settings/ reference/logging-library/.checkstyle reference/logging-library/.classpath reference/logging-library/.project reference/logging-library/.settings/ reference/target/ obrienbiometrics:logging-analytics michaelobrien$ git commit -m "Initial logging lib/demo proj struct" [master 9e42247] Initial logging lib/demo proj struct 7 files changed, 306 insertions(+) create mode 100644 reference/logging-demo/.gitignore create mode 100644 reference/logging-demo/pom.xml create mode 100644 reference/logging-demo/src/main/java/org/onap/logging/LoggingReference.java create mode 100644 reference/logging-library/.gitignore create mode 100644 reference/logging-library/pom.xml create mode 100644 reference/logging-library/src/main/java/org/onap/logging/LoggingReference.java create mode 100644 reference/pom.xml obrienbiometrics:logging-analytics michaelobrien$ git review -s obrienbiometrics:logging-analytics michaelobrien$ git commit -s --amend [master c11cd50] Initial logging lib/demo proj struct Date: Sun Dec 31 13:30:04 2017 -0500 7 files changed, 306 insertions(+) create mode 100644 reference/logging-demo/.gitignore create mode 100644 reference/logging-demo/pom.xml create mode 100644 reference/logging-demo/src/main/java/org/onap/logging/LoggingReference.java create mode 100644 reference/logging-library/.gitignore create mode 100644 reference/logging-library/pom.xml create mode 100644 reference/logging-library/src/main/java/org/onap/logging/LoggingReference.java create mode 100644 reference/pom.xml obrienbiometrics:logging-analytics michaelobrien$ git review Warning: Permanently added '[gerrit.onap.org]:29418,[198.145.29.92]:29418' (RSA) to the list of known hosts. remote: Processing changes: new: 1, refs: 1, done remote: remote: New Changes: remote: https://gerrit.onap.org/r/27195 Initial logging lib/demo proj struct remote: To ssh://gerrit.onap.org:29418/logging-analytics * [new branch] HEAD -> refs/publish/master
For eclipse you will need to make sure the spring nature in the .classpath is set correctly - in order for maven build to also work in eclipse for war projects - this will show up in the project build path
<classpathentry exported="true" kind="con" path="org.eclipse.m2e.MAVEN2_CLASSPATH_CONTAINER"> <attributes> <attribute name="maven.pomderived" value="true"/> <attribute name="org.eclipse.jst.component.nondependency" value=""/> <attribute name="org.eclipse.jst.component.dependency" value="/WEB-INF/lib"/> </attributes> </classpathentry>
DI 2: 20171231: Audit: EELF implementors
DI 3: 20171231: Audit: Mixed version of EELF dependency
LOG-110 - Getting issue details... STATUS
Figuring out the reason for the different dependencies
aai-service, ajsc-aai, common-modules/common-logging
<dependency>
<groupId>com.att.eelf</groupId>
<artifactId>eelf-core</artifactId>
<version>0.0.1</version>
</dependency>
aai-common, eelf-logging, ONAP-logging, ONAP-SDK-APP
<dependency>
<groupId>com.att.eelf</groupId>
<artifactId>eelf-core</artifactId>
<version>1.0.0</version>
</dependency>
DI 4: 20170104: ETag propagation
Passing UUID based requestID's via rest calls
https://en.wikipedia.org/wiki/HTTP_ETag
DI 5: 20170120: Docker image design
PV for logback.xml,
container for mock-log-nbi
container for mock-log-proc (the scalable TX substrate)
DI 6: 20170120: Docker wrapper of Logging Library
LOG-136 - Getting issue details... STATUS
Implementation
Testing
http://127.0.0.1:8080/logging-demo/rest/read/test
Links
EELF
git clone https://github.com/att/EELF.git
https://tomcat.apache.org/download-80.cgi
https://spring.io/tools/sts/all
https://www.sitepoint.com/inside-java-9-part-ii/#performanceimprovements