Table of Contents |
---|
WIP
This page is the design and reference site for the Logging RI. The following topics are being provided to the rest of the ONAP community.
- How to use the EELF Logging Library
- How to adhere to the logging guidelines
- How to implement the filebeat sidecar container to push logs to the ELK stack pod
- How to use the EELF AOP wrapping library
- How to ensure requestID tracking during distributed transactions
Questions on onap-discuss from the teams
https://lists.onap.org/pipermail/onap-discuss/2018-January/007415.html
Jira | ||||||
---|---|---|---|---|---|---|
|
Purpose
The goals of the logging project are the following
provide for standardized logs across all ONAP components
provide ELK framework for log capture, indexing and presentation/search
provide framework for requestID tracking of distributed transactions (correlation ID's in p.162 of Building Microservices 2015 by Sam Newman) via HTTP ETags
Architecture
Deployment Diagram
Gliffy Diagram | ||||
---|---|---|---|---|
|
Quickstart
WIP (as of 20180121) - will notify when complete
Goto your IDE workspace
Clone the logging repo
https://gerrit.onap.org/r/#/admin/projects/logging-analytics
https://git.onap.org/logging-analytics/tree/
goto the reference folder and load the pom.xml project
Code Block |
---|
obrienbiometrics:wse_sts392b michaelobrien$ git clone ssh://michaelobrien@gerrit.onap.org:29418/logging-analytics Cloning into 'logging-analytics'... Warning: Permanently added '[gerrit.onap.org]:29418,[198.145.29.92]:29418' (RSA) to the list of known hosts. remote: Total 374 (delta 0), reused 374 (delta 0) Receiving objects: 100% (374/374), 1001.91 KiB | 1.22 MiB/s, done. Resolving deltas: 100% (120/120), done. obrienbiometrics:wse_sts392b michaelobrien$ cd logging-analytics/reference/ obrienbiometrics:reference michaelobrien$ ls\ logging-demo logging-library logging-mock-service pom.xml # build the project to verify obrienbiometrics:reference michaelobrien$ mvn clean install -U -DskipTests=true -Dmaven.test.skip=true -Dmaven.javadoc.skip=true -Dadditionalparam=-Xdoclint:none [INFO] logging-reference .................................. SUCCESS [ 3.069 s] [INFO] logging-library .................................... SUCCESS [ 1.313 s] [INFO] logging-mock-service ............................... SUCCESS [ 0.395 s] [INFO] logging-demo ....................................... SUCCESS [ 2.329 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 9.232 s [INFO] Finished at: 2018-01-21T09:29:15-05:00 |
Audits
Existing EELF implementations
Please use the latest 1.0.0 from https://github.com/att/EELF - nexus artifacts pending
app | project | repo | version | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
aai | aai-common | 1.0.0 | |||||||||
aai | aai-core | 1.0.0 | |||||||||
aai | aai-service (deprecated) | 0.0.1 (not in use) | |||||||||
aai | aai-traversal | 1.0.0 | |||||||||
aai | ajsc-aai | 1.0.0 | |||||||||
aai | eelf-logging | logging-service | 0.0.1 (not in use) | ||||||||
appc | (pax.logging = wrapper on log4j, slf4j, commons-logging 1.2 and logback 1.1.1) needs this RI - provide a demo to the appc team | 0.0.1 (upgrade?) | |||||||||
ccsdk | nbapi | ccsdk/platform/nbapi | 1.0.0 | ||||||||
dmaap | buscontroller | dmaap/buscontroller | 0.0.1 | ||||||||
ui | dcae_dmaap_webapp | dmaapbc | 1.0.0 | ||||||||
ui | dmaap-bc-app-common | 1.0.0 | |||||||||
ui | dmaap-bc-app-parent | 1.0.0 | |||||||||
policy | common-modules/ | common | 1.0.0 | ||||||||
policy | drools-pdp | 1.0.0 | |||||||||
portal | ecompportal-be-common | portal | <eelf.version>1.0.0 | ||||||||
policy | feature-eelf | drools-pdp | 1.0.0 | ||||||||
policy | PolicyEngineSuite/ONAP-logging | common | 0.0.1 (not in use) | ||||||||
policy | ONAP-SDK-APP | engine | 0.0.1 (not in use) | ||||||||
sdc | ? | ||||||||||
so | common | so | 0.0.1 (upgrade ?) | ||||||||
vid | epsdk-app-onap | vid | 1.0.0 | ||||||||
vid | vid-app-common | vid | 1.0.0 | ||||||||
open-o | |||||||||||
multicloud | python based |
|
Projects with a filebeat container as of 20170114
Code Block |
---|
ubuntu@ip-172-31-49-200:~$ kubectl get pods --all-namespaces | grep 2/2 onap-aai aai-resources-1039856271-3ph1d 2/2 Running 0 37m onap-aai aai-traversal-143081090-f4cls 2/2 Running 0 37m onap-aai model-loader-service-4144225433-0jr21 2/2 Running 0 37m onap-aai search-data-service-3842430948-53j1w 2/2 Running 0 37m onap-aai sparky-be-4222608366-966p4 2/2 Running 0 37m onap-appc appc-1828810488-mhsf7 2/2 Running 0 37m onap-mso mso-736294529-c2p8v 2/2 Running 0 37m onap-policy drools-2600956298-dp7bq 2/2 Running 0 37m onap-policy pap-466625067-n23bp 2/2 Running 0 37m onap-policy pdp-2354817903-kmxv5 2/2 Running 0 37m onap-portal portalapps-1783099045-79lcq 2/2 Running 0 37m onap-sdc sdc-be-2336519847-p6p5l 2/2 Running 0 37m onap-sdc sdc-fe-2862673798-xcpnk 2/2 Running 0 37m onap-sdnc sdnc-0 2/2 Running 0 37m onap-sdnc sdnc-dbhost-0 2/2 Running 0 37m onap-vid vid-server-3026751708-rpmq4 2/2 Running 0 37m |
Current Logging Libraries
Current Framework usage impacting Logging
Logging in VNFs
Design Issues
DI 1: 20171231: Initial Maven Project Structure
https://gerrit.onap.org/r/#/admin/projects/logging-analytics
git clone ssh://michaelobrien@gerrit.onap.org:29418/logging-analytics
see
Code Block |
---|
obrienbiometrics:logging-analytics michaelobrien$ git status On branch master Your branch is up-to-date with 'origin/master'. Changes to be committed: (use "git reset HEAD <file>..." to unstage) new file: reference/logging-demo/.gitignore new file: reference/logging-demo/pom.xml new file: reference/logging-demo/src/main/java/org/onap/logging/LoggingReference.java new file: reference/logging-library/.gitignore new file: reference/logging-library/pom.xml new file: reference/logging-library/src/main/java/org/onap/logging/LoggingReference.java new file: reference/pom.xml Untracked files: (use "git add <file>..." to include in what will be committed) reference/.project reference/.settings/ reference/logging-demo/.checkstyle reference/logging-demo/.classpath reference/logging-demo/.project reference/logging-demo/.settings/ reference/logging-library/.checkstyle reference/logging-library/.classpath reference/logging-library/.project reference/logging-library/.settings/ reference/target/ obrienbiometrics:logging-analytics michaelobrien$ git commit -m "Initial logging lib/demo proj struct" [master 9e42247] Initial logging lib/demo proj struct 7 files changed, 306 insertions(+) create mode 100644 reference/logging-demo/.gitignore create mode 100644 reference/logging-demo/pom.xml create mode 100644 reference/logging-demo/src/main/java/org/onap/logging/LoggingReference.java create mode 100644 reference/logging-library/.gitignore create mode 100644 reference/logging-library/pom.xml create mode 100644 reference/logging-library/src/main/java/org/onap/logging/LoggingReference.java create mode 100644 reference/pom.xml obrienbiometrics:logging-analytics michaelobrien$ git review -s obrienbiometrics:logging-analytics michaelobrien$ git commit -s --amend [master c11cd50] Initial logging lib/demo proj struct Date: Sun Dec 31 13:30:04 2017 -0500 7 files changed, 306 insertions(+) create mode 100644 reference/logging-demo/.gitignore create mode 100644 reference/logging-demo/pom.xml create mode 100644 reference/logging-demo/src/main/java/org/onap/logging/LoggingReference.java create mode 100644 reference/logging-library/.gitignore create mode 100644 reference/logging-library/pom.xml create mode 100644 reference/logging-library/src/main/java/org/onap/logging/LoggingReference.java create mode 100644 reference/pom.xml obrienbiometrics:logging-analytics michaelobrien$ git review Warning: Permanently added '[gerrit.onap.org]:29418,[198.145.29.92]:29418' (RSA) to the list of known hosts. remote: Processing changes: new: 1, refs: 1, done remote: remote: New Changes: remote: https://gerrit.onap.org/r/27195 Initial logging lib/demo proj struct remote: To ssh://gerrit.onap.org:29418/logging-analytics * [new branch] HEAD -> refs/publish/master |
For eclipse you will need to make sure the spring nature in the .classpath is set correctly - in order for maven build to also work in eclipse for war projects - this will show up in the project build path
Code Block |
---|
<classpathentry exported="true" kind="con" path="org.eclipse.m2e.MAVEN2_CLASSPATH_CONTAINER"> <attributes> <attribute name="maven.pomderived" value="true"/> <attribute name="org.eclipse.jst.component.nondependency" value=""/> <attribute name="org.eclipse.jst.component.dependency" value="/WEB-INF/lib"/> </attributes> </classpathentry> |
DI 2: 20171231: Audit: EELF implementors
DI 3: 20171231: Audit: Mixed version of EELF dependency
Jira | ||||||
---|---|---|---|---|---|---|
|
Figuring out the reason for the different dependencies
aai-service, ajsc-aai, common-modules/common-logging
<dependency>
<groupId>com.att.eelf</groupId>
<artifactId>eelf-core</artifactId>
<version>0.0.1</version>
</dependency>
aai-common, eelf-logging, ONAP-logging, ONAP-SDK-APP
<dependency>
<groupId>com.att.eelf</groupId>
<artifactId>eelf-core</artifactId>
<version>1.0.0</version>
</dependency>
DI 4: 20170104: ETag propagation
Passing UUID based requestID's via rest calls
https://en.wikipedia.org/wiki/HTTP_ETag
DI 5: 20170120: Docker image design
PV for logback.xml,
container for mock-log-nbi
container for mock-log-proc (the scalable TX substrate)
DI 6: 20170120: Docker wrapper of Logging Library
Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key LOG-136
Code Block |
---|
obrienbiometrics:docker michaelobrien$ vi DockerFile obrienbiometrics:docker michaelobrien$ docker build -t mock-logging-tomcat -f DockerFile . Sending build context to Docker daemon 18.04MB Step 1/2 : FROM tomcat:8.0.48-jre8 ---> e072422ca96f Step 2/2 : COPY logging-demo-1.2.0-SNAPSHOT.war /usr/local/tomcat/webapps/logging-demo.war ---> f0d234622af6 Successfully built f0d234622af6 Successfully tagged mock-logging-tomcat:latest obrienbiometrics:docker michaelobrien$ docker images | grep mock-logging mock-logging-tomcat latest f0d234622af6 9 seconds ago 576MB obrienbiometrics:docker michaelobrien$ docker run -d -it --rm -p 8888:8080 mock-logging-tomcat:latest ef5bdffb0e0aef9f185a51be2a915eae3d72fdb9a2125e1bb373516137c205b4 obrienbiometrics:docker michaelobrien$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ef5bdffb0e0a mock-logging-tomcat:latest "catalina.sh run" 5 seconds ago Up 4 seconds 0.0.0.0:8888->8080/tcp adoring_hawking obrienbiometrics:docker michaelobrien$ docker logs -f adoring_hawking 20-Jan-2018 04:30:26.092 INFO [localhost-startStop-1] org.springframework.web.context.ContextLoader.initWebApplicationContext Root WebApplicationContext: initialization completed in 1352 ms 20-Jan-2018 04:30:26.592 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive /usr/local/tomcat/webapps/logging-demo.war has finished in 3,652 ms 20-Jan-2018 04:30:26.841 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 3943 ms ^C obrienbiometrics:docker michaelobrien$ curl http://localhost:8888/logging-demo/rest/read/test testing: org.onap.logging.ApplicationService@433552a obrienbiometrics:docker michaelobrien$ docker build -t obrienlabs/mock-logging-tomcat -f DockerFile . Sending build context to Docker daemon 18.04MB Step 1/2 : FROM tomcat:8.0.48-jre8 ---> e072422ca96f Step 2/2 : COPY logging-demo-1.2.0-SNAPSHOT.war /usr/local/tomcat/webapps/logging-demo.war ---> Using cache ---> f0d234622af6 Successfully built f0d234622af6 Successfully tagged obrienlabs/mock-logging-tomcat:latest obrienbiometrics:docker michaelobrien$ docker tag obrienlabs/mock-logging-tomcat obrienlabs/mock-logging-tomcat:1 obrienbiometrics:docker michaelobrien$ docker login Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one. Username (michaelobrien): obrienlabs Password: Login Succeeded obrienbiometrics:docker michaelobrien$ docker push obrienlabs/mock-logging-tomcat:1 The push refers to a repository [docker.io/obrienlabs/mock-logging-tomcat] 8bb560dc792d: Pushed 7a5faefa0b46: Pushed d1a0f91a1ad8: Pushed c4d99fd86e94: Pushed 04178c827c65: Pushed c571db474d75: Pushed 93742a1ae069: Pushed 7ffe9646653b: Pushed 6ef532e39c1e: Pushed a4d7b0ac0438: Pushed 06f4de5fefea: Pushed 851f3e348c69: Pushed e27a10675c56: Pushed 1: digest: sha256:a583f08001b21635e74e205f130ffbd23d933032686345be50f39a46b537e11b size: 3048 run obrienbiometrics:docker michaelobrien$ docker run -d -it --rm -p 8888:8080 obrienlabs/mock-logging-tomcat:1 316023a8abed5911000a1c366b7c897a33abcd7e9e70f48a4e2c2611f4c98d93 obrienbiometrics:docker michaelobrien$ curl http://localhost:8888/logging-demo/rest/read/test testing: org.onap.logging.ApplicationService@75f0964b obrienbiometrics:docker michaelobrien$ test on separate machine ubuntu@ip-172-31-52-161:~$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ubuntu@ip-172-31-52-161:~$ docker run -d -it -p 8888:8080 obrienlabs/mock-logging-tomcat:1 Unable to find image 'obrienlabs/mock-logging-tomcat:1' locally 1: Pulling from obrienlabs/mock-logging-tomcat 723254a2c089: Pull complete abe15a44e12f: Pull complete 409a28e3cc3d: Pull complete a9511c68044a: Pull complete 9d1b16e30bc8: Pull complete 0fc5a09c9242: Pull complete d34976006493: Pull complete 3b70003f0c10: Pull complete bc7887582e2e: Pull complete d2ab4f165865: Pull complete 7a0257d5eec9: Pull complete a23d5c1d9825: Pull complete 42f73552ced4: Pull complete Digest: sha256:a583f08001b21635e74e205f130ffbd23d933032686345be50f39a46b537e11b Status: Downloaded newer image for obrienlabs/mock-logging-tomcat:1 4262ca54a570a177185b646ec92b440f37250f7ee2922e018aec5d226c5b4f2a ubuntu@ip-172-31-52-161:~$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 4262ca54a570 obrienlabs/mock-logging-tomcat:1 "catalina.sh run" 9 seconds ago Up 8 seconds 0.0.0.0:8888->8080/tcp kickass_hoover ubuntu@ip-172-31-52-161:~$ curl http://dev.onap.info:8888/logging-demo/rest/read/test testing: org.onap.logging.ApplicationService@86a2339 |
Implementation
Deploying the RI
WIP
Kubernetes deployment of the RI
Code Block |
---|
"kind": "Deployment", "apiVersion": "extensions/v1beta1", "metadata": { "name": "mock-logging-tomcat", "namespace": "default", "selfLink": "/apis/extensions/v1beta1/namespaces/default/deployments/mock-logging-tomcat", "uid": "740b40f3-fda1-11e7-a4b4-02e171d3f6c2", "resourceVersion": "677", "generation": 1, "creationTimestamp": "2018-01-20T05:19:13Z", "labels": { "app": "mock-logging-tomcat", "version": "1" }, "annotations": { "deployment.kubernetes.io/revision": "1" } }, "spec": { "replicas": 1, "selector": { "matchLabels": { "app": "mock-logging-tomcat", "version": "1" } }, "template": { "metadata": { "name": "mock-logging-tomcat", "creationTimestamp": null, "labels": { "app": "mock-logging-tomcat", "version": "1" } }, "spec": { "containers": [ { "name": "mock-logging-tomcat", "image": "obrienlabs/mock-logging-tomcat:1", "resources": {}, "terminationMessagePath": "/dev/termination-log", "terminationMessagePolicy": "File", "imagePullPolicy": "IfNotPresent", "securityContext": { "privileged": false } } ], "restartPolicy": "Always", "terminationGracePeriodSeconds": 30, "dnsPolicy": "ClusterFirst", "securityContext": {}, "schedulerName": "default-scheduler" } }, "strategy": { "type": "RollingUpdate", "rollingUpdate": { "maxUnavailable": 1, "maxSurge": 1 } } }, "status": { "observedGeneration": 1, "replicas": 1, "updatedReplicas": 1, "readyReplicas": 1, "availableReplicas": 1, "conditions": [ { "type": "Available", "status": "True", "lastUpdateTime": "2018-01-20T05:19:13Z", "lastTransitionTime": "2018-01-20T05:19:13Z", "reason": "MinimumReplicasAvailable", "message": "Deployment has minimum availability." } ] } } |
Local Debugging via Eclipse
Jira | ||||||
---|---|---|---|---|---|---|
|
Depending on how you set your port overrides for Tomcat 8.x
http://localhost:8988/logging-demo/rest/read/test
Testing
http://127.0.0.1:8080/logging-demo/rest/read/test
Links
EELF
git clone https://github.com/att/EELF.git
https://tomcat.apache.org/download-80.cgi
https://spring.io/tools/sts/all
https://www.sitepoint.com/inside-java-9-part-ii/#performanceimprovements