...

Questions on onap-discuss from the teams
https://lists.onap.org/pipermail/onap-discuss/2018-January/007415.html

Latest Filebeat/ELK implementation by Clamp for reference - https://gerrit.onap.org/r/#/c/48961/ on

Jira

server	ONAP JIRA
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	CLAMP-170

Jira

server	ONAP JIRA
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	LOG-118

Purpose

The goals of the logging project are the following

...

Processing Node WAR

Project Structure

Project	repo	Dependencies	Artifact
logging-reference	logging-reference		pom
logging-demo	logging-demo		WAR
logging-mock-service	logging-mock-service	logging-library	WAR
Logging Library	logging-library		JAR
Kubernetes demo spec	kubernetes/log-mock-

nbi

demo		yaml, config
Kubernetes service spec	kubernetes/log-mock-

proc

service		yaml, config
	logging-docker-root		pom
	logging-docker-

nbi

demo	logging-demo	DockerFile
	logging-docker-

proc

service

logging-mock-service

DockerFile

API

Deployment Diagram

Gliffy Diagram

name	Logging-RI-Deployment
pagePin	3

Quickstart

WIP (as of 20180121) - will notify when complete

...

app

project

repo

version

aai

aai-common

1.0.0

aai

aai-core

1.0.0

aai

aai-service (deprecated)

0.0.1 (not in use)

aai

aai-traversal

1.0.0

aai

ajsc-aai

1.0.0

aai

eelf-logging

logging-service

0.0.1 (not in use)

appc

(pax.logging = wrapper on log4j, slf4j, commons-logging 1.2 and logback 1.1.1)

needs this RI - provide a demo to the appc team

0.0.1 (upgrade?)

ccsdk

nbapi

ccsdk/platform/nbapi

1.0.0

dmaap

buscontroller

dmaap/buscontroller

0.0.1

ui

dcae_dmaap_webapp

dmaapbc

1.0.0

ui

dmaap-bc-app-common

1.0.0

ui

dmaap-bc-app-parent

1.0.0

policy

common-modules/

common

1.0.0

policy

drools-pdp

1.0.0

portal

ecompportal-be-common

portal

<eelf.version>1.0.0
</eelf.version>

policy

feature-eelf

drools-pdp

1.0.0

policy

PolicyEngineSuite/ONAP-logging

common

0.0.1 (not in use)

policy

ONAP-SDK-APP

engine

0.0.1 (not in use)

sdc

?

so

common

so

0.0.1 (upgrade ?)

vid

epsdk-app-onap

vid

1.0.0

vid

vid-app-common

vid

1.0.0

open-o

multicloud

python based

Jira

server	ONAP JIRA
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	MULTICLOUD-151

ONAP components with a Filebeat container

Projects with a filebeat container as of 20170114

...

container for mock-log-proc (the scalable TX substrate)

DI 6: 20170120: Docker wrapper of Logging Library

Jira

server	ONAP JIRA
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	LOG-136

DockerFile

Code Block
FROM tomcat:8.0.48-jre8 COPY target/logging-demo-1.2.0-SNAPSHOT.war /usr/local/tomcat/webapps/logging-demo.war

build.sh

Building, tagging and pushing the docker image so we can consume it in Kubernetes

Code Block

mkdir target
cp ../../logging-demo/target/*.war target

Code Block

obrienbiometrics:docker michaelobrien$ vi DockerFile
obrienbiometrics:docker michaelobrien$ docker build -t mockobrienlabs/logging-loggingdemo-tomcatnbi -f DockerFile .
Sendingdocker buildimages context| to Docker daemon  18.04MB
Step 1/2 : FROM tomcat:8.0.48-jre8
---> e072422ca96f
Step 2/2 : COPY grep logging-demo-nbi
docker tag obrienlabs/logging-demo-nbi obrienlabs/logging-demo-1.2.0-SNAPSHOT.war /usr/local/tomcat/webappsnbi:1
docker login
docker push obrienlabs/logging-demo-nbi:1
#docker run -d -it --rm -p 8888:8080 obrienlabs/logging-demo-nbi:latest

DI 6: 20170120: Docker wrapper of Logging Library

Jira

server	ONAP JIRA
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	LOG-136

Code Block

obrienbiometrics:docker michaelobrien$ vi DockerFile
obrienbiometrics:docker michaelobrien$ docker build -t mock-logging-tomcat -f DockerFile .
Sending build context to Docker daemon  18.04MB
Step 1/2 : FROM tomcat:8.0.48-jre8
---> e072422ca96f
Step 2/2 : COPY logging-demo-1.2.0-SNAPSHOT.war /usr/local/tomcat/webapps/logging-demo.war
---> f0d234622af6
Successfully built f0d234622af6
Successfully tagged mock-logging-tomcat:latest
obrienbiometrics:docker michaelobrien$ docker images | grep mock-logging
mock-logging-tomcat.war
---> f0d234622af6
Successfully built f0d234622af6
Successfully tagged mock-logging-tomcat:latest
obrienbiometrics:docker michaelobrien$ docker images | grep mock-logging
mock-logging-tomcat                                           latest                            f0d234622af6        9 seconds ago       576MB
obrienbiometrics:docker michaelobrien$ dockerlatest run -d -it --rm -p 8888:8080                       f0d234622af6        9 seconds ago       576MB
obrienbiometrics:docker michaelobrien$ docker run -d -it --rm -p 8888:8080 mock-logging-tomcat:latest
ef5bdffb0e0aef9f185a51be2a915eae3d72fdb9a2125e1bb373516137c205b4
obrienbiometrics:docker michaelobrien$ docker ps
CONTAINER ID        IMAGE                        COMMAND             CREATED             STATUS              PORTS                    NAMES
ef5bdffb0e0a        mock-logging-tomcat:latest   "catalina.sh run"   5 seconds ago       Up 4 seconds        0.0.0.0:8888->8080/tcp   adoring_hawking
obrienbiometrics:docker michaelobrien$ docker logs -f adoring_hawking

20-Jan-2018 04:30:26.092 INFO [localhost-startStop-1] org.springframework.web.context.ContextLoader.initWebApplicationContext Root WebApplicationContext: initialization completed in 1352 ms
20-Jan-2018 04:30:26.592 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive /usr/local/tomcat/webapps/logging-demo.war has finished in 3,652 ms
20-Jan-2018 04:30:26.841 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 3943 ms
^C
obrienbiometrics:docker michaelobrien$ curl http://localhost:8888/logging-demo/rest/read/test
testing: org.onap.logging.ApplicationService@433552a
obrienbiometrics:docker michaelobrien$ docker build -t obrienlabs/mock-logging-tomcat -f DockerFile .
Sending build context to Docker daemon  18.04MB
Step 1/2 : FROM tomcat:8.0.48-jre8
---> e072422ca96f
Step 2/2 : COPY logging-demo-1.2.0-SNAPSHOT.war /usr/local/tomcat/webapps/logging-demo.war
---> Using cache
---> f0d234622af6
Successfully built f0d234622af6
Successfully tagged obrienlabs/mock-logging-tomcat:latest
obrienbiometrics:docker michaelobrien$ docker tag obrienlabs/mock-logging-tomcat obrienlabs/mock-logging-tomcat:1
obrienbiometrics:docker michaelobrien$ docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username (michaelobrien): obrienlabs
Password:
Login Succeeded
obrienbiometrics:docker michaelobrien$ docker push obrienlabs/mock-logging-tomcat:1
The push refers to a repository [docker.io/obrienlabs/mock-logging-tomcat]
8bb560dc792d: Pushed
7a5faefa0b46: Pushed
d1a0f91a1ad8: Pushed
c4d99fd86e94: Pushed
04178c827c65: Pushed
c571db474d75: Pushed
93742a1ae069: Pushed
7ffe9646653b: Pushed
6ef532e39c1e: Pushed
a4d7b0ac0438: Pushed
06f4de5fefea: Pushed
851f3e348c69: Pushed
e27a10675c56: Pushed
1: digest: sha256:a583f08001b21635e74e205f130ffbd23d933032686345be50f39a46b537e11b size: 3048

run

obrienbiometrics:docker michaelobrien$ docker run -d -it --rm -p 8888:8080 obrienlabs/mock-logging-tomcat:1
316023a8abed5911000a1c366b7c897a33abcd7e9e70f48a4e2c2611f4c98d93
obrienbiometrics:docker michaelobrien$ curl http://localhost:8888/logging-demo/rest/read/test
testing: org.onap.logging.ApplicationService@75f0964b
obrienbiometrics:docker michaelobrien$


test on separate machine

ubuntu@ip-172-31-52-161:~$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
ubuntu@ip-172-31-52-161:~$ docker run -d -it  -p 8888:8080 obrienlabs/mock-logging-tomcat:1
Unable to find image 'obrienlabs/mock-logging-tomcat:1' locally
1: Pulling from obrienlabs/mock-logging-tomcat
723254a2c089: Pull complete
abe15a44e12f: Pull complete
409a28e3cc3d: Pull complete
a9511c68044a: Pull complete
9d1b16e30bc8: Pull complete
0fc5a09c9242: Pull complete
d34976006493: Pull complete
3b70003f0c10: Pull complete
bc7887582e2e: Pull complete
d2ab4f165865: Pull complete
7a0257d5eec9: Pull complete
a23d5c1d9825: Pull complete
42f73552ced4: Pull complete
Digest: sha256:a583f08001b21635e74e205f130ffbd23d933032686345be50f39a46b537e11b
Status: Downloaded newer image for obrienlabs/mock-logging-tomcat:1
4262ca54a570a177185b646ec92b440f37250f7ee2922e018aec5d226c5b4f2a
ubuntu@ip-172-31-52-161:~$ docker ps
CONTAINER ID        IMAGE                              COMMAND             CREATED             STATUS              PORTS                    NAMES
4262ca54a570        obrienlabs/mock-logging-tomcat:1   "catalina.sh run"   9 seconds ago       Up 8 seconds        0.0.0.0:8888->8080/tcp   kickass_hoover
ubuntu@ip-172-31-52-161:~$ curl http://dev.onap.info:8888/logging-demo/rest/read/test
testing: org.onap.logging.ApplicationService@86a2339

DI 7: 20180126: Config for Logging Demo Containers

Jira

server	ONAP JIRA
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	OOM-628

DI 8: 20180204: NodePort port reservation framework required

Jira

server	ONAP JIRA
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	OOM-656

DI 9: 20180417: Restart a failed ELK stack container

Periodically one of the higher containers in a dependency tree will not get restarted in time to pick up running child containers - usually this is the kibana container

Fix this or "any" container by deleting the container in question and kubernetes will bring another one up.

Code Block

theme	Midnight

root@a-onap-auto-20180412-ref:~# kubectl get services --all-namespaces | grep log
onap          dev-vfc-catalog               ClusterIP   10.43.210.8     <none>        8806/TCP                                                                     5d
onap          log-es                        NodePort    10.43.77.87     <none>        9200:30254/TCP                                                               5d
onap          log-es-tcp                    ClusterIP   10.43.159.93    <none>        9300/TCP                                                                     5d
onap          log-kibana                    NodePort    10.43.41.102    <none>        5601:30253/TCP                                                               5d
onap          log-ls                        NodePort    10.43.180.165   <none>        5044:30255/TCP                                                               5d
onap          log-ls-http                   ClusterIP   10.43.13.180    <none>        9600/TCP                                                                     5d
root@a-onap-auto-20180412-ref:~# kubectl get pods --all-namespaces | grep log
onap          dev-log-elasticsearch-66cdc4f855-wmpkz         1/1       Running                 0          5d
onap          dev-log-kibana-5b6f86bcb4-drpzq                0/1       Running                 1076       5d
onap          dev-log-logstash-6d9fdccdb6-ngq2f              1/1       Running                 0          5d
onap          dev-vfc-catalog-7d89bc8b9d-vxk74               2/2       Running                 0          5d
root@a-onap-auto-20180412-ref:~# kubectl delete pod dev-log-kibana-5b6f86bcb4-drpzq -n onap
pod "dev-log-kibana-5b6f86bcb4-drpzq" deleted
root@a-onap-auto-20180412-ref:~# kubectl get pods --all-namespaces | grep log
onap          dev-log-elasticsearch-66cdc4f855-wmpkz         1/1       Running                 0          5d
onap          dev-log-kibana-5b6f86bcb4-drpzq                0/1       Terminating             1076       5d
onap          dev-log-kibana-5b6f86bcb4-gpn2m                0/1       Pending                 0          12s
onap          dev-log-logstash-6d9fdccdb6-ngq2f              1/1       Running                 0          5d
onap          dev-vfc-catalog-7d89bc8b9d-vxk74               2/2       Running                 0          5d

Implementation

Deploying the RI

WIP

Jira

server	ONAP JIRA
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	LOG-137

Deploy a Kubernetes cluster as normal via (No ONAP yet, just the K8S cluster) - ONAP on Kubernetes#InstallDocker/Rancher/Helm/Kubectl

Run the config and log pods

Code Block

# clone oom
git clone ssh://michaelobrien@gerrit.onap.org:29418/oom
# clone logging-analytics
git COMMAND             CREATED             STATUS              PORTS                    NAMES
ef5bdffb0e0a        mock-logging-tomcat:latest   "catalina.sh run"   5 seconds agoclone ssh://michaelobrien@gerrit.onap.org:29418/logging-analytics

# 
./createConfig.sh -n onap
./createAll.bash -n onap -a log
# temp create config dirs
ubuntu@ip-172-31-52-161:~/oom/kubernetes/oneclick$ sudo mkdir /dockerdata-nfs/onap/logdemo
ubuntu@ip-172-31-52-161:~/oom/kubernetes/oneclick$ sudo mkdir /dockerdata-nfs/onap/logdemo/eteshare
# copy the logdemo folder into the oom folder
root@obriensystemsu0:~/logging-analytics/reference/logging-kubernetes# cp -R logdemo ~/oom/kubernetes/
# make sure the 3 ELK log containers are up
root@obriensystemsu0:~/oom/kubernetes/oneclick# ./createAll.bash -n onap -a log
root@obriensystemsu0:~/oom/kubernetes/oneclick# kubectl get pods --all-namespaces -a
onap-log      elasticsearch-3138115406-mlwjf         1/1       UpRunning 4 seconds   0     0.0.0.0:8888->8080/tcp   adoring_hawking
obrienbiometrics:docker michaelobrien$ docker12m
onap-log logs -f adoring_hawking

20-Jan-2018 04:30:26.092 INFO [localhost-startStop-1] org.springframework.web.context.ContextLoader.initWebApplicationContext Root WebApplicationContext: initialization completed in 1352 ms
20-Jan-2018 04:30:26.592 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive /usr/local/tomcat/webapps/logging-demo.war has finished in 3,652 ms
20-Jan-2018 04:30:26.841 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 3943 ms
^C
obrienbiometrics:docker michaelobrien$ curl http://localhost:8888/logging-demo/rest/read/test
testing: org.onap.logging.ApplicationService@433552a
obrienbiometrics:docker michaelobrien$ docker build -t obrienlabs/mock-logging-tomcat -f DockerFile .
Sending build context to Docker daemon  18.04MB
Step 1/2 : FROM tomcat:8.0.48-jre8
---> e072422ca96f
Step 2/2 : COPY logging-demo-1.2.0-SNAPSHOT.war /usr/local/tomcat/webapps/logging-demo.war
---> Using cache
---> f0d234622af6
Successfully built f0d234622af6
Successfully tagged obrienlabs/mock-logging-tomcat:latest
obrienbiometrics:docker michaelobrien$ docker tag obrienlabs/mock-logging-tomcat obrienlabs/mock-logging-tomcat:1
obrienbiometrics:docker michaelobrien$ docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username (michaelobrien): obrienlabs
Password:
Login Succeeded
obrienbiometrics:docker michaelobrien$ docker push obrienlabs/mock-logging-tomcat:1
The push refers to a repository [docker.io/obrienlabs/mock-logging-tomcat]
8bb560dc792d: Pushed
7a5faefa0b46: Pushed
d1a0f91a1ad8: Pushed
c4d99fd86e94: Pushed
04178c827c65: Pushed
c571db474d75: Pushed
93742a1ae069: Pushed
7ffe9646653b: Pushed
6ef532e39c1e: Pushed
a4d7b0ac0438: Pushed
06f4de5fefea: Pushed
851f3e348c69: Pushed
e27a10675c56: Pushed
1: digest: sha256:a583f08001b21635e74e205f130ffbd23d933032686345be50f39a46b537e11b size: 3048

run

obrienbiometrics:docker michaelobrien$ docker run -d -it --rm -p 8888:8080 obrienlabs/mock-logging-tomcat:1
316023a8abed5911000a1c366b7c897a33abcd7e9e70f48a4e2c2611f4c98d93
obrienbiometrics:docker michaelobrien$ curl http://localhost:8888/logging-demo/rest/read/test
testing: org.onap.logging.ApplicationService@75f0964b
obrienbiometrics:docker michaelobrien$


test on separate machine

ubuntu@ip-172-31-52-161:~$ docker ps
CONTAINER ID    kibana-3372627750-260f2                1/1       Running     0          12m
onap-log      logstash-1708188010-bhlqf              1/1       Running     0          12m

# create the log demo
root@obriensystemsu0:~/oom/kubernetes/oneclick# ./createAll.bash -n onap -a logdemo

********** Creating instance 1 of ONAP with port range 30200 and 30399

********** Creating ONAP: 


********** Creating deployments for logdemo ********** 

Creating namespace **********
namespace "onap-logdemo" created

Creating service account **********
clusterrolebinding "onap-logdemo-admin-binding" created

Creating registry secret **********
secret "onap-docker-registry-key" created

Creating deployments and services **********
NAME:   onap-logdemo
LAST DEPLOYED: Mon Jan 29 11:25:44 2018
NAMESPACE: onap
STATUS: DEPLOYED

RESOURCES:
==> v1/Service
NAME     TYPE      CLUSTER-IP   EXTERNAL-IP  PORT(S)         AGE
logdemo  NodePort  10.43.26.45  <none>       8080:30299/TCP  0s

==> v1beta1/Deployment
NAME     DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE
logdemo  1        IMAGE1        1       COMMAND    0         CREATED 0s

==> v1/Pod(related)
NAME           STATUS           READY  STATUS PORTS            RESTARTS   NAMESAGE
ubuntu@iplogdemo-172-31-52-161:~$ docker run -d -it  -p 8888:8080 obrienlabs/mock-logging-tomcat:1
Unable to find image 'obrienlabs/mock-logging-tomcat:1' locally
1: Pulling from obrienlabs/mock-logging-tomcat
723254a2c089: Pull complete
abe15a44e12f: Pull complete
409a28e3cc3d: Pull complete
a9511c68044a: Pull complete
9d1b16e30bc8: Pull complete
0fc5a09c9242: Pull complete
d34976006493: Pull complete
3b70003f0c10: Pull complete
bc7887582e2e: Pull complete
d2ab4f165865: Pull complete
7a0257d5eec9: Pull complete
a23d5c1d9825: Pull complete
42f73552ced4: Pull complete
Digest: sha256:a583f08001b21635e74e205f130ffbd23d933032686345be50f39a46b537e11b
Status: Downloaded newer image for obrienlabs/mock-logging-tomcat:1
4262ca54a570a177185b646ec92b440f37250f7ee2922e018aec5d226c5b4f2a
ubuntu@ip-172-31-52-161:~$ docker ps
CONTAINER ID 3133092390-g4qx7  0/1    ContainerCreating  0         0s





#check
root@obriensystemsu0:~/oom/kubernetes/oneclick#  kubectl get pods --all-namespaces | grep logdemo
onap-logdemo   logdemo-3133092390-g4qx7               1/1       Running   0       IMAGE   5m
root@obriensystemsu0:~/oom/kubernetes/oneclick# kubectl get services  --all-namespaces -o wide | grep logdemo
onap-logdemo   logdemo              COMMAND  NodePort    10.43.26.45     <none>  CREATED      8080:30299/TCP   6m    STATUS    app=logdemo


root@obriensystemsu0:~/logging-analytics/reference# kubectl -n onap-logdemo logs -f logdemo-3133092390-g4qx7
29-Jan-2018 17:28:03.456 INFO  PORTS                    NAMES
4262ca54a570        obrienlabs/mock-logging-tomcat:1   "catalina.sh run"   9 seconds ago       Up 8 seconds        0.0.0.0:8888->8080/tcp   kickass_hoover[main] org.apache.catalina.startup.Catalina.start Server startup in 11584 ms

# test in-container and out-container
ubuntu@ip-172-31-52-161:~/oom/kubernetes/oneclick$ kubectl -n onap-logdemo exec -it logdemo-4091951930-zwd53 bash
root@logdemo-4091951930-zwd53:/usr/local/tomcat# ls webapps/
ROOT  docs  examples  host-manager  logging-demo  logging-demo.war  manager

root@logdemo-4091951930-zwd53:/usr/local/tomcat# curl http://127.0.0.1:8080/logging-demo/rest/read/test
testing: org.onap.demo.logging.ApplicationService@7a4fa9b

outside
ubuntu@ip-172-31-52-161:~$~/oom/kubernetes/oneclick$ curl http://dev.onap.info:888830299/logging-demo/rest/read/test
testing: org.onap.demo.logging.ApplicationService@86a2339

Implementation

Deploying the RI

ApplicationService@52772382

http://dev.onap.info:30299/logging-demo/rest/read/testWIP

Kubernetes deployment of the RI

...

Space shortcuts

Page tree

Versions Compared

Old Version 28

New Version Current

Key

Purpose

Processing Node WAR

Project Structure

API

Deployment Diagram

Quickstart

ONAP components with a Filebeat container

DI 6: 20170120: Docker wrapper of Logging Library

DockerFile

DI 6: 20170120: Docker wrapper of Logging Library

DI 7: 20180126: Config for Logging Demo Containers

DI 8: 20180204: NodePort port reservation framework required

DI 9: 20180417: Restart a failed ELK stack container

Implementation

Deploying the RI

Implementation

Deploying the RI

Kubernetes deployment of the RI

Space shortcuts

Page tree

Page History

Versions Compared

Old Version 28

New Version Current

Key

Purpose

Processing Node WAR

Project Structure

API

Deployment Diagram

Quickstart

ONAP components with a Filebeat container

DI 6: 20170120: Docker wrapper of Logging Library

DockerFile

DI 6: 20170120: Docker wrapper of Logging Library

DI 7: 20180126: Config for Logging Demo Containers

DI 8: 20180204: NodePort port reservation framework required

DI 9: 20180417: Restart a failed ELK stack container

Implementation

Deploying the RI

Implementation

Deploying the RI

Kubernetes deployment of the RI