Date: Thu, 28 Mar 2024 16:45:08 +0000 (UTC) Message-ID: <2064662179.105381.1711644308990@aws-us-west-2-onap-confluence-1.web.codeaurora.org> Subject: Exported From Confluence MIME-Version: 1.0 Content-Type: multipart/related; boundary="----=_Part_105380_1016080620.1711644308989" ------=_Part_105380_1016080620.1711644308989 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Content-Location: file:///C:/exported.html
The below steps covers manual setup of DCAE VM=E2=80=99s and DCAE servic= e components.
DCAE VES Collector can be configured on VM with ubuntu-16.04 image (m1.s= mall should suffice if this is only service) and 20Gb cinder storage
1) Install docker
=09sudo apt-get update
=09sudo apt install docker.io
2) Pull the latest container from onap nex= us
sudo docker login -u docker -p docker <= a href=3D"http://nexus.onap.org" class=3D"external-link" rel=3D"nofollow">n= exus3.onap.org:10001
sudo docker pull nexus3.onap= .org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:v1.1.0
3) Start the VESCollector with below comma= nd
sudo docker run -d --name vescollector = -p 8080:8080/tcp -p 8443:8443/tcp -P -e DMAAPHOST=3D'<dmaap IP>' nexu= s3.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:v1= .1.0
= Note: Change the dmaaphost to required DMAAP ip. T= o change the dmaap information for a running container, stop the acti= ve container and rerun above command changing the dmaap IP.
4) Verification
curl -i -X POST -d @<sampleves= > --header "Content-Type: application/json" http://loca= lhost:8080/eventListener/v5 -k
Note: If DMAAPHOST provided is invalid, y= ou will see exception around publish on the collector.logs (collector queue= s and attempts to resend the event hence exceptions reported will be period= ic).
3. Below two topic configuration ar= e pre-set into this container. When valid DMAAP instance ip was provi= ded and VES events are received, the collector will post to below topics.= p>
Fault - http://<dmaaphost>:39= 04/events/unauthenticated.SEC_FAULT_OUTPUT
Measurement -http://<dmaaphost>:390= 4/events/unauthenticated.SEC_MEASUREMENT_OUTPUT
To address windriver server in-stability, the below init.sh script was used to start the container on VM restart.
#!/bin/sh sudo docker ps | grep "vescollector" if [ $? -ne 0 ]; then sudo docker login -u docker -p d= ocker nexus3.onap.org:10001 sudo docker pull nexus3.onap.org= :10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.1 sudo docker rm -f vescollector echo "Collector process not runn= ing - $(date)" >> /home/ubuntu/startuplog sudo docker run -d --name vescol= lector -p 8080:8080/tcp -p 8443:8443/tcp -P -e DMAAPHOST=3D'10.12.25.96' ne= xus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:v1.1= .0 else echo "Collector process running = - $(date)" >> /home/ubuntu/startuplog fi
This script was invoked via VM init script (rc.d).
=09ln -s /home/ubuntu/init.sh /etc/init.d/init.sh
=09sudo update-rc.d init.sh start 2
The platform deploys CDAP as cluster and instantiates TCA. For the manua= l setup, we will leverage the CDAP SDK docker container to deploy TCA insta= nces. To setup TCA, choose VM with ubuntu-16.04 image, m1.medium size= and 50gb cinder volumes.
1) Install docker
sudo apt-get update
sudo apt install docker.io
2) Pull CDAP SDK container
=09sudo docker pull caskdata/cdap-standalone:4.1.2
3) Deploy and run the CDAP container
sudo docker run -d --name cdap-sdk-2 -= p 11011:11011 -p 11015:11015 caskdata/cdap-standalone:4.1.2
4) Create Namespace on CDAP application
=09curl -X PUT http://localhost:11015/v3= /namespaces/cdap_tca_hi_lo
5) Create TCA app config file - "tca_app_c= onfig.json" under ~ubuntu as below
{ "artifact": { "name": "dcae-analytics-cdap-tca", "version": "2.0.1", "scope": "user" }, "config": { "appName": "dcae-tca", "appDescription": "DCAE Analytics Threshold Crossing Alert Appl= ication", "tcaVESMessageStatusTableName": "TCAVESMessageStatusTable", "tcaVESMessageStatusTableTTLSeconds": 86400.0, "tcaAlertsAbatementTableName": "TCAAlertsAbatementTable", "tcaAlertsAbatementTableTTLSeconds": 1728000.0, "tcaVESAlertsTableName": "TCAVESAlertsTable", "tcaVESAlertsTableTTLSeconds": 1728000.0, "thresholdCalculatorFlowletInstances": 2.0, "tcaSubscriberOutputStreamName": "TCASubscriberOutputStream" } }
6) Create TCA app preference file - "tca_a= pp_preferences.json" under ~ubuntu as below
{ "publisherContentType" : "application/json", "publisherHostName" : "10.12.25.96", "publisherHostPort" : "3904", "publisherMaxBatchSize" : "1", "publisherMaxRecoveryQueueSize" : "100000", "publisherPollingInterval" : "20000", "publisherProtocol" : "http", "publisherTopicName" : "unauthenticated.DCAE_CL_OUTPUT", "subscriberConsumerGroup" : "OpenDCAE-c1", "subscriberConsumerId" : "c1", "subscriberContentType" : "application/json", "subscriberHostName" : "10.12.25.96", "subscriberHostPort" : "3904", "subscriberMessageLimit" : "-1", "subscriberPollingInterval" : "20000", "subscriberProtocol" : "http", "subscriberTimeoutMS" : "-1", "subscriberTopicName" : "unauthenticated.SEC_MEASUREMENT_OUTPUT", "enableAAIEnrichment" : false, "aaiEnrichmentHost" : "10.12.25.72", "aaiEnrichmentPortNumber" : 8443, "aaiEnrichmentProtocol" : "https", "aaiEnrichmentUserName" : "DCAE", "aaiEnrichmentUserPassword" : "DCAE", "aaiEnrichmentIgnoreSSLCertificateErrors" : true, "aaiVNFEnrichmentAPIPath" : "/aai/v11/network/generic-vnfs/generic-v= nf", "aaiVMEnrichmentAPIPath" : "/aai/v11/search/nodes-query", "tca_policy" : "{ \"domain\": \"measurementsForVfS= caling\", \"metricsPerEventName\": [{ &nb= sp; \"eventName\": \"vFirewallBroadcastPackets\", &nb= sp; \"controlLoopSchemaType\": \"VNF\", &nb= sp; \"policyScope\": \"DCAE\", &nb= sp; \"policyName\": \"DCAE.Config_tca-hi-lo\", &nb= sp; \"policyVersion\": \"v0.0.1\", &nb= sp; \"thresholds\": [{ &nb= sp; \"closedLoo= pControlName\": \"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850= a\", &nb= sp; \"version\"= : \"1.0.2\", &nb= sp; \"fieldPath= \": \"$.event.measurementsForVfScalingFields.vNicUsageArray[*].receivedTota= lPacketsDelta\", &nb= sp; \"threshold= Value\": 300, &nb= sp; \"direction= \": \"LESS_OR_EQUAL\", &nb= sp; \"severity\= ": \"MAJOR\", &nb= sp; \"closedLoo= pEventStatus\": \"ONSET\" &nb= sp; }, { &nb= sp; \"closedLoo= pControlName\": \"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850= a\", &nb= sp; \"version\"= : \"1.0.2\", &nb= sp; \"fieldPath= \": \"$.event.measurementsForVfScalingFields.vNicUsageArray[*].receivedTota= lPacketsDelta\", &nb= sp; \"threshold= Value\": 700, &nb= sp; \"direction= \": \"GREATER_OR_EQUAL\", &nb= sp; \"severity\= ": \"CRITICAL\", &nb= sp; \"closedLoo= pEventStatus\": \"ONSET\" &nb= sp; }] }, { &nb= sp; \"eventName\": \"vLoadBalancer\", &nb= sp; \"controlLoopSchemaType\": \"VM\", &nb= sp; \"policyScope\": \"DCAE\", &nb= sp; \"policyName\": \"DCAE.Config_tca-hi-lo\", &nb= sp; \"policyVersion\": \"v0.0.1\", &nb= sp; \"thresholds\": [{ &nb= sp; \"closedLoo= pControlName\": \"ControlLoop-vDNS-6f37f56d-a87d-4b85-b6a9-cc953cf779b3\", &nb= sp; \"version\"= : \"1.0.2\", &nb= sp; \"fieldPath= \": \"$.event.measurementsForVfScalingFields.vNicUsageArray[*].receivedTota= lPacketsDelta\", &nb= sp; \"threshold= Value\": 300, &nb= sp; \"direction= \": \"GREATER_OR_EQUAL\", &nb= sp; \"severity\= ": \"CRITICAL\", &nb= sp; \"closedLoo= pEventStatus\": \"ONSET\" &nb= sp; }] }, { &nb= sp; \"eventName\": \"Measurement_vGMUX\", &nb= sp; \"controlLoopSchemaType\": \"VNF\", &nb= sp; \"policyScope\": \"DCAE\", &nb= sp; \"policyName\": \"DCAE.Config_tca-hi-lo\", &nb= sp; \"policyVersion\": \"v0.0.1\", &nb= sp; \"thresholds\": [{ &nb= sp; \"closedLoo= pControlName\": \"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\", &nb= sp; \"version\"= : \"1.0.2\", &nb= sp; \"fieldPath= \": \"$.event.measurementsForVfScalingFields.additionalMeasurements[*].arra= yOfFields[0].value\", &nb= sp; \"threshold= Value\": 0, &nb= sp; \"direction= \": \"EQUAL\", &nb= sp; \"severity\= ": \"MAJOR\", &nb= sp; \"closedLoo= pEventStatus\": \"ABATED\" &nb= sp; }, { &nb= sp; \"closedLoo= pControlName\": \"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\", &nb= sp; \"version\"= : \"1.0.2\", &nb= sp; \"fieldPath= \": \"$.event.measurementsForVfScalingFields.additionalMeasurements[*].arra= yOfFields[0].value\", &nb= sp; \"threshold= Value\": 0, &nb= sp; \"direction= \": \"GREATER\", &nb= sp; \"severity\= ": \"CRITICAL\", &nb= sp; \"closedLoo= pEventStatus\": \"ONSET\" &nb= sp; }] }] }" }
Note: Dmaap configuration are speci= fied on this file on publisherHostName and subscriberHostName. To be change= d as required**
6 a) To enable AAI Enrichment in TC= A for CL event, following updates should be done to TCA app preference file= before proceeding
"enableAAIEnric= hment" : true
"aaiEnrichmentH= ost" : <should be set to aai1_ip_addr>
"aaiEnrich= mentIgnoreSSLCertificateErrors" : true
7) Copy below script to CDAP server (this = gets latest image from nexus and deploys TCA application) and execute it
#!/bin/sh TCA_JAR=3Ddcae-analytics-cdap-tca-2.0.0.jar rm -f /home/ubuntu/$TCA_JAR cd /home/ubuntu/ wget https://nexus3.onap.org/service/local/repositories/releases/content/or= g/onap/dcaegen2/analytics/tca/dcae-analytics-cdap-tca/2.0.0/$TCA_JAR if [ $? -eq 0 ]; then if [ -f /home/ubuntu/$TCA_JAR ];= then &nb= sp; echo "Restarting TCA CDAP application using $TCA_JAR artifa= ct" else &nb= sp; echo "ERROR: $TCA_JAR missing" &nb= sp; exit 1 fi else echo "ERROR: $TCA_JAR not found = in nexus" exit 1 fi # stop programs curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-= tca/workers/TCADMaaPMRPublisherWorker/stop curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-= tca/workers/TCADMaaPMRSubscriberWorker/stop curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-= tca/flows/TCAVESCollectorFlow/stop # delete application curl -X DELETE http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dca= e-tca # delete artifact curl -X DELETE http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/artifact= s/dcae-analytics-cdap-tca/versions/2.0.1 # load artifact curl -X POST --data-binary @/home/ubuntu/$TCA_JAR http://localhost:11015/v3= /namespaces/cdap_tca_hi_lo/artifacts/dcae-analytics-cdap-tca # create app curl -X PUT -d @/home/ubuntu/tca_app_config.json http://localhost:11015/v3/= namespaces/cdap_tca_hi_lo/apps/dcae-tca # load preferences curl -X PUT -d @/home/ubuntu/tca_app_preferences.json http://localhost:1101= 5/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/preferences # start programs curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-= tca/workers/TCADMaaPMRPublisherWorker/start curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-= tca/workers/TCADMaaPMRSubscriberWorker/start curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-= tca/flows/TCAVESCollectorFlow/start echo # get status of programs curl http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/work= ers/TCADMaaPMRPublisherWorker/status curl http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/work= ers/TCADMaaPMRSubscriberWorker/status curl http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/flow= s/TCAVESCollectorFlow/status echo
Note: Attached the script also for reference : reload-tca.sh
8) Verify TCA application and logs via CDA= P GUI processes
http://<hostname= /ip>:11011/oldcdap/ns/cdap_tca_hi_lo/apps/dcae-tca/overview/programs=
The overall flow can be checked here
http://<hostname/ip>:11011/oldcda= p/ns/cdap_tca_hi_lo/apps/dcae-tca/programs/flows/TCAVESCollectorFlow/runs= pre>TCA Configuration Change
Typical configuration changes include changing DMAAP host and/or Policy = configuration. If necessary, modify the file on step #6 and run the script = noted as step #7 to redeploy TCA with updated configuration.
VM Init
To address windriver server in-stability, the below init.sh script was used to restart the container on VM restart. This scr= ipt was invoked via VM init script (rc.d).
init.sh#!/bin/sh #docker run -d --name cdap-sdk -p 11011:11011 -p 11015:11015 caskdata/cdap-= standalone:4.1.2 sudo docker restart cdap-sdk-2 sleep 30 # start program curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-= tca/workers/TCADMaaPMRPublisherWorker/start curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-= tca/workers/TCADMaaPMRSubscriberWorker/start curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-= tca/flows/TCAVESCollectorFlow/start
This script was invoked via VM init script (rc.d).
=09ln -s /home/ubuntu/init.sh /etc/init.d/init.sh=09sudo update-rc.d init.sh start 2