This document highlights design considered for to Provision authenticated, authorized DMaaP topics on Message Router and feeds on Data Router .
Table of Contents |
---|
Overview
Bus Controller is a RESTful web service used to provision DMaaP topics (on Message Router) and feeds (on Data Router), with associated authorization (on AAF).
...
- Authorized topic on MR, and to create and grant permission for Publishers and Subscribers.
- Feed on Data Router, with associated user authentication.
- Bus Controller API link : https://docs.onap.org/projects/onap-dmaap-buscontroller/en/latest/apis/api.html
Gerrit review links
Ticket No. | Gerrit link | Comments | 5gbulkpm Test Result | Status |
---|---|---|---|---|
DMAAP-1561 | https://gerrit.onap.org/r/c/dmaap/buscontroller/+/121543 | Updated dmaap-dbc script | Pass | Merged |
DCAEGEN2-2715 (DCAE Data File Collector) | https://gerrit.onap.org/r/c/oom/+/120492 | To be tested with 5gBulkpm gaiting testcase | Pass | Merged |
DCAEGEN2-2714 (DCAE PM-Mapper) | https://gerrit.onap.org/r/c/oom/+/120213 | To be tested with 5gBulkpm gaiting testcase | Pass | Merged |
Dmaap dbc-client docker Image (onap/dmaap/dbc-client:
...
2.0.
...
7)
This dbc-client image is utility (containing shell script) by making use of which we can initiate HTTP REST Request towards Dmaap Bus Controller app for creating dynamic Feeds, Topics.
Design details
- Existing design consist fo kubernetes Job Manifest which consist of onap/dmaap/dbc-client docker image for making http request toward dmaap-bc (Dmaap Bus Controller) pod.
- In DCAE Design-1 highlighted in below pic is considered where we need to create dmmap provisionning Init-Container which will be making use of same onap/dmaap/dbc-client docker image to make Http Rest Request towards dmap-db pod.
- Once the http request is made response will be written back to shared EmptyDir voulme.
- Second Init-container (Update Config) will be reading the response from share EmptyDir voulme and will merge it with application config.
Draw.io design-1 :
- Dmaap Provisioning init-container sends POST request to Dmaap Bus-Controller Pod for creation of Topics, Feeds.
- DCAE Merge init config container merges dcae application config and response received back from dmaap provisioning init-container.
draw.io Diagram | ||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
Data Router Feed, Publisher, Subscriber creation input details.
Old approach-
------------------------------------------
feedConfig:
owner: dcaecm
feedVersion: 0.0
feedName: bulk_pm_feed
asprClassification: unclassified
feedDescription: DFC Feed Creation
pubs:
- username: dcaepub
userpwd: tpJN3gjaAzPAiAa4
dcaeLocationName: loc00
------------------------------------------
New approach-
------------------------------------------
# DataRouter Feed Configuration
drFeedConfig:
- feedName: bulk_pm_feed
owner: dcaecm
feedVersion: 0.0
asprClassification: unclassified
feedDescription: DFC Feed Creation
# DataRouter Publisher Configuration
drPubConfig:
- feedName: bulk_pm_feed
dcaeLocationName: loc00
# DataRouter Subscriber Configuration
drSubConfig:
- feedName: bulk_pm_feed
decompress: true
username: ${DR_USERNAME}
userpwd: ${DR_PASSWORD}
dcaeLocationName: loc00
privilegedSubscriber: true
deliveryURL: https://dcae-pm-mapper:8443/delivery
# MessageRouter Topic, Publisher Configuration
mrTopicsConfig:
- topicName: PERFORMANCE_MEASUREMENTS
topicDescription: PM Mapper publishes perf3gpp VES PM Events to authenticated MR topic
owner: dcaecm
tnxEnabled: false
clients:
- dcaeLocationName: san-francisco
clientRole: org.onap.dcae.pmPublisher
action:
- pub
- view
------------------------------------------
ConfigMap Configuration for Feed, Dr_Publisher
------------------------------------------
volumes:
- name: feeds-config
path: /opt/app/config/feeds/
- name: drpub-config
path: /opt/app/config/dr_pubs/
- name: drsub-config
path: /opt/app/config/dr_subs/
- name: topics-config
path: /opt/app/config/topics |
Code snip for updating application config with dmaap DR Feed/Pub/Sub and MR Topics details.
Code Block | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||
- name: {{ include "common.name" $dot }}-init-merge-config
image: {{ include "repositoryGenerator.image.envsubst" $dot }}
imagePullPolicy: {{ $dot.Values.global.pullPolicy | default $dot.Values.pullPolicy }}
command:
- /bin/sh
args:
- -c
- |
if [ -d /opt/app/config/cache ]; then
cd /opt/app/config/cache
for file in $(ls feed*); do
NUM=$(echo "$file" | sed 's/feedConfig-\([0-9]\+\)-resp.json/\1/')
export DR_LOG_URL_"$NUM"="$(grep -o '"logURL":"[^"]*' "$file" | cut -d '"' -f4)"
export DR_FILES_PUBLISHER_URL_"$NUM"="$(grep -o '"publishURL":"[^"]*' "$file" | cut -d '"' -f4)"
done
for file in $(ls drpub*); do
NUM=$(echo "$file" | sed 's/drpubConfig-\([0-9]\+\)-resp.json/\1/')
export DR_USERNAME_"$NUM"="$(grep -o '"username":"[^"]*' "$file" | cut -d '"' -f4)"
export DR_PASSWORD_"$NUM"="$(grep -o '"userpwd":"[^"]*' "$file" | cut -d '"' -f4)"
export DR_FILES_PUBLISHER_ID_"$NUM"="$(grep -o '"pubId":"[^"]*' "$file" | cut -d '"' -f4)"
done
for file in $(ls drsub*); do
NUM=$(echo "$file" | sed 's/drsubConfig-\([0-9]\+\)-resp.json/\1/')
export DR_FILES_SUBSCRIBER_ID_"$NUM"="$(grep -o '"subId":"[^"]*' "$file" | cut -d '"' -f4)"
done
for file in $(ls topics*); do
NUM=$(echo "$file" | sed 's/topicsConfig-\([0-9]\+\)-resp.json/\1/')
export MR_FILES_PUBLISHER_CLIENT_ID_"$NUM"="$(grep -o '"mrClientId":"[^"]*' "$file" | cut -d '"' -f4)"
done
else
echo "No Response logged for Dmaap BusController Http POST Request..!"
fi
cd /config-input && for PFILE in `ls -1`; do envsubst <${PFILE} >/config/${PFILE}; done
env:
{{- range $cred := $dot.Values.credentials }}
- name: {{ $cred.name }}
{{- include "common.secret.envFromSecretFast" (dict "global" $dot "uid" $cred.uid "key" $cred.key) | nindent 4 }}
{{- end }}
volumeMounts:
- mountPath: /opt/app/config/cache
name: dbc-response-cache
- mountPath: /config-input
name: app-config-input
- mountPath: /config
name: app-config |
Container Logs
1. Log snippet collected for dmaap-bc-dmaap-provisioning conatiner making sure it work with updated changes in dbc script.
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
$ kubectl logs -f -n onap dev-dmaap-bc-dmaap-provisioning-n9zfp
+ export 'PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin'
+ RESP_CACHE=
+ RESP=/dev/null
+ APP_ROOT=/opt/app/dbc-client
+ CONFIGMAP_ROOT=/opt/app/config
+ PORT=8443
+ DBC=dmaap-bc
+ PROTO=https
+ PARAM='useExisting=true'
+ REQUESTID=dmaap-bc-dmaap-provisioning
+ URL=https://dmaap-bc:8443/webapi/
+ CA_PEM=ca.pem
+ KEY_PEM=key.pem
+ CLIENT_PEM=client.pem
+ PEM_DIR=/opt/app/osaaf/local
+ CERT_PWD='2U[iOZzMHI:.#tdCwlBqc;}S'
+ BA_PWD='demo123456!'
+ AUTH_METHOD=basicAuth
+ BA_IDENTITY=dmaap-bc@dmaap-bc.onap.org
+ init_config
+ '[' '!' -d /opt/app/dbc-client -a '!' -d /opt/app/config ]
+ cd /opt/app/osaaf/local
+ '[' basicAuth '=' basicAuth ]
+ echo '-u dmaap-bc@dmaap-bc.onap.org:demo123456!'
+ CURL_CRED='-K /opt/app/osaaf/local/curl.cred'
+ init_dbc_provisioning
+ cd /opt/app/config
+ '[' -d dmaap ]
+ ls dmaap/onap.json
+ do_http_post dmaap/onap.json dmaap
+ RETRY_TIME=60
+ '[' -n ]
+ true
+ '[' dmaap '!=' feeds -a dmaap '!=' topics ]
+ cat dmaap/onap.json
+ envsubst
+ json_input='{
"dmaapName": "mr",
"drProvUrl": "https://dmaap-dr-prov",
"version": "1",
"topicNsRoot": "org.onap.dmaap",
"bridgeAdminTopic": "DCAE_MM_AGENT"
}'
+ xcurl -o /dev/null -d '{
"dmaapName": "mr",
"drProvUrl": "https://dmaap-dr-prov",
"version": "1",
"topicNsRoot": "org.onap.dmaap",
"bridgeAdminTopic": "DCAE_MM_AGENT"
}' https://dmaap-bc:8443/webapi/dmaap
+ curl -X POST -s '-K /opt/app/osaaf/local/curl.cred' -w '%{http_code}' -H 'X-ECOMP-RequestID: dmaap-bc-dmaap-provisioning' -H 'Content-Type: application/json' -o /dev/null -d '{
"dmaapName": "mr",
"drProvUrl": "https://dmaap-dr-prov",
"version": "1",
"topicNsRoot": "org.onap.dmaap",
"bridgeAdminTopic": "DCAE_MM_AGENT"
}' https://dmaap-bc:8443/webapi/dmaap
Http Post request is successful with response code=200
+ rc=200
+ '[' 200 '=' 200 -o 200 '=' 201 -o 200 '=' 409 ]
+ echo 'Http Post request is successful with response code=200'
+ break
+ '[' -d dcaeLocations ]
+ ls dcaeLocations/san-francisco.json
+ do_http_post dcaeLocations/san-francisco.json dcaeLocations
+ RETRY_TIME=60
+ '[' -n ]
+ true
+ '[' dcaeLocations '!=' feeds -a dcaeLocations '!=' topics ]
+ cat dcaeLocations/san-francisco.json
+ envsubst
+ json_input='{
"dcaeLayer": "kubernetes-central",
"dcaeLocationName": "san-francisco"
}'
+ xcurl -o /dev/null -d '{
"dcaeLayer": "kubernetes-central",
"dcaeLocationName": "san-francisco"
}' https://dmaap-bc:8443/webapi/dcaeLocations
+ curl -X POST -s '-K /opt/app/osaaf/local/curl.cred' -w '%{http_code}' -H 'X-ECOMP-RequestID: dmaap-bc-dmaap-provisioning' -H 'Content-Type: application/json' -o /dev/null -d '{
"dcaeLayer": "kubernetes-central",
"dcaeLocationName": "san-francisco"
}' https://dmaap-bc:8443/webapi/dcaeLocations
+ rc=201
+ '[' 201 '=' 200 -o 201 '=' 201 -o 201 '=' 409 ]
+ echo 'Http Post request is successful with response code=201'
+ break
+ '[' -d mr_clusters ]
Http Post request is successful with response code=201
+ ls mr_clusters/san-francisco.json
+ do_http_post mr_clusters/san-francisco.json mr_clusters
+ RETRY_TIME=60
+ '[' -n ]
+ true
+ '[' mr_clusters '!=' feeds -a mr_clusters '!=' topics ]
+ cat mr_clusters/san-francisco.json
+ envsubst
+ json_input='{
"dcaeLocationName": "san-francisco",
"fqdn": "message-router",
"topicProtocol": "http",
"topicPort": "3904"
}'
+ xcurl -o /dev/null -d '{
"dcaeLocationName": "san-francisco",
"fqdn": "message-router",
"topicProtocol": "http",
"topicPort": "3904"
}' https://dmaap-bc:8443/webapi/mr_clusters
+ curl -X POST -s '-K /opt/app/osaaf/local/curl.cred' -w '%{http_code}' -H 'X-ECOMP-RequestID: dmaap-bc-dmaap-provisioning' -H 'Content-Type: application/json' -o /dev/null -d '{
"dcaeLocationName": "san-francisco",
"fqdn": "message-router",
"topicProtocol": "http",
"topicPort": "3904"
}' https://dmaap-bc:8443/webapi/mr_clusters
+ rc=201
+ '[' 201 '=' 200 -o 201 '=' 201 -o 201 '=' 409 ]
+ echo 'Http Post request is successful with response code=201'
+ break
+ '[' -d topics ]
Http Post request is successful with response code=201
+ ls topics/PNF_READY.json topics/PNF_REGISTRATION.json topics/mirrormakeragent.json
+ do_http_post topics/PNF_READY.json topics
+ RETRY_TIME=60
+ '[' -n ]
+ true
+ '[' topics '!=' feeds -a topics '!=' topics ]
+ xcurl -o /dev/null -d @topics/PNF_READY.json 'https://dmaap-bc:8443/webapi/topics/?useExisting=true'
+ curl -X POST -s '-K /opt/app/osaaf/local/curl.cred' -w '%{http_code}' -H 'X-ECOMP-RequestID: dmaap-bc-dmaap-provisioning' -H 'Content-Type: application/json' -o /dev/null -d @topics/PNF_READY.json 'https://dmaap-bc:8443/webapi/topics/?useExisting=true'
Http Post request for feed creation is successful with response code=201
+ rc=201
+ '[' 201 '=' 200 -o 201 '=' 201 -o 201 '=' 409 ]
+ echo 'Http Post request for feed creation is successful with response code=201'
+ break
+ do_http_post topics/PNF_REGISTRATION.json topics
+ RETRY_TIME=60
+ '[' -n ]
+ true
+ '[' topics '!=' feeds -a topics '!=' topics ]
+ xcurl -o /dev/null -d @topics/PNF_REGISTRATION.json 'https://dmaap-bc:8443/webapi/topics/?useExisting=true'
+ curl -X POST -s '-K /opt/app/osaaf/local/curl.cred' -w '%{http_code}' -H 'X-ECOMP-RequestID: dmaap-bc-dmaap-provisioning' -H 'Content-Type: application/json' -o /dev/null -d @topics/PNF_REGISTRATION.json 'https://dmaap-bc:8443/webapi/topics/?useExisting=true'
+ rc=201
+ '[' 201 '=' 200 -o 201 '=' 201 -o 201 '=' 409 ]
+ echo 'Http Post request for feed creation is successful with response code=201'
+ break
Http Post request for feed creation is successful with response code=201
+ do_http_post topics/mirrormakeragent.json topics
+ RETRY_TIME=60
+ '[' -n ]
+ true
+ '[' topics '!=' feeds -a topics '!=' topics ]
+ xcurl -o /dev/null -d @topics/mirrormakeragent.json 'https://dmaap-bc:8443/webapi/topics/?useExisting=true'
+ curl -X POST -s '-K /opt/app/osaaf/local/curl.cred' -w '%{http_code}' -H 'X-ECOMP-RequestID: dmaap-bc-dmaap-provisioning' -H 'Content-Type: application/json' -o /dev/null -d @topics/mirrormakeragent.json 'https://dmaap-bc:8443/webapi/topics/?useExisting=true'
Http Post request for feed creation is successful with response code=201
+ rc=201
+ '[' 201 '=' 200 -o 201 '=' 201 -o 201 '=' 409 ]
+ echo 'Http Post request for feed creation is successful with response code=201'
+ break
+ '[' -d mr_clients ]
+ '[' -d dr_nodes ]
+ ls 'dr_nodes/*.json'
ls: dr_nodes/*.json: No such file or directory
+ '[' -d feeds ]
+ ls 'feeds/*.json'
ls: feeds/*.json: No such file or directory
+ '[' -d dr_pubs ]
+ '[' -d dr_subs ] |
2. Log snippet from dcae-datafile-collector initContainer dcae-datafile-collector-init-dmaap-provisioning for creation of Feeds and Dr Publisher.
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
$ kubectl logs -f -n onap dev-dcae-datafile-collector-d996487db-6wdjb -c dcae-datafile-collector-init-dmaap-provisioning
+ export 'PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin'
+ RESP_CACHE=/opt/app/config/cache
+ RESP=/dev/null
+ APP_ROOT=/opt/app/dbc-client
+ CONFIGMAP_ROOT=/opt/app/config
+ PORT=8443
+ DBC=dmaap-bc
+ PROTO=https
+ PARAM='useExisting=true'
+ REQUESTID=dcae-datafile-collector-dmaap-provisioning
+ URL=https://dmaap-bc:8443/webapi/
+ CA_PEM=ca.pem
+ KEY_PEM=key.pem
+ CLIENT_PEM=client.pem
+ PEM_DIR=/opt/app/osaaf/local
+ CERT_PWD='2U[iOZzMHI:.#tdCwlBqc;}S'
+ BA_PWD='demo123456!'
+ AUTH_METHOD=basicAuth
+ BA_IDENTITY=dmaap-bc@dmaap-bc.onap.org
+ init_config
+ '[' '!' -d /opt/app/dbc-client -a '!' -d /opt/app/config ]
+ cd /opt/app/osaaf/local
+ '[' basicAuth '=' basicAuth ]
+ echo '-u dmaap-bc@dmaap-bc.onap.org:demo123456!'
+ CURL_CRED='-K /opt/app/osaaf/local/curl.cred'
+ init_dbc_provisioning
+ cd /opt/app/config
+ '[' -d dmaap ]
+ '[' -d dcaeLocations ]
+ '[' -d mr_clusters ]
+ '[' -d topics ]
+ '[' -d mr_clients ]
+ '[' -d dr_nodes ]
+ '[' -d feeds ]
+ ls feeds/feedConfig-0.json
+ do_http_post feeds/feedConfig-0.json feeds
+ RETRY_TIME=60
+ '[' -n /opt/app/config/cache ]
+ cut -d . -f1
+ echo feedConfig-0.json
+ RESP=/opt/app/config/cache/feedConfig-0-resp.json
+ true
+ '[' feeds '!=' feeds -a feeds '!=' topics ]
+ xcurl -o /opt/app/config/cache/feedConfig-0-resp.json -d @feeds/feedConfig-0.json 'https://dmaap-bc:8443/webapi/feeds/?useExisting=true'
+ curl -X POST -s '-K /opt/app/osaaf/local/curl.cred' -w '%{http_code}' -H 'X-ECOMP-RequestID: dcae-datafile-collector-dmaap-provisioning' -H 'Content-Type: application/json' -o /opt/app/config/cache/feedConfig-0-resp.json -d @feeds/feedConfig-0.json 'https://dmaap-bc:8443/webapi/feeds/?useExisting=true'
Http Post request for feed creation is successful with response code=200
+ rc=200
+ '[' 200 '=' 200 -o 200 '=' 201 -o 200 '=' 409 ]
+ echo 'Http Post request for feed creation is successful with response code=200'
+ break
+ '[' -d dr_pubs ]
+ ls dr_pubs/drpubConfig-0.json
+ do_http_post dr_pubs/drpubConfig-0.json dr_pubs
+ RETRY_TIME=60
+ '[' -n /opt/app/config/cache ]
+ echo drpubConfig-0.json
+ cut -d . -f1
+ RESP=/opt/app/config/cache/drpubConfig-0-resp.json
+ true
+ '[' dr_pubs '!=' feeds -a dr_pubs '!=' topics ]
+ cat dr_pubs/drpubConfig-0.json
+ envsubst
+ json_input='{"dcaeLocationName":"loc00","feedName":"bulk_pm_feed"}'
+ xcurl -o /opt/app/config/cache/drpubConfig-0-resp.json -d '{"dcaeLocationName":"loc00","feedName":"bulk_pm_feed"}' https://dmaap-bc:8443/webapi/dr_pubs
+ curl -X POST -s '-K /opt/app/osaaf/local/curl.cred' -w '%{http_code}' -H 'X-ECOMP-RequestID: dcae-datafile-collector-dmaap-provisioning' -H 'Content-Type: application/json' -o /opt/app/config/cache/drpubConfig-0-resp.json -d '{"dcaeLocationName":"loc00","feedName":"bulk_pm_feed"}' https://dmaap-bc:8443/webapi/dr_pubs
+ rc=201
+ '[' 201 '=' 200 -o 201 '=' 201 -o 201 '=' 409 ]
+ echo 'Http Post request is successful with response code=201'
+ break
+ '[' -d dr_subs ]
Http Post request is successful with response code=201
|
3. Log snippet from dcae-pm-mapper initContainer dcae-pm-mapper-init-dmaap-provisioning for creation of Feeds and Dr Subscriber and MessageRouter Topics.
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
$ kubectl logs -f -n onap dev-dcae-pm-mapper-6d84cb4757-mljn9 -c dcae-pm-mapper-init-dmaap-provisioning
+ export 'PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin'
+ RESP_CACHE=/opt/app/config/cache
+ RESP=/dev/null
+ APP_ROOT=/opt/app/dbc-client
+ CONFIGMAP_ROOT=/opt/app/config
+ PORT=8443
+ DBC=dmaap-bc
+ PROTO=https
+ PARAM='useExisting=true'
+ REQUESTID=dcae-pm-mapper-dmaap-provisioning
+ URL=https://dmaap-bc:8443/webapi/
+ CA_PEM=ca.pem
+ KEY_PEM=key.pem
+ CLIENT_PEM=client.pem
+ PEM_DIR=/opt/app/osaaf/local
+ CERT_PWD='2U[iOZzMHI:.#tdCwlBqc;}S'
+ BA_PWD='demo123456!'
+ AUTH_METHOD=basicAuth
+ BA_IDENTITY=dmaap-bc@dmaap-bc.onap.org
+ init_config
+ '[' '!' -d /opt/app/dbc-client -a '!' -d /opt/app/config ]
+ cd /opt/app/osaaf/local
+ '[' basicAuth '=' basicAuth ]
+ echo '-u dmaap-bc@dmaap-bc.onap.org:demo123456!'
+ CURL_CRED='-K /opt/app/osaaf/local/curl.cred'
+ init_dbc_provisioning
+ cd /opt/app/config
+ '[' -d dmaap ]
+ '[' -d dcaeLocations ]
+ '[' -d mr_clusters ]
+ '[' -d topics ]
+ ls topics/topicsConfig-0.json
+ do_http_post topics/topicsConfig-0.json topics
+ RETRY_TIME=60
+ '[' -n /opt/app/config/cache ]
+ echo+ topicsConfig-0.json
cut -d . -f1
+ RESP=/opt/app/config/cache/topicsConfig-0-resp.json
+ true
+ '[' topics '!=' feeds -a topics '!=' topics ]
+ xcurl -o /opt/app/config/cache/topicsConfig-0-resp.json -d @topics/topicsConfig-0.json 'https://dmaap-bc:8443/webapi/topics/?useExisting=true'
+ curl -X POST -s '-K /opt/app/osaaf/local/curl.cred' -w '%{http_code}' -H 'X-ECOMP-RequestID: dcae-pm-mapper-dmaap-provisioning' -H 'Content-Type: application/json' -o /opt/app/config/cache/topicsConfig-0-resp.json -d @topics/topicsConfig-0.json 'https://dmaap-bc:8443/webapi/topics/?useExisting=true'
+ rc=201
+ '[' 201 '=' 200 -o 201 '=' 201 -o 201 '=' 409 ]
+ echo 'Http Post request for feed creation is successful with response code=201'
+ break
+ '[' -d mr_clients ]
+ '[' -d dr_nodes ]
+ '[' -d feeds ]
Http Post request for feed creation is successful with response code=201
+ ls feeds/feedConfig-0.json
+ do_http_post feeds/feedConfig-0.json feeds
+ RETRY_TIME=60
+ '[' -n /opt/app/config/cache ]
+ echo feedConfig-0.json
+ cut -d . -f1
+ RESP=/opt/app/config/cache/feedConfig-0-resp.json
+ true
+ '[' feeds '!=' feeds -a feeds '!=' topics ]
+ xcurl -o /opt/app/config/cache/feedConfig-0-resp.json -d @feeds/feedConfig-0.json 'https://dmaap-bc:8443/webapi/feeds/?useExisting=true'
+ curl -X POST -s '-K /opt/app/osaaf/local/curl.cred' -w '%{http_code}' -H 'X-ECOMP-RequestID: dcae-pm-mapper-dmaap-provisioning' -H 'Content-Type: application/json' -o /opt/app/config/cache/feedConfig-0-resp.json -d @feeds/feedConfig-0.json 'https://dmaap-bc:8443/webapi/feeds/?useExisting=true'
+ rc=200
+ '[' 200 '=' 200 -o 200 '=' 201 -o 200 '=' 409 ]
+ echo 'Http Post request for feed creation is successful with response code=200'
+ break
+ '[' -d dr_pubs ]
+ '[' -d dr_subs ]
Http Post request for feed creation is successful with response code=200
+ ls dr_subs/drsubConfig-0.json
+ do_http_post dr_subs/drsubConfig-0.json dr_subs
+ RETRY_TIME=60
+ '[' -n /opt/app/config/cache ]
+ echo drsubConfig-0.json
+ cut -d . -f1
+ RESP=/opt/app/config/cache/drsubConfig-0-resp.json
+ true
+ '[' dr_subs '!=' feeds -a dr_subs '!=' topics ]
+ cat dr_subs/drsubConfig-0.json
+ envsubst
+ json_input='{"dcaeLocationName":"loc00","decompress":true,"deliveryURL":"https://dcae-pm-mapper:8443/delivery","feedName":"bulk_pm_feed","privilegedSubscriber":true,"username":"username","userpwd":"password"}'
+ xcurl -o /opt/app/config/cache/drsubConfig-0-resp.json -d '{"dcaeLocationName":"loc00","decompress":true,"deliveryURL":"https://dcae-pm-mapper:8443/delivery","feedName":"bulk_pm_feed","privilegedSubscriber":true,"username":"username","userpwd":"password"}' https://dmaap-bc:8443/webapi/dr_subs
+ curl -X POST -s '-K /opt/app/osaaf/local/curl.cred' -w '%{http_code}' -H 'X-ECOMP-RequestID: dcae-pm-mapper-dmaap-provisioning' -H 'Content-Type: application/json' -o /opt/app/config/cache/drsubConfig-0-resp.json -d '{"dcaeLocationName":"loc00","decompress":true,"deliveryURL":"https://dcae-pm-mapper:8443/delivery","feedName":"bulk_pm_feed","privilegedSubscriber":true,"username":"username","userpwd":"password"}' https://dmaap-bc:8443/webapi/dr_subs
Http Post request is successful with response code=201
+ rc=201
+ '[' 201 '=' 200 -o 201 '=' 201 -o 201 '=' 409 ]
+ echo 'Http Post request is successful with response code=201'
+ break |
4. GET Request for DR Feeds, MR Topics from Bus Controller API.
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
1. >> GET : https://172.16.0.70:30242/webapi/feeds/
[
{
"lastMod": 1625906852119,
"status": "VALID",
"feedId": "1",
"feedName": "bulk_pm_feed",
"feedVersion": "0",
"feedDescription": "DFC Feed Creation",
"owner": "dcaecm",
"asprClassification": "unclassified",
"publishURL": "https://dmaap-dr-prov/publish/1",
"subscribeURL": "https://dmaap-dr-prov/subscribe/1",
"suspended": false,
"logURL": "https://dmaap-dr-prov/feedlog/1",
"formatUuid": null,
"pubs": [
{
"lastMod": 1625906852105,
"status": "VALID",
"dcaeLocationName": "loc00",
"username": "tmp_edftprplxki85pw",
"userpwd": "gi6nvqz4skzepa1",
"feedId": "1",
"pubId": "1.da72f",
"feedName": null,
"feedVersion": null,
"statusValid": true
},
{
"lastMod": 1625906851630,
"status": "VALID",
"dcaeLocationName": "san-francisco",
"username": "tmp_rmu408ke6c47cjd",
"userpwd": "3ae2ahymjl6sndw",
"feedId": "1",
"pubId": "1.9hl81",
"feedName": null,
"feedVersion": null,
"statusValid": true
}
],
"subs": [
{
"lastMod": 1625906974644,
"status": "VALID",
"dcaeLocationName": "loc00",
"username": "username",
"userpwd": "password",
"feedId": "1",
"deliveryURL": "https://dcae-pm-mapper:8443/delivery",
"logURL": "https://dmaap-dr-prov/sublog/1",
"subId": "1",
"use100": false,
"suspended": false,
"owner": "DGL",
"guaranteedDelivery": false,
"guaranteedSequence": false,
"privilegedSubscriber": true,
"decompress": true,
"feedName": null,
"feedVersion": null,
"statusValid": true
}
],
"bytes": "eyJuYW1lIjogImJ1bGtfcG1fZmVlZCIsICJ2ZXJzaW9uIjogIjAiLCAiZGVzY3JpcHRpb24iOiAiREZDIEZlZWQgQ3JlYXRpb24iLCAic3VzcGVuZCI6IGZhbHNlLCAiYXV0aG9yaXphdGlvbiI6IHsgImNsYXNzaWZpY2F0aW9uIjogInVuY2xhc3NpZmllZCIsICJlbmRwb2ludF9hZGRycyI6IFtdLCJlbmRwb2ludF9pZHMiOiBbCXsiaWQiOiAidG1wX2VkZnRwcnBseGtpODVwdyIsInBhc3N3b3JkIjogImdpNm52cXo0c2t6ZXBhMSJ9CSx7ImlkIjogInRtcF9ybXU0MDhrZTZjNDdjamQiLCJwYXNzd29yZCI6ICIzYWUyYWh5bWpsNnNuZHcifV19fQ==",
"statusValid": true
}
]
2. >> GET Topics : https://172.16.0.70:30242/webapi/topics/
[
{
"lastMod": 1625906819480,
"status": "VALID",
"fqtn": "org.onap.dmaap.mr.PNF_REGISTRATION",
"topicName": "PNF_REGISTRATION",
"topicDescription": "the VES collector will be publishing pnfRegistration events in this topic",
"tnxEnabled": "false",
"owner": "VEScollector",
"formatUuid": null,
"replicationCase": "REPLICATION_NONE",
"globalMrURL": null,
"fqtnStyle": null,
"version": null,
"partitionCount": "2",
"replicationCount": "1",
"publisherRole": "org.onap.dmaap.mr.PNF_REGISTRATION.publisher",
"subscriberRole": "org.onap.dmaap.mr.PNF_REGISTRATION.subscriber",
"clients": [
{
"lastMod": 1625906820030,
"status": "VALID",
"dcaeLocationName": "san-francisco",
"topicURL": "http://message-router:3904/events/org.onap.dmaap.mr.PNF_REGISTRATION",
"fqtn": "org.onap.dmaap.mr.PNF_REGISTRATION",
"clientRole": "org.onap.dmaap.mr.PNF_REGISTRATION.sub",
"action": [
"sub",
"view"
],
"mrClientId": "1625906820030",
"clientIdentity": null,
"publisher": false,
"subscriber": true,
"statusValid": true
}
],
"numClients": 1,
"bytes": "eyAidG9waWNOYW1lIjogIm9yZy5vbmFwLmRtYWFwLm1yLlBORl9SRUdJU1RSQVRJT04iLCAidG9waWNEZXNjcmlwdGlvbiI6ICJ0aGUgVkVTIGNvbGxlY3RvciB3aWxsIGJlIHB1Ymxpc2hpbmcgcG5mUmVnaXN0cmF0aW9uIGV2ZW50cyBpbiB0aGlzIHRvcGljIiwgInBhcnRpdGlvbkNvdW50IjogIjIiLCAicmVwbGljYXRpb25Db3VudCI6ICIxIiB9IA==",
"statusValid": true
},
{
"lastMod": 1625906820603,
"status": "VALID",
"fqtn": "org.onap.dmaap.mr.mirrormakeragent",
"topicName": "mirrormakeragent",
"topicDescription": "the topic used to provision the MM agent whitelist",
"tnxEnabled": "false",
"owner": "dmaap",
"formatUuid": null,
"replicationCase": "REPLICATION_NONE",
"globalMrURL": null,
"fqtnStyle": null,
"version": null,
"partitionCount": "1",
"replicationCount": "1",
"publisherRole": "org.onap.dmaap.mr.mirrormakeragent.publisher",
"subscriberRole": "org.onap.dmaap.mr.mirrormakeragent.subscriber",
"clients": [
{
"lastMod": 1625906821157,
"status": "VALID",
"dcaeLocationName": "san-francisco",
"topicURL": "http://message-router:3904/events/org.onap.dmaap.mr.mirrormakeragent",
"fqtn": "org.onap.dmaap.mr.mirrormakeragent",
"clientRole": null,
"action": [
"pub",
"sub",
"view"
],
"mrClientId": "1625906821157",
"clientIdentity": "dmaap-bc-mm-prov@dmaap-bc-mm-prov.onap.org",
"publisher": true,
"subscriber": true,
"statusValid": true
},
{
"lastMod": 1625906827509,
"status": "VALID",
"dcaeLocationName": "san-francisco",
"topicURL": "http://message-router:3904/events/org.onap.dmaap.mr.mirrormakeragent",
"fqtn": "org.onap.dmaap.mr.mirrormakeragent",
"clientRole": null,
"action": [
"pub",
"sub",
"view"
],
"mrClientId": "1625906827509",
"clientIdentity": "demo@people.osaaf.org",
"publisher": true,
"subscriber": true,
"statusValid": true
},
{
"lastMod": 1625906826491,
"status": "VALID",
"dcaeLocationName": "san-francisco",
"topicURL": "http://message-router:3904/events/org.onap.dmaap.mr.mirrormakeragent",
"fqtn": "org.onap.dmaap.mr.mirrormakeragent",
"clientRole": null,
"action": [
"pub",
"sub",
"view"
],
"mrClientId": "1625906826491",
"clientIdentity": "dmaap-bc-topic-mgr@dmaap-bc-topic-mgr.onap.org",
"publisher": true,
"subscriber": true,
"statusValid": true
}
],
"numClients": 3,
"bytes": "eyAidG9waWNOYW1lIjogIm9yZy5vbmFwLmRtYWFwLm1yLm1pcnJvcm1ha2VyYWdlbnQiLCAidG9waWNEZXNjcmlwdGlvbiI6ICJ0aGUgdG9waWMgdXNlZCB0byBwcm92aXNpb24gdGhlIE1NIGFnZW50IHdoaXRlbGlzdCIsICJwYXJ0aXRpb25Db3VudCI6ICIxIiwgInJlcGxpY2F0aW9uQ291bnQiOiAiMSIgfSA=",
"statusValid": true
},
{
"lastMod": 1625906971092,
"status": "VALID",
"fqtn": "org.onap.dmaap.mr.PERFORMANCE_MEASUREMENTS",
"topicName": "PERFORMANCE_MEASUREMENTS",
"topicDescription": "PM Mapper publishes perf3gpp VES PM Events to authenticated MR topic",
"tnxEnabled": "false",
"owner": "dcaecm",
"formatUuid": null,
"replicationCase": "REPLICATION_NONE",
"globalMrURL": null,
"fqtnStyle": null,
"version": null,
"partitionCount": "2",
"replicationCount": "1",
"publisherRole": "org.onap.dmaap.mr.PERFORMANCE_MEASUREMENTS.publisher",
"subscriberRole": "org.onap.dmaap.mr.PERFORMANCE_MEASUREMENTS.subscriber",
"clients": [
{
"lastMod": 1625906971956,
"status": "VALID",
"dcaeLocationName": "san-francisco",
"topicURL": "http://message-router:3904/events/org.onap.dmaap.mr.PERFORMANCE_MEASUREMENTS",
"fqtn": "org.onap.dmaap.mr.PERFORMANCE_MEASUREMENTS",
"clientRole": "org.onap.dcae.pmPublisher",
"action": [
"pub",
"view"
],
"mrClientId": "1625906971956",
"clientIdentity": null,
"publisher": true,
"subscriber": false,
"statusValid": true
}
],
"numClients": 1,
"bytes": "eyAidG9waWNOYW1lIjogIm9yZy5vbmFwLmRtYWFwLm1yLlBFUkZPUk1BTkNFX01FQVNVUkVNRU5UUyIsICJ0b3BpY0Rlc2NyaXB0aW9uIjogIlBNIE1hcHBlciBwdWJsaXNoZXMgcGVyZjNncHAgVkVTIFBNIEV2ZW50cyB0byBhdXRoZW50aWNhdGVkIE1SIHRvcGljIiwgInBhcnRpdGlvbkNvdW50IjogIjIiLCAicmVwbGljYXRpb25Db3VudCI6ICIxIiB9IA==",
"statusValid": true
},
{
"lastMod": 1625906816791,
"status": "VALID",
"fqtn": "org.onap.dmaap.mr.PNF_READY",
"topicName": "PNF_READY",
"topicDescription": "This topic will be used to publish the PNF_READY events generated by the PNF REgistration Handler service in the DCAE platform.",
"tnxEnabled": "false",
"owner": "PNFRegistrationHandler",
"formatUuid": null,
"replicationCase": "REPLICATION_NONE",
"globalMrURL": null,
"fqtnStyle": null,
"version": null,
"partitionCount": "2",
"replicationCount": "1",
"publisherRole": "org.onap.dmaap.mr.PNF_READY.publisher",
"subscriberRole": "org.onap.dmaap.mr.PNF_READY.subscriber",
"clients": [
{
"lastMod": 1625906817506,
"status": "VALID",
"dcaeLocationName": "san-francisco",
"topicURL": "http://message-router:3904/events/org.onap.dmaap.mr.PNF_READY",
"fqtn": "org.onap.dmaap.mr.PNF_READY",
"clientRole": "org.onap.dmaap.mr.PNF_READY.pub",
"action": [
"pub",
"view"
],
"mrClientId": "1625906817506",
"clientIdentity": null,
"publisher": true,
"subscriber": false,
"statusValid": true
}
],
"numClients": 1,
"bytes": "eyAidG9waWNOYW1lIjogIm9yZy5vbmFwLmRtYWFwLm1yLlBORl9SRUFEWSIsICJ0b3BpY0Rlc2NyaXB0aW9uIjogIlRoaXMgdG9waWMgd2lsbCBiZSB1c2VkIHRvIHB1Ymxpc2ggdGhlIFBORl9SRUFEWSBldmVudHMgZ2VuZXJhdGVkIGJ5IHRoZSBQTkYgUkVnaXN0cmF0aW9uIEhhbmRsZXIgc2VydmljZSBpbiB0aGUgRENBRSBwbGF0Zm9ybS4iLCAicGFydGl0aW9uQ291bnQiOiAiMiIsICJyZXBsaWNhdGlvbkNvdW50IjogIjEiIH0g",
"statusValid": true
}
] |
Test details
...
- Create Sftp server.
- Upload PM Files to xNF SFTP Server.
- Send File Ready Event to VES Collector.
- Verify Ves Collector logs for Event send to unauthenticated.VES_NOTIFICATION_OUTPUT : Ves Collector log file.
- Verify Data File Collector Logs if it contains message > Publishing file A1625946870.67.xml.gz to DR successful! : DFC log file.
- Vefify PM-Mapper log if it contains > org.onap.dcaegen2.services.pmmapper.messagerouter.VESPublisher. Successfully published VES events to messagerouter : Pm-Mapper log file.
- Verify Message-Router log if it contains Publisher Log Details : [publisherId=dcae, topicId=org.onap.dmaap.mr.PERFORMANCE_MEASUREMENTS, messageTimestamp=null, publisherIp=10.42.6.27, messageBatchId=10072021195549, messageSequence=1, messageLengthInBytes=2530, transactionEnabled=true, transactionId=10-07-2021::07:55:44:917::10.42.6.27::10072021195549::1, publishTimestamp=10-07-2021::07:55:44:918, serverIp=10.42.3.28]
Improvements
- Datafile Collector not sending dr publisher credentails as secret.