DCAE Platform support dynamic topic/feed provisioning through usage for DMaap plugin which interfaces with Dmaap-DBCL component to carry out this request and provision AAF based topic/feed; once the provisioning is completed, the corresponding configuration are returned back for MS (as configuration in consul)
Overview
<Brief write on how plugin is invoked and internal of plugin work flow)
The dmaap-plugin is invoked via k8s plugin or docker plugin. There are several test blueprints that you can run to test the DMaap-plugin in ONAP environment.
Dmaap Plugin Dependencies
<components and configuration required for this plugin in ONAP DCAE eg Consul, Cloudify, Dmaap-DBCL, DMAAP)
Running the dmaap-plugin depends on the setup of Consul, Cloudify, Dmaap-DBCL, and AAF in ONAP environment. In the following discussion we assume that the Consul, Cloudify, DMaap-DBCL and AAF are already set up.
Blueprints
<list all the test blueprint along with small description on what the scope of each blueprints)
All test blueprint should be uploaded into https://git.onap.org/dcaegen2/platform/blueprints/tree/reference_templates
Currently we have the following blueprints:
- test_mr_pubsub_extra.yaml
- test_mr_pubsub.yaml
- test_mr_multipub.yaml
- test_mr_multisub.yaml
- test_dr_pubsub.yaml
- test_dr_mr.yaml
In the following discussion we assume that you already download these test scripts into /blueprints/ directory on the test VM. (We loaded the blueprints manually in our test.)
Test Setup
<env dependencies; and pre-requisite for testing + any data load (e.g location details + consul load)>
Pre-requisite for running the test blueprints:
1) Add the following key/value into consul with "dmaap-plugin" as the key.
Note: This step might not be needed, in particular "protocol" : "http" can cause a problem with dmaap-bc connection. Default configuration will be added into consul upon Cloudify Container deployment.
{
"dmaap" :
{
"username": "whatever",
"password": "whatever",
"owner" : "dcaeorch",
"protocol" : "http"
}
}
2) (With the help of DBCL owner) add location san-francisco into DMaap DBCL if the location is not in there yet.
Deployment Steps
<steps for running blueprint listed earlier >
For each of the test blueprints, run the following commands:
- cfy install -b <blueprint-id> -d <deployment-id> <blueprint_name>
where <blueprint-id> and <deployment-id> can be any string choose by the user.
During deployment, Monitor the output of deployment for error messages.
After the blueprint is successfully deployed, check the DMaap DBCL url to verify the configuration. (Detailed commands are described later.)
After validation, uninstall the blueprint with the following command:
- cfy uninstall <deployment_id>
Recheck the DMaap DBCL url to verify that the previous configuration is deleted from DMaap DBCL.
Notice: since most of the blueprints reuse the same topic name(s) and feed name, please uninstall the current blueprint before testing the next one.
For example, the screen output of deploying/verifying test_mr_pubsub_extra.yaml is as follows:
[root@dev-dcaegen2-dcae-bootstrap-55cb5596b7-phdct blueprints]# cfy install -b test0 -d test0 -i dmaap_inputs.yaml test_mr_pubsub.yaml
Uploading blueprint test_mr_pubsub.yaml...
test_mr_pubsub.yaml |#################################################| 100.0%
Blueprint uploaded. The blueprint's id is test0
Creating new deployment from blueprint test0...
Deployment created. The deployment's id is test0
Executing workflow install on deployment test0 [timeout=900 seconds]
Deployment environment creation is pending...
2019-05-10 23:09:54.252 CFY <test0> Starting 'create_deployment_environment' workflow execution
2019-05-10 23:09:54.881 CFY <test0> Installing deployment plugins
2019-05-10 23:09:54.881 CFY <test0> Sending task 'cloudify_agent.operations.install_plugins'
2019-05-10 23:09:54.881 CFY <test0> Task started 'cloudify_agent.operations.install_plugins'
2019-05-10 23:09:55.568 LOG <test0> INFO: Installing plugin: k8s
2019-05-10 23:09:56.181 LOG <test0> INFO: Using existing installation of managed plugin: 99e0f852-a688-4ecf-b606-37df263f6fa0 [package_name: k8splugin, package_version: 1.4.13, supported_platform: linux_x86_64, distribution: centos, distribution_release: core]
2019-05-10 23:09:56.181 LOG <test0> INFO: Installing plugin: dmaapplugin
2019-05-10 23:09:57.576 LOG <test0> INFO: Using existing installation of managed plugin: d8abab9c-2e52-4caa-a9fe-d74bf24609ad [package_name: dmaap, package_version: 1.3.2, supported_platform: any]
2019-05-10 23:09:57.576 CFY <test0> Task succeeded 'cloudify_agent.operations.install_plugins'
2019-05-10 23:09:57.576 CFY <test0> Skipping starting deployment policy engine core - no policies defined
2019-05-10 23:09:57.576 CFY <test0> Creating deployment work directory
2019-05-10 23:09:57.576 CFY <test0> 'create_deployment_environment' workflow execution succeeded
2019-05-10 23:10:01.752 CFY <test0> Starting 'install' workflow execution
2019-05-10 23:10:03.571 CFY <test0> [topic01_xjti5i] Creating node instance
2019-05-10 23:10:04.268 CFY <test0> [topic00_ehyrmr] Creating node instance
2019-05-10 23:10:04.268 CFY <test0> [topic01_xjti5i.create] Sending task 'dmaapplugin.mr_lifecycle.create_topic'
2019-05-10 23:10:04.268 CFY <test0> [topic00_ehyrmr.create] Sending task 'dmaapplugin.mr_lifecycle.create_topic'
2019-05-10 23:10:05.961 LOG <test0> [topic01_xjti5i.create] INFO: Attempting to create topic name topic01_name
2019-05-10 23:10:06.612 LOG <test0> [topic01_xjti5i.create] INFO: Posting to URL: http://dmaap-bc:8080/webapi/topics with body: {'owner': u'dcaeorch', 'topicDescription': 'No description provided', 'topicName': u'topic01_name', 'txenable': False}
2019-05-10 23:10:06.612 LOG <test0> [topic00_ehyrmr.create] INFO: Attempting to create topic name topic00_name
2019-05-10 23:10:06.612 LOG <test0> [topic00_ehyrmr.create] INFO: Posting to URL: http://dmaap-bc:8080/webapi/topics with body: {'owner': u'dcaeorch', 'topicDescription': 'No description provided', 'topicName': u'topic00_name', 'txenable': False}
2019-05-10 23:10:08.715 CFY <test0> [topic00_ehyrmr.create] Task succeeded 'dmaapplugin.mr_lifecycle.create_topic'
2019-05-10 23:10:09.321 CFY <test0> [topic01_xjti5i.create] Task succeeded 'dmaapplugin.mr_lifecycle.create_topic'
2019-05-10 23:10:09.321 CFY <test0> [topic00_ehyrmr] Node instance created
2019-05-10 23:10:09.321 CFY <test0> [topic01_xjti5i] Node instance created
2019-05-10 23:10:09.960 CFY <test0> [topic00_ehyrmr] Configuring node instance: nothing to do
2019-05-10 23:10:09.960 CFY <test0> [topic01_xjti5i] Configuring node instance: nothing to do
2019-05-10 23:10:09.960 CFY <test0> [topic00_ehyrmr] Starting node instance: nothing to do
2019-05-10 23:10:10.570 CFY <test0> [topic01_xjti5i] Starting node instance: nothing to do
2019-05-10 23:10:10.570 CFY <test0> [topic00_ehyrmr] Node instance started
2019-05-10 23:10:10.570 CFY <test0> [topic01_xjti5i] Node instance started
2019-05-10 23:10:11.241 CFY <test0> [web_server_cqf9i4] Creating node instance
2019-05-10 23:10:11.925 CFY <test0> [web_server_cqf9i4.create] Sending task 'k8splugin.create_for_components_with_streams'
2019-05-10 23:10:16.273 LOG <test0> [web_server_cqf9i4.create] INFO: Added config for s61b2002abe554457a78050174c431719-nginx-web
2019-05-10 23:10:16.784 LOG <test0> [web_server_cqf9i4.create] INFO: Done setting up: s61b2002abe554457a78050174c431719-nginx-web
2019-05-10 23:10:17.527 CFY <test0> [web_server_cqf9i4.create] Task succeeded 'k8splugin.create_for_components_with_streams'
2019-05-10 23:10:18.146 CFY <test0> [web_server_cqf9i4] Node instance created
2019-05-10 23:10:18.146 CFY <test0> [web_server_cqf9i4] Pre-configuring relationships
2019-05-10 23:10:18.146 CFY <test0> [preconfigure] Sending task 'dmaapplugin.mr_relationships.add_mr_publisher'
2019-05-10 23:10:20.266 LOG <test0> [preconfigure] INFO: Attempting to add web_server as publisher to topic org.onap.dmaap.mr.topic00_name
2019-05-10 23:10:20.266 LOG <test0> [preconfigure] INFO: Attempting to add web_server as publisher to topic org.onap.dmaap.mr.topic00_name
2019-05-10 23:10:20.788 LOG <test0> [preconfigure] INFO: Added publisher id 1557529820110 to feed org.onap.dmaap.mr.topic00_name at san-francisco
2019-05-10 23:10:20.788 CFY <test0> [preconfigure] Task succeeded 'dmaapplugin.mr_relationships.add_mr_publisher'
2019-05-10 23:10:21.470 CFY <test0> [preconfigure] Sending task 'dmaapplugin.mr_relationships.add_mr_subscriber'
2019-05-10 23:10:23.467 LOG <test0> [preconfigure] INFO: Attempting to add web_server as subscriber to topic org.onap.dmaap.mr.topic01_name
2019-05-10 23:10:24.004 LOG <test0> [preconfigure] INFO: Posting to URL: http://dmaap-bc:8080/webapi/mr_clients with body: {'action': ['view', 'sub'], 'clientRole': u'org.onap.dcae.pnfSubscriber', 'fqtn': u'org.onap.dmaap.mr.topic01_name', 'dcaeLocationName': u'san-francisco'}
2019-05-10 23:10:24.004 LOG <test0> [preconfigure] INFO: Added subscriber id 1557529823378 to feed org.onap.dmaap.mr.topic01_name at san-francisco
2019-05-10 23:10:24.658 CFY <test0> [preconfigure] Task succeeded 'dmaapplugin.mr_relationships.add_mr_subscriber'
2019-05-10 23:10:24.658 CFY <test0> [web_server_cqf9i4] Relationships pre-configured
2019-05-10 23:10:25.402 CFY <test0> [web_server_cqf9i4] Configuring node instance: nothing to do
2019-05-10 23:10:25.402 CFY <test0> [web_server_cqf9i4] Starting node instance
2019-05-10 23:10:25.998 CFY <test0> [web_server_cqf9i4.start] Sending task 'k8splugin.create_and_start_container_for_components_with_streams'
2019-05-10 23:10:27.038 LOG <test0> [web_server_cqf9i4.start] INFO: Starting k8s deployment for s61b2002abe554457a78050174c431719-nginx-web, image: nginx, env: {'CONSUL_HOST': u'consul-server.onap', 'SERVICE_TAGS': 'test0', 'CONFIG_BINDING_SERVICE': 'config-binding-service'}, kwargs: {'readiness': {u'endpoint': u'/', u'type': u'http'}, 'labels': {'cfydeployment': u'test0', 'cfynodeinstance': u'web_server_cqf9i4', 'cfynode': u'web_server'}, 'tls_info': {}, 'envs': {'SERVICE_TAGS': 'test0'}, 'liveness': {}, 'resource_config': {}, 'volumes': [], 'log_info': {}, 'ports': [u'80:0'], 'k8s_location': u'central'}
2019-05-10 23:10:27.542 LOG <test0> [web_server_cqf9i4.start] INFO: Passing k8sconfig: {'tls': {u'cert_path': u'/opt/tls/shared', u'image': u'nexus3.onap.org:10001/onap/org.onap.dcaegen2.deployments.tls-init-container:1.0.3'}, 'filebeat': {u'config_map': u'dcae-filebeat-configmap', u'config_path': u'/usr/share/filebeat/filebeat.yml', u'log_path': u'/var/log/onap', u'image': u'docker.elastic.co/beats/filebeat:5.5.0', u'data_path': u'/usr/share/filebeat/data', u'config_subpath': u'filebeat.yml'}, 'consul_dns_name': u'consul-server.onap', 'image_pull_secrets': [u'onap-docker-registry-key'], 'namespace': u'onap', 'consul_host': 'consul-server:8500', 'default_k8s_location': u'central'}
2019-05-10 23:10:28.181 LOG <test0> [web_server_cqf9i4.start] INFO: k8s deployment initiated successfully for s61b2002abe554457a78050174c431719-nginx-web: {'services': ['s61b2002abe554457a78050174c431719-nginx-web'], 'namespace': u'onap', 'location': u'central', 'deployment': 'dep-s61b2002abe554457a78050174c431719-nginx-web'}
2019-05-10 23:10:28.181 LOG <test0> [web_server_cqf9i4.start] INFO: Waiting up to 1800 secs for s61b2002abe554457a78050174c431719-nginx-web to become ready
2019-05-10 23:10:55.773 LOG <test0> [web_server_cqf9i4.start] INFO: Done starting: s61b2002abe554457a78050174c431719-nginx-web
2019-05-10 23:10:55.251 LOG <test0> [web_server_cqf9i4.start] INFO: k8s deployment is ready for: s61b2002abe554457a78050174c431719-nginx-web
2019-05-10 23:10:56.506 CFY <test0> [web_server_cqf9i4.start] Task succeeded 'k8splugin.create_and_start_container_for_components_with_streams'
2019-05-10 23:10:57.058 CFY <test0> [web_server_cqf9i4] Node instance started
2019-05-10 23:10:57.058 CFY <test0> 'install' workflow execution succeeded
Finished executing workflow install on deployment test0
* Run 'cfy events list -e 7a4e25eb-cf1e-4d91-9004-60e4ecdff536' to retrieve the execution's events/logs
[root@dev-dcaegen2-dcae-bootstrap-55cb5596b7-phdct blueprints]# curl http://dmaap-bc:8080/webapi/topics/"org.onap.dmaap.mr.topic00_name" | python -mjson.tool
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 715 100 715 0 0 36634 0 --:--:-- --:--:-- --:--:-- 37631
{
"clients": [
{
"action": [
"view",
"pub"
],
"clientRole": "org.onap.dcae.pnfPublisher",
"dcaeLocationName": "san-francisco",
"fqtn": "org.onap.dmaap.mr.topic00_name",
"lastMod": "2019-05-10T23:10:20.11",
"mrClientId": "1557529820110",
"status": "VALID",
"topicURL": "http://message-router:3904/events/org.onap.dmaap.mr.topic00_name"
}
],
"fqtn": "org.onap.dmaap.mr.topic00_name",
"lastMod": "2019-05-10T23:10:06.078",
"owner": "dcaeorch",
"partitionCount": "2",
"publisherRole": "org.onap.dmaap.mr.topic00_name.publisher",
"replicationCase": "REPLICATION_NONE",
"replicationCount": "1",
"status": "VALID",
"subscriberRole": "org.onap.dmaap.mr.topic00_name.subscriber",
"topicDescription": "No description provided",
"topicName": "topic00_name",
"type": "topic"
}
[root@dev-dcaegen2-dcae-bootstrap-55cb5596b7-phdct blueprints]# curl http://dmaap-bc:8080/webapi/topics/"org.onap.dmaap.mr.topic01_name" | python -mjson.tool
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 717 100 717 0 0 43184 0 --:--:-- --:--:-- --:--:-- 44812
{
"clients": [
{
"action": [
"view",
"sub"
],
"clientRole": "org.onap.dcae.pnfSubscriber",
"dcaeLocationName": "san-francisco",
"fqtn": "org.onap.dmaap.mr.topic01_name",
"lastMod": "2019-05-10T23:10:23.378",
"mrClientId": "1557529823378",
"status": "VALID",
"topicURL": "http://message-router:3904/events/org.onap.dmaap.mr.topic01_name"
}
],
"fqtn": "org.onap.dmaap.mr.topic01_name",
"lastMod": "2019-05-10T23:10:05.877",
"owner": "dcaeorch",
"partitionCount": "2",
"publisherRole": "org.onap.dmaap.mr.topic01_name.publisher",
"replicationCase": "REPLICATION_NONE",
"replicationCount": "1",
"status": "VALID",
"subscriberRole": "org.onap.dmaap.mr.topic01_name.subscriber",
"topicDescription": "No description provided",
"topicName": "topic01_name",
"type": "topic"
}
[root@dev-dcaegen2-dcae-bootstrap-55cb5596b7-phdct blueprints]# curl http://dmaap-bc:8080/webapi/mr_clients/1557529820110 | python -mjson.tool
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 320 100 320 0 0 9200 0 --:--:-- --:--:-- --:--:-- 11428
{
"action": [
"view",
"pub"
],
"clientRole": "org.onap.dcae.pnfPublisher",
"dcaeLocationName": "san-francisco",
"fqtn": "org.onap.dmaap.mr.topic00_name",
"lastMod": "2019-05-10T23:10:20.11",
"mrClientId": "1557529820110",
"status": "VALID",
"topicURL": "http://message-router:3904/events/org.onap.dmaap.mr.topic00_name",
"type": "mr_Client"
}
[root@dev-dcaegen2-dcae-bootstrap-55cb5596b7-phdct blueprints]# curl http://dmaap-bc:8080/webapi/mr_clients/1557529823378 | python -mjson.tool
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 322 100 322 0 0 7471 0 --:--:-- --:--:-- --:--:-- 7666
{
"action": [
"view",
"sub"
],
"clientRole": "org.onap.dcae.pnfSubscriber",
"dcaeLocationName": "san-francisco",
"fqtn": "org.onap.dmaap.mr.topic01_name",
"lastMod": "2019-05-10T23:10:23.378",
"mrClientId": "1557529823378",
"status": "VALID",
"topicURL": "http://message-router:3904/events/org.onap.dmaap.mr.topic01_name",
"type": "mr_Client"
}
[root@dev-dcaegen2-dcae-bootstrap-55cb5596b7-phdct blueprints]# /opt/consul/bin/consul kv export | grep key
"key": "dcae-dashboard",
"key": "dcae-hv-ves-collector",
"key": "dcae-prh",
"key": "dcae-snmptrap-collector",
"key": "dcae-tca-analytics",
"key": "dcae-ves-collector",
"key": "deployment_handler",
"key": "dmaap-plugin",
"key": "holmes-engine-mgmt",
"key": "holmes-rule-mgmt",
"key": "k8s-plugin",
"key": "s61b2002abe554457a78050174c431719-nginx-web",
"key": "s61b2002abe554457a78050174c431719-nginx-web:dmaap",
[root@dev-dcaegen2-dcae-bootstrap-55cb5596b7-phdct blueprints]# /opt/consul/bin/consul kv get s61b2002abe554457a78050174c431719-nginx-web:dmaap | python -mjson.tool
{
"topic00": {
"client_id": "1557529820110",
"client_role": "org.onap.dcae.pnfPublisher",
"location": "san-francisco",
"topic_url": "http://message-router:3904/events/org.onap.dmaap.mr.topic00_name"
},
"topic01": {
"client_id": "1557529823378",
"client_role": "org.onap.dcae.pnfSubscriber",
"location": "san-francisco",
"topic_url": "http://message-router:3904/events/org.onap.dmaap.mr.topic01_name"
}
}
[root@dev-dcaegen2-dcae-bootstrap-55cb5596b7-phdct blueprints]# cfy uninstall test0
Executing workflow uninstall on deployment test0 [timeout=900 seconds]
2019-05-10 23:14:49.904 CFY <test0> Starting 'uninstall' workflow execution
2019-05-10 23:14:52.322 CFY <test0> [web_server_cqf9i4] Stopping node instance
2019-05-10 23:14:53.438 CFY <test0> [web_server_cqf9i4.stop] Sending task 'k8splugin.stop_and_remove_container'
2019-05-10 23:15:01.207 CFY <test0> [web_server_cqf9i4.stop] Task succeeded 'k8splugin.stop_and_remove_container'
2019-05-10 23:15:01.921 CFY <test0> [web_server_cqf9i4] Stopped node instance
2019-05-10 23:15:02.464 CFY <test0> [web_server_cqf9i4] Unlinking relationships
2019-05-10 23:15:02.464 CFY <test0> [unlink] Sending task 'dmaapplugin.mr_relationships.delete_mr_client'
2019-05-10 23:15:06.207 LOG <test0> [unlink] INFO: Deleting URL: http://dmaap-bc:8080/webapi/mr_clients/1557529823378
2019-05-10 23:15:05.464 LOG <test0> [unlink] INFO: Attempting to delete client 1557529823378
2019-05-10 23:15:06.207 LOG <test0> [unlink] INFO: Deleted client 1557529823378
2019-05-10 23:15:06.207 CFY <test0> [unlink] Task succeeded 'dmaapplugin.mr_relationships.delete_mr_client'
2019-05-10 23:15:06.867 CFY <test0> [unlink] Sending task 'dmaapplugin.mr_relationships.delete_mr_client'
2019-05-10 23:15:07.929 LOG <test0> [unlink] INFO: Attempting to delete client 1557529820110
2019-05-10 23:15:08.469 LOG <test0> [unlink] INFO: Deleting URL: http://dmaap-bc:8080/webapi/mr_clients/1557529820110
2019-05-10 23:15:08.469 LOG <test0> [unlink] INFO: Deleted client 1557529820110
2019-05-10 23:15:09.102 CFY <test0> [unlink] Task succeeded 'dmaapplugin.mr_relationships.delete_mr_client'
2019-05-10 23:15:09.102 CFY <test0> [web_server_cqf9i4] Relationships unlinked
2019-05-10 23:15:09.645 CFY <test0> [web_server_cqf9i4] Deleting node instance
2019-05-10 23:15:09.645 CFY <test0> [web_server_cqf9i4.delete] Sending task 'k8splugin.cleanup_discovery'
2019-05-10 23:15:12.474 CFY <test0> [web_server_cqf9i4.delete] Task succeeded 'k8splugin.cleanup_discovery'
2019-05-10 23:15:13.113 CFY <test0> [web_server_cqf9i4] Deleted node instance
2019-05-10 23:15:13.773 CFY <test0> [topic01_xjti5i] Stopping node instance
2019-05-10 23:15:14.311 CFY <test0> [topic00_ehyrmr] Stopping node instance
2019-05-10 23:15:15.004 CFY <test0> [topic01_xjti5i] Stopped node instance: nothing to do
2019-05-10 23:15:15.004 CFY <test0> [topic00_ehyrmr] Stopped node instance: nothing to do
2019-05-10 23:15:15.623 CFY <test0> [topic01_xjti5i] Unlinking relationships
2019-05-10 23:15:16.196 CFY <test0> [topic00_ehyrmr] Unlinking relationships
2019-05-10 23:15:16.196 CFY <test0> [topic01_xjti5i] Relationships unlinked
2019-05-10 23:15:16.807 CFY <test0> [topic01_xjti5i] Deleting node instance
2019-05-10 23:15:17.481 CFY <test0> [topic00_ehyrmr] Relationships unlinked
2019-05-10 23:15:17.481 CFY <test0> [topic01_xjti5i.delete] Sending task 'dmaapplugin.mr_lifecycle.delete_topic'
2019-05-10 23:15:18.122 CFY <test0> [topic00_ehyrmr] Deleting node instance
2019-05-10 23:15:18.122 CFY <test0> [topic00_ehyrmr.delete] Sending task 'dmaapplugin.mr_lifecycle.delete_topic'
2019-05-10 23:15:19.570 LOG <test0> [topic01_xjti5i.delete] INFO: Attempting to delete topic org.onap.dmaap.mr.topic01_name
2019-05-10 23:15:20.084 LOG <test0> [topic01_xjti5i.delete] INFO: Deleting URL: http://dmaap-bc:8080/webapi/topics/org.onap.dmaap.mr.topic01_name
2019-05-10 23:15:20.616 CFY <test0> [topic01_xjti5i.delete] Task succeeded 'dmaapplugin.mr_lifecycle.delete_topic'
2019-05-10 23:15:20.616 CFY <test0> [topic01_xjti5i] Deleted node instance
2019-05-10 23:15:21.133 LOG <test0> [topic00_ehyrmr.delete] INFO: Attempting to delete topic org.onap.dmaap.mr.topic00_name
2019-05-10 23:15:21.133 LOG <test0> [topic00_ehyrmr.delete] INFO: Deleting URL: http://dmaap-bc:8080/webapi/topics/org.onap.dmaap.mr.topic00_name
2019-05-10 23:15:21.849 CFY <test0> [topic00_ehyrmr.delete] Task succeeded 'dmaapplugin.mr_lifecycle.delete_topic'
2019-05-10 23:15:22.628 CFY <test0> [topic00_ehyrmr] Deleted node instance
2019-05-10 23:15:23.263 CFY <test0> 'uninstall' workflow execution succeeded
Finished executing workflow uninstall on deployment test0
* Run 'cfy events list -e 352271ca-ed0d-4823-9ebf-b2fd827f1e83' to retrieve the execution's events/logs
Deleting deployment test0...
Deployment deleted
Deleting blueprint test0...
An error occurred on the server: 400: Can't delete blueprint test0 - There exist deployments for this blueprint; Deployments ids: test0
[root@dev-dcaegen2-dcae-bootstrap-55cb5596b7-phdct blueprints]# cfy blueprints delete test0
Deleting blueprint test0...
Blueprint deleted
[root@dev-dcaegen2-dcae-bootstrap-55cb5596b7-phdct blueprints]# curl http://dmaap-bc:8080/webapi/topics/"org.onap.dmaap.mr.topic00_name" | python -mjson.tool
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 97 100 97 0 0 10192 0 --:--:-- --:--:-- --:--:-- 10777
{
"code": 404,
"fields": "fqtn",
"message": "topic with fqtn org.onap.dmaap.mr.topic00_name not found"
}
[root@dev-dcaegen2-dcae-bootstrap-55cb5596b7-phdct blueprints]# curl http://dmaap-bc:8080/webapi/topics/"org.onap.dmaap.mr.topic01_name" | python -mjson.tool
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 97 100 97 0 0 6631 0 --:--:-- --:--:-- --:--:-- 6928
{
"code": 404,
"fields": "fqtn",
"message": "topic with fqtn org.onap.dmaap.mr.topic01_name not found"
}
[root@dev-dcaegen2-dcae-bootstrap-55cb5596b7-phdct blueprints]# curl http://dmaap-bc:8080/webapi/mr_clients/1557529820110 | python -mjson.tool
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 81 100 81 0 0 9157 0 --:--:-- --:--:-- --:--:-- 10125
{
"code": 404,
"fields": "mrClientId",
"message": "mrClientId 1557529820110 not found"
}
[root@dev-dcaegen2-dcae-bootstrap-55cb5596b7-phdct blueprints]# curl http://dmaap-bc:8080/webapi/mr_clients/1557529823378 | python -mjson.tool
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 81 100 81 0 0 1762 0 --:--:-- --:--:-- --:--:-- 3240
{
"code": 404,
"fields": "mrClientId",
"message": "mrClientId 1557529823378 not found"
}
[root@dev-dcaegen2-dcae-bootstrap-55cb5596b7-phdct blueprints]# /opt/consul/bin/consul kv export | grep key
"key": "dcae-dashboard",
"key": "dcae-hv-ves-collector",
"key": "dcae-prh",
"key": "dcae-snmptrap-collector",
"key": "dcae-tca-analytics",
"key": "dcae-ves-collector",
"key": "deployment_handler",
"key": "dmaap-plugin",
"key": "holmes-engine-mgmt",
"key": "holmes-rule-mgmt",
"key": "k8s-plugin",
[root@dev-dcaegen2-dcae-bootstrap-55cb5596b7-phdct blueprints]#
The screen output for all the blueprints are available in the attached file. <link to be added later>
Validation
<expected logs + consul entry + curl commands/postman + any other steps for verification from MS standpoint>
After deploying a blueprint, we run curl commands to query DMaap DBCL configuration. For example, after deploying test_mr_pubsub.yaml, we can check whether topic00 is created in DBCL:
curl http://dmaap-bc:8080/webapi/topics/"org.onap.dmaap.mr.topic00_name" | python -mjson.tool
The return value of the query command is:
[root@dev-dcaegen2-dcae-bootstrap-55cb5596b7-phdct blueprints]# curl http://dmaap-bc:8080/webapi/topics/"org.onap.dmaap.mr.topic00_name" | python -mjson.tool
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 715 100 715 0 0 36634 0 --:--:-- --:--:-- --:--:-- 37631
{
"clients": [
{
"action": [
"view",
"pub"
],
"clientRole": "org.onap.dcae.pnfPublisher",
"dcaeLocationName": "san-francisco",
"fqtn": "org.onap.dmaap.mr.topic00_name",
"lastMod": "2019-05-10T23:10:20.11",
"mrClientId": "1557529820110",
"status": "VALID",
"topicURL": "http://message-router:3904/events/org.onap.dmaap.mr.topic00_name"
}
],
"fqtn": "org.onap.dmaap.mr.topic00_name",
"lastMod": "2019-05-10T23:10:06.078",
"owner": "dcaeorch",
"partitionCount": "2",
"publisherRole": "org.onap.dmaap.mr.topic00_name.publisher",
"replicationCase": "REPLICATION_NONE",
"replicationCount": "1",
"status": "VALID",
"subscriberRole": "org.onap.dmaap.mr.topic00_name.subscriber",
"topicDescription": "No description provided",
"topicName": "topic00_name",
"type": "topic"
}
The output contains the client ID associated with the topic ("mrClientId": "1557529820110"). We use the client ID to check the client creation in DBCL:
[root@dev-dcaegen2-dcae-bootstrap-55cb5596b7-phdct blueprints]# curl http://dmaap-bc:8080/webapi/mr_clients/1557529820110 | python -mjson.tool
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 320 100 320 0 0 9200 0 --:--:-- --:--:-- --:--:-- 11428
{
"action": [
"view",
"pub"
],
"clientRole": "org.onap.dcae.pnfPublisher",
"dcaeLocationName": "san-francisco",
"fqtn": "org.onap.dmaap.mr.topic00_name",
"lastMod": "2019-05-10T23:10:20.11",
"mrClientId": "1557529820110",
"status": "VALID",
"topicURL": "http://message-router:3904/events/org.onap.dmaap.mr.topic00_name",
"type": "mr_Client"
}
Similarly we can verify the creation of topic01 and its client.
The consul kv should contain the service's information.
[root@dev-dcaegen2-dcae-bootstrap-55cb5596b7-phdct blueprints]# /opt/consul/bin/consul kv export | grep key
"key": "dcae-dashboard",
"key": "dcae-hv-ves-collector",
"key": "dcae-prh",
"key": "dcae-snmptrap-collector",
"key": "dcae-tca-analytics",
"key": "dcae-ves-collector",
"key": "deployment_handler",
"key": "dmaap-plugin",
"key": "holmes-engine-mgmt",
"key": "holmes-rule-mgmt",
"key": "k8s-plugin",
"key": "s61b2002abe554457a78050174c431719-nginx-web",
"key": "s61b2002abe554457a78050174c431719-nginx-web:dmaap",
The detailed dmaap information of the new service is available in the value of key s61b2002abe554457a78050174c431719-nginx-web:dmaap:
[root@dev-dcaegen2-dcae-bootstrap-55cb5596b7-phdct blueprints]# /opt/consul/bin/consul kv get s61b2002abe554457a78050174c431719-nginx-web:dmaap | python -mjson.tool
{
"topic00": {
"client_id": "1557529820110",
"client_role": "org.onap.dcae.pnfPublisher",
"location": "san-francisco",
"topic_url": "http://message-router:3904/events/org.onap.dmaap.mr.topic00_name"
},
"topic01": {
"client_id": "1557529823378",
"client_role": "org.onap.dcae.pnfSubscriber",
"location": "san-francisco",
"topic_url": "http://message-router:3904/events/org.onap.dmaap.mr.topic01_name"
}
}
After validation we undeploy the blueprints with command "cfy uninstall test0". Then we rerun the validation to make sure that the newly create topics and their clients are deleted from DMaap DBCL and consul.
6 Comments
Muhammad Rehan Raza
Some clarifications are needed in the Test Setup.
"1) Add the following dmaap kv into consul":
The dmaap kv needs to be added into consul with the entry name "dmaap-plugin". Otherwise, an error is thrown when installing the blueprint.
"2) Add location san-francisco into DMaap DBCL":
It is not clear how to add this location in DBCL. Could you please clarify this? (Anyways, the plugin seems to work even without doing this step)
Vijay Venkatesh Kumar
Thanks. Added clarification on #1. Reg #2 - I think that san-francisco is added as default config in DBCL. Dominic Lunanuova - Can you confirm? And if new location should be added, is the steps defined in some place?
Joseph O'Leary
The san-francisco location is added as part of the dmaap-bc post-install job, if this job happens to fail you can always add it manually yourself:
Dominic Lunanuova
Joseph O'Leary
almost correct. The resource you list is actually used to add an MR cluster in a particular dcaeLocation and assumes that dcaeLocation "san-francisco" has already been created.
The correct curl command (different API URI and different data file) to add the dcaeLocation would be:
Joseph O'Leary
Dominic Lunanuova
Thanks for the correction, I especially appreciate the command.
Would it be accurate to say that the bus controller needs both a location and a message router cluster (in that location) to be present for this plugin to work?
Dominic Lunanuova
Yes. The correct dependency order is dcaeLocation → mr_clusters→topic→ mr_client