Skip to end of metadata
Go to start of metadata

Overview

VES-Mapper microservice of DCAE can help to collect telemetry data / events / alerts from VNFs and collectors of DCAE and convert them into VES compliant events. It supports both VES-5.4 and VES-7.0 specifications.

Based on the configuration, the VES-Mapper subscribes to non-VES notifications published on the DMaaP. By using mapping files, the mapper converts the received notifications into VES events and publish them back on DMaaP on the configured topic. It uses a Java framework called Smooks (smooks.org) for the mapping files.

In case of BroadBand Service (BBS), VES-Mapper will receive the PNF registration notifications from DMaaP that are published by RestConf Collector, convert them into VES-7.0 events and publish them back on DMaaP for PNF registration handler service to pick them.

Blueprint/model/image



Deployment Prerequisite/dependencies

VES-Mapper can be deployed individually though it will throw errors if it can not reach to DMaaP instance's APIs. To functionally test the mapper, only DMaaP is needed.

Deployment Steps

  • Transfer blueprint file in DCAE bootstrap POD under /blueprints directory
  • Log-in to the DCAE bootstrap POD's main container


To Deploy::

  1. Validate blueprint

    cfy blueprints validate /blueprints/k8s-vesmapper.yaml-template.yaml
  2. Upload validated blueprint

    cfy blueprints upload -b ves-mapper /blueprints/k8s-vesmapper.yaml-template.yaml
  3. Create Cloudify Deployment

    cfy deployments create -b ves-mapper ves-mapper
  4. Deploy component

    cfy executions start -d ves-mapper install


To Undeploy::

  1. Uninstall component

    cfy uninstall ves-mapper
  2. Delete blueprint

    cfy blueprints delete ves-mapper

Validation


  1. Check that VES-Mapper POD is created and it is in running state

    kubectl get pods -n onap | grep mapper
    dep-s36f1a2fadd614a42872bd680516e271a-dcaegen2-svc-mapper-l2cms 1/1 Running 0 1d
  2. Check the logs. Here are some important log statements that could be looked for.

    kubectl logs -n onap dep-s36f1a2fadd614a42872bd680516e271a-dcaegen2-svc-mapper-l2cms | more
    11:10:12.078 [main] INFO debugLogger - Triggering controller's start url
    11:10:12.078 [main] INFO debugLogger - Running curl command for url:http://localhost:8085/start
    11:10:12.186 [http-nio-8085-exec-1] INFO o.a.c.c.C.[Tomcat].[localhost].[/] - Initializing Spring DispatcherServlet 'dispatcherServlet'
    11:10:12.186 [http-nio-8085-exec-1] INFO o.s.web.servlet.DispatcherServlet - Initializing Servlet 'dispatcherServlet'
    11:10:12.200 [http-nio-8085-exec-1] INFO o.s.web.servlet.DispatcherServlet - Completed initialization in 14 ms
    11:10:12.243 [http-nio-8085-exec-1] INFO metricsLogger - UniversalVesAdapter Application starting...
    11:10:12.243 [http-nio-8085-exec-1] INFO debugLogger - Creating Subcriber and Publisher with creator.............
    ...
    ...
    ...
    11:10:13.219 [http-nio-8085-exec-1] INFO debugLogger - Created scriber topic:unauthenticated.DCAE_RCC_OUTPUTpublisher topic:unauthenticated.VES_PNFREG_OUTPUT
    11:10:13.230 [rcc-notification] INFO metricsLogger - fetch and publish from and to Dmaap started:rcc-notification
    ...
    ...
    ...
    11:10:13.322 [http-nio-8085-exec-1] INFO debugLogger - Created scriber topic:unauthenticated.ONAP-COLLECTOR-SNMPTRAPpublisher topic:unauthenticated.SEC_FAULT_OUTPUT
    11:10:13.326 [snmp-notification] INFO metricsLogger - fetch and publish from and to Dmaap started:snmp-notification
    ...
    ...
    ...
    11:13:11.873 [snmp-notification] DEBUG debugLogger - number of messages to be converted :0
    11:13:13.044 [rcc-notification] DEBUG debugLogger - number of messages to be converted :0

Functional testing of VES-Mapper individually

  1. Send a PNF registration notification on behalf of RestConf Collector by calling DMaaP API

    curl -v -k -H 'Content-Type:application/json' -POST http://message-router.onap.svc.cluster.local:3904/events/unauthenticated.DCAE_RCC_OUTPUT -d @rcc_noti.json

    rcc_noti.json

  2. Invoke DMaaP API to fetch message on the topic unauthenticated.VES_PNFREG_OUTPUT . VES-Mapper should have published converted VES event on this topic.

    curl -X GET 'http://message-router.onap.svc.cluster.local:3904/events/unauthenticated.VES_PNFREG_OUTPUT/grp2/con2?timeout=1600'


    Converted VES event

  3. VES-Mapper logs

    12:35:27.886 [rcc-notification] DEBUG debugLogger - number of messages to be converted :1
    12:35:27.889 [rcc-notification] INFO debugLogger - Retrieved JsonArray from Collector Config File
    12:35:27.889 [rcc-notification] INFO debugLogger - returning identifier array from Collector Config
    12:35:27.890 [rcc-notification] DEBUG debugLogger - Using Default Mapping file as Mapping file is not available for Enterprise Id / identifer ID:2541256
    12:35:28.075 [rcc-notification] INFO debugLogger - Transforming incoming json
    12:35:28.114 [rcc-notification] INFO debugLogger - Context created
    12:35:28.125 [rcc-notification] DEBUG debugLogger - consversion successful to VES Event
    12:35:28.125 [rcc-notification] INFO debugLogger - Incoming json transformed to VES format successfully:rcc-notification
    12:35:28.137 [rcc-notification] INFO debugLogger - Serialized VES json
    12:35:28.138 [rcc-notification] DEBUG debugLogger - Batch Queue capacity exceeds messages size. Flushing of all pending messages to DMaaP MR Publisher Topic.
    12:35:28.138 [rcc-notification] DEBUG debugLogger - Force publishing messages to DMaaP MR Topic. Messages Size: 1
    12:35:28.138 [rcc-notification] DEBUG debugLogger - DMaaP MR Publisher Authentication is disabled as username or password is not present.
    12:35:28.167 [rcc-notification] DEBUG debugLogger - DMaaP MR Messages published successfully. DMaaP Response Code: 200. DMaaP Response Body: {
    "serverTimeMs": 0,
    "count": 1
    }, Number of Messages published: 1
    12:35:28.168 [rcc-notification] INFO metricsLogger - Message successfully published to DMaaP Topic-
    {"event":{"commonEventHeader":{"domain":"pnfRegistration","eventId":"registration_1556627728115","eventName":"pnfRegistration","eventType":"pnfRegistration","lastEpochMicrosec":1.556627728115E12,"priority":"Normal","reportingEntityName":"VESMapper","sequence":0,"sourceName":"Huawei-HWTCC01B7504","startEpochMicrosec":1.556627728115E12,"version":"4.0.1","vesEventListenerVersion":"7.0.1"},"pnfRegistrationFields":{"additionalFields":{"remote-id":"some-remote-id","attachment-point":"NE=167772165-5-1","cvlan":"1005","svlan":"100"},"lastServiceDate":"somestring","manufactureDate":"","modelNumber":" somestring","oamV4IpAddress":"somestring","oamV6IpAddress":"somestring","pnfRegistrationFieldsVersion":"2.0","serialNumber":"HWTCC01B7504","softwareVersion":"v1","unitFamily":"somestring","unitType":"somestring","vendorName":"Huawei"}}}
  • No labels

1 Comment

  1. Kedar Ambekar Pooja Malik  - Few comments below from the meeting/demo today.

    1) Include snmptrap config also in blueprint
    2) Wiki update to include detail validation steps (provide sample curl and test events so integration team can use)

    Pls refer https://wiki.onap.org/pages/viewpage.action?pageId=60891185 and include similar information for Mapper here.

    Future work discussed are capture in this wiki - El-ALTO - Tracker; pls update if anything missed