This page is a detailed user guide for creating a very simple ONAP service that contains a valid monitoring configuration. This page covers the flow in SDC. The emphasis is on the flow through the various designers and other tools and none of these is explored in much detail except for the parts that are necessary for completion of this flow. Some caveats are highlighted though.

To some extent this aims to cover similar things as in DCAE-Design Studio but from a user perspective and in a more detailed level. This page also covers the full flow in SDC and does not restrict itself to steps that are implemented by DCAE-DS. Another point of comparison is Onboarding steps for DCAE MS through SDC/Policy/CLAMP (Dublin) and can be considered to cover non-policy parts of its section "Load Policy Model and blueprint into SDC repo (Development phase)".

There are also instructions for doing partly the same kind of thing in "the old way" by using Cloudify blueprints directly in services in https://onap.readthedocs.io/en/latest/guides/onap-user/design/control-loop/index.html#. TODO: The relation of the procedure described in the above page to the procedure described in this page is not clear. Are both now supported? Or either one? Combination of both?

This flow is for a micro service that does not use a policy. Additional consideration from including a policy should be added later.

This has been tested with SDC/Dublin with software from around RC0 time. As the relevant parts were not really developed much during Dublin, these probably mostly apply to Casablanca as well. This has not been tested though.

Terminology

High Level Flow

1. Onboard a DCAE micro service (From Spec to VF)
2. Create a monitoring template (From VF to VFCMT)
3. Create a monitoring configuration (From VFCMT to MC)
4. Test, approve and distribute a service (From SDC to CLAMP, DCAE and Policy)

At the time of writing there are no DCAE typed VFs or any VFCMTs coming with the default SDC installation so you need to do all these steps to create your first service with an MC. VFs and VFCMTs can be used in multiple services, so only steps 3 and 4 need to be repeated for each service.

Onboard a DCAE Micro Service (From Spec to VF)

This is a special case of VF Creation as described in https://onap.readthedocs.io/en/latest/guides/onap-user/design/vfcreation/index.html. Those instructions do not really apply much to this DCAE VF case on practical level, but conceptually fulfill the same part of the design process. Especially note that license model (VLM)  or vendor software product (VSP) concepts are not involved at all. 

Input

The Spec file for DCAE Microservice. Further details of what this means and steps for creating a valid spec file are described in DCAE documentation. This is one possible start point for exploring that: MicroServices Onboarding in ONAP

An example Spec file for TCA without a policy is available in 

https://gerrit.onap.org/r/gitweb?p=dcaegen2/analytics/tca.git;a=blob;f=dpo/tcadocker_spec.json;h=4eb3488cd5ed6fd27d29996a70edca5d103e9348;hb=refs/heads/dublin

Procedure from Spec to TOSCA

SDC does not understand Spec file, so the first step is to translate it to a set of TOSCA files. This can only be done using a special command line tool as detailed here: How to generate artefacts for SDC catalog using Tosca Lab Tool

You can either clone the tosca-lab repo or kubectl exec into the toscalab container in a running SDC to be able to run the commands. Either way, copy the output TOSCA files (template.yaml, schema.yaml and translate.yaml) to a place that is accessible by the browser you are using with SDC GUI.

Procedure from TOSCA to VF

Open SDC GUI as service designer. On the top tab open HOME tab and choose "Add VF". Fill in the "General" fields as you wish but choose as Category "Microservice" under the "DCAE Component" heading. Press Create. 

To import the TOSCA files to the VF, click "Deployment Artifact" on the left hand side and click "Add Other Artifact" in the center of the screen. Fill in the name and description of the file, choose type "DCAE_TOSCA" and choose one of the TOSCA files generated above (template.yaml, schema.yaml, translate.yaml). Repeat this for all three files. 

To complete the VF part, click "Certify" near the top-right corner of the screen.

Output

There is a certified VF with type DCAE Microservice in SDC Catalogue that has the above mentioned three TOSCA files as deployment artifacts. 

Create a Monitoring Template (From VF to VFCMT)

This step creates a new monitoring template containing the DCAE VF. VFCMTs are visible only in DCAE-DS so don't worry if you don't see them in SDC catalogue. 

Input

A DCAE Microservice VF in SDC Catalogue. 

Procedure

Click "DCAE-DS" in the top tabs. This stands for DCAE Design Studio. The screen that opens is sometimes called DCAE-DS MC Catalog or VFCMT Catalog and these entities are not visible in the normal SDC catalogue. Unfortunately these terms are no visible in the GUI itself. If you started from clean SDC and just created a VF, the catalogue is empty.

Click "Create New Asset" button in the top-left. Fill your the name and description as you wish and press "save" to create the initial VFCMT.

Click "Composition" on the left hand side menu and the visual composition designer opens. For this example choose "TCA Hi Lo" from the "Select Flow Type" listbox.

TODO: The writer of this document has no clue what this choice of Flow Type actually affects, so if someone reading this knows, please add it here.

The VF you added in the previous chapter is visible in the "Microservice" box on the right, but you need to click it to expand to show a list of VFs. The screenshot below shows it expanded in a case of only one DCAE VF in SDC. If you have more VFs with type DCAE in SDC, they are visible here organized in a separate box by subcategory. If you have no DCAE VFs in SDC, no boxes are visible and you cannot effectively do anything here.

Last step to reach the state shown in the screenshot below is to drag a VF from the box on the right to the central area. It visualizes the structure of the VF. In this case one container "tca-docker" subcribing to one DMaaP MR topic "topic0" and publishing to another "topic1".

TODO: This view could be used for editing the topology or composing more complex flow over multiple VFs at least in principle, but this document does not cover this.

Press "Save" to complete the VFCMT design. If you now check the DCAE-DS catalogue (By first clicking some other top tab and then "DCAE-DS"), you can see your VFCMT listed there in state "NOT_CERTIFIED_CHECKIN".

TODO: The writer of this document does not know any way of getting a VFCMT to "Certified" state. This does not seem to matter in practice as it can be used in further steps anyway with this state.

Output

A VFCMT in state "NOT_CERTIFIED_CHECKIN" containing the given VF in DCAE-DS Catalogue.

Create a Monitoring Configuration (From VFCMT to MC)

This step creates a monitoring configuration (MC) for a particular service based on a VFCMT created in DCAE-DS in the previous step. No actual useful service is designed here, only the bare minimum needed for attaching a monitoring configuration to it.

Input

SDC Catalogue has a DCAE VF in state "Certified" and DCAE-DS Catalogue has a VFCMT in state "NOT_CERTIFIED_CHECKIN".

Procedure

Create a service normally. "Add Service" from HOME tab, fill in the general tab with whatever values and click "Create".  

TODO: No useful reference for the generic service design found.

A Monitoring Configuration currently requires a VF in the service. Usually for a useful service there would be a VF (VNF) implementing some actual network function, but for simplicity we will use the same DCAE VF created above for this. To use it in a service, open "Composition" from the menu on the left and drag your VF from the "ELEMENTS" menu on the left to the central composition area. The DCAE VF is visible under "DCAE Component" heading in the "ELEMENTS" menu. 

To start Monitoring Configuration creation, click "Monitoring" on the left hand side menu. 

For a new service no monitoring configurations exist, so this view is pretty empty. However, the trick is that you cannot add a new MC at this point by either "Add First MC" or "Create New MC" as there are no VFCMTs available in this service context. You first have to "import" a VFCMT to this service context. You can do this by clicking the button at the head of the big read arrow. This will effectively copy a VFCMT to this service context, so it will also be available as a template to any further monitoring configurations. Pick one of the available options in list boxes and choose a name and description of your choice. If you have followed all of this page and have only one VF in the service and only one VFCMT, all the list boxes will have only one item available. For some reason only the latest version of VFCMT is available. When doing this import way of creating an MC, the flow diagram is not active for some reason. Click "Import" to complete the import or in some sense customization of a VFCMT to monitoring configuration of this service.

This gets you to normal Monitoring Configuration view like the one below, complete with a flow diagram. You can change some parameters of the MC by clicking on one of the sub-tabs and the cogwheel in each sub-tab's top right hand corner.

To complete the Monitoring Configuration creation, click "Submit" on the right. This submits only the MC. TODO: It is not clear what does the submit actually mean here compared to the state space of other SDC artifacts.

Take care not to accidentally click "Submit for testing" a bit higher on the window, as that will submit the whole service for testing.

Optional check of state of the monitoring confifuration

To check the state of things now, if you click some other item in the left side menu and then again on "Monitoring", you can see the newly created Monitoring configuration in the list with state "Submitted".

Also, if you go back to non-service context DCAE-DS (Choose DCAE-DS from small arrow menu on the top), you can see your Monitoring configuration listed here in DCAE-DS Catalog as a VFCMT and state "CERTIFIED".

Finally in service context, click "Submit for Testing" and give a commit message.

Output

A service in state "Ready for Testing" containing a monitoring configuration.

Test, approve and distribute a service (From SDC to CLAMP, DCAE and Policy)

Input

A service in state "Ready for Testing" containing a Monitoring Configuration (MC) .

Procedure

From user point of view this procedure is exactly the same as with SDC in general, like described here: https://onap.readthedocs.io/en/latest/guides/onap-user/design/service-distribution/index.html

Some ONAP demo videos also show the step by step procedure from the user point of view.

Behind the scenes the difference is that relevant artifacts get distributed to the control loop runtime components: CLAMP, DCAE, Policy.

A Service CSAR Resulting from this Procedure

A distribution of a CSAR created using the procedure of this page succeeds from the SDC point of view. However, CLAMP complains that the VF is not valid and seems to reject the service CSAR due to this. Clearly there are further requirements at least for the VF. This guide will be updated when this is resolved.

The problem seems to be related to VF TOSCA template generation in SDC ( SDC-2330 - Getting issue details... STATUS ) which is still unresolved and at the moment does not seem to get fixed for Dublin. Unfortunately the resulting csar is not usable in runtime until this gets fixed some time in the El Alto timeframe. 

The distributed csar: 

service-Timoservice-csar.csar

CLAMP debug log from the time of distribution

2019-05-14T13:10:42.853Z|INFO|||Successfully downloaded the artifact /sdc/v1/catalog/services/Timoservice/1.0/artifacts/service-Timoservice-csar.csar UUID 13982d81-ce2f-4a7f-b679-cce67e9f9353Size of pa
yload 50389
2019-05-14T13:10:42.854Z|INFO|||Writing CSAR file to: /tmp/sdc-controllers/sdc-controller/service-Timoservice-csar.csar UUID 13982d81-ce2f-4a7f-b679-cce67e9f9353)
2019-05-14T13:10:42.930Z|DEBUG|||going to scan these urls:
2019-05-14T13:10:42.968Z|INFO|||Reflections took 34 ms to scan 1 urls, producing 4 keys and 4 values
2019-05-14T13:10:43.236Z|DEBUG|||going to scan these urls:
2019-05-14T13:10:43.239Z|INFO|||Reflections took 3 ms to scan 1 urls, producing 4 keys and 4 values
2019-05-14T13:10:43.352Z|DEBUG|||ToscaTemplate Loading YAMEL file /tmp/JTP2670299658087974073/Definitions/service-Timoservice-template.yml
2019-05-14T13:10:45.045Z|DEBUG|||CSAR conformance level is 9.0
2019-05-14T13:10:45.049Z|WARN|||####################################################################################################
2019-05-14T13:10:45.049Z|WARN|||CSAR Warnings found! CSAR name - /tmp/sdc-controllers/sdc-controller/service-Timoservice-csar.csar
2019-05-14T13:10:45.049Z|WARN|||ToscaTemplate - verifyTemplate - 2 Parsing Warnings occurred...
2019-05-14T13:10:45.050Z|WARN|||JTosca Exception [JE003]: MissingRequiredFieldError: properties of template "TimoDCAEVF1 0" are missing field(s): [nf_naming_code, nf_function, sdnc_artifact_name, max_i
nstances, nf_type, sdnc_model_version, nf_role, min_instances, sdnc_model_name]. CSAR name - /tmp/sdc-controllers/sdc-controller/service-Timoservice-csar.csar
2019-05-14T13:10:45.050Z|WARN|||JTosca Exception [JE003]: MissingRequiredFieldError: Data value of type "org.openecomp.datatypes.Naming" is missing required field(s) "[supplemental_data]". CSAR name -
/tmp/sdc-controllers/sdc-controller/service-Timoservice-csar.csar
2019-05-14T13:10:45.050Z|WARN|||####################################################################################################
2019-05-14T13:10:45.051Z|ERROR||org.onap.clamp.clds.exception.sdc.controller.SdcArtifactInstallerException: Error when searching for timodcaevf10 as ResourceInstanceName in Sdc notification and did not
 find it\n\tat org.onap.clamp.clds.sdc.controller.installer.CsarHandler.searchForResourceByInstanceName(CsarHandler.java:126)\n\tat org.onap.clamp.clds.sdc.controller.installer.CsarHandler.loadDcaeBlue
print(CsarHandler.java:144)\n\tat org.onap.clamp.clds.sdc.controller.installer.CsarHandler.save(CsarHandler.java:110)\n\tat org.onap.clamp.clds.sdc.controller.SdcSingleController.treatNotification(SdcS
ingleController.java:273)\n\tat org.onap.clamp.clds.sdc.controller.SdcSingleController$SdcNotificationCallBack.activateCallback(SdcSingleController.java:101)\n\tat org.onap.sdc.impl.NotificationConsume
r.run(NotificationConsumer.java:71)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)\n\tat java.util.
concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)\n\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThrea
dPoolExecutor.java:294)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat
 java.lang.Thread.run(Thread.java:748)\n|SdcArtifactInstallerException exception caught during the notification processing
2019-05-14T13:10:45.052Z|INFO|||Sending DEPLOY(DEPLOY_ERROR) notification to SDC for artifact:/sdc/v1/catalog/services/Timoservice/1.0/artifacts/service-Timoservice-csar.csar(Error when searching for t
imodcaevf10 as ResourceInstanceName in Sdc notification and did not find it)
2019-05-14T13:10:45.052Z|INFO|||DistributionClient - sendDeploymentStatus with errorReason
2019-05-14T13:10:45.053Z|INFO|||DistributionClient - sendStatus