| Table of Contents |
|---|
cdap-tca-hi-lo: a TCA application enhanced for Amsterdam/ONAP R1
This page is intended to give a broad overview of a microservice which has been onboarded with the DCAE platform and should be considered the "launch page" for others to find out basic information about your microservice. "Others" will include testers, DCAE/ASDC Service Designers, team members from other subsystems (Policy, CLAMP), etc.
Name:
cdap-tca-hi-lo
Synopsis/Description:
The cdap-tca-hi-lo app was first delivered as part of ONAP R0. In that release, it was intended to be an application that established a software architecture for building CDAP applications that demonstrate sufficient unit test coverage and reusable libraries for ingesting DMaaP MR feeds formatted according to the VES standard. Functionally, it performs a simple comparison of an incoming performance metric(s) against both a high and low threshold.
...
- Enhance TCA app to support R1 vCPE use case (requires new configuration model)
- also needs to support the ONAP R1 vFW, vDNS/vLB use cases
- unchanged from R0 DCAE analytics perspective; primary difference is the deployment of the TCA app is via CLAMP and the new DCAE controller
Gerrit:
Sonar:
Jira:
https://jira.onap.org/browse/DCAEGEN2-123?jql=labels%20%3D%20cdap-tca-hi-lo
DCAEGEN2-123 - TCA needs to have fall-back for obtaining of DMaaP preferences
DCAEGEN2-116 - Add support for VES/A&AI enrichment within the CDAP TCA app
DCAEGEN2-107 - Add support for ABATED alerts within the CDAP TCA app
DCAEGEN2-65 - Analytics app upgrade to support 5.x
Deferred to Beijing Release:
DCAEGEN2-119 - Update CDAP/TCA application
Detailed Description:
This CDAP application is driven by the VES collector which outputs to Message Router. This Message Router topic is the source for the CDAP application which will read each incoming message. If a message meets the Common Event Format (CEF, v28.3) as specified by the VES 5.3 standard (AttServiceSpecification-VesEventListener-v5.3.docx, Rev: 5.3, 6/22/17), it will be parsed and if it contains a message which matches the policy configuration for a given metric (denoted primarily by the "eventName" and the "fieldPath"), the value of the metric will be compared to the "thresholdValue". If that comparison indicates that a Control Loop Event Message should be generated, the application will output the alarm to the Message Router Sink topic in a format that matches the interface spec defined in the ONAP Control Loop Operational Policy (see section titled "Control Loop Event Messages").
Assumptions:
TCA output will be similar to R0 implementation, where CL event will be triggered each time threshold rules are met.
- Support for ABATEMENT event is not strictly necessary in the vFW or vDNS/vLB use cases as the Policy team has an "ABATEMENT not expected" configuration setting. In the context of the vCPE use case, the CLEAR event (aka ABATED event) is driven by a measured metric (i.e. packet loss equal to 0) rather than by the lapse of a threshold crossing event over some minimum number of measured intervals. Thus, this requirement can be accommodated by use of the low threshold with a policy of "direction = 0". Hence, for this release, the cdap-tca-hi-lo implementation will keep only the minimal state needed to correlate an ABATED event with the corresponding ONSET event. This correlation will be indicated by the requestID in the Control Loop Event Message.
CDAP Programming Paradigm
The ONAP R0 version of this microservice was built using the flowlet paradigm. We have also successfully built it as a batch pipeline. For Amsterdam/ONAP R1, the code has been refactored to allow for delivery as either a flowlet or a batch pipeline. We expect that for this release the implementation will stay with the flowlet version since other parts of the DCAE platform do not yet support CDAP pipelines.
Public APIs:
MR Interface - Source
The source interface to TCA is Message Router; the input topic is expected to contain CEF messages (v28.3) as defined in the VES specification. Sample 1 and Sample 2 contain sample data based on the known schema in this spec as of 8/9/17. This data can be used as a guide for planning and development, but should not be considered authoritative for any of the use cases.
...
Sample 1 - Sample data for vFW and vDNS use cases; downloadable: ves5.3-measurement-vf5vdns.txt
| Code Block | ||||||||
|---|---|---|---|---|---|---|---|---|
| ||||||||
{
"event": {
"commonEventHeader": {
"startEpochMicrosec": 1500584201765465,
"sourceId": "Dummy VM UUID - No Metadata available",
"eventId": "11",
"reportingEntityId": "No UUID available",
"internalHeaderFields": {
"collectorTimeStamp": "Thu, 07 20 2017 08:56:52 GMT"
},
"eventType": "HTTP request rate",
"priority": "Normal",
"version": 1.2,
"reportingEntityName": "fwll",
"sequence": 11,
"domain": "measurementsForVfScaling",
"lastEpochMicrosec": 1500584212017216,
"eventName": "vnfScalingMeasurement",
"sourceName": "Dummy VM name - No Metadata available"
},
"measurementsForVfScalingFields": {
"cpuUsageArray": [{
"percentUsage": 0,
"cpuIdentifier": "cpu1",
"cpuIdle": 100,
"cpuUsageSystem": 0,
"cpuUsageUser": 0
}],
"measurementInterval": 10,
"requestRate": 7832,
"vNicUsageArray": [{
"transmittedOctetsDelta": 5680,
"receivedTotalPacketsDelta": 14,
"vNicIdentifier": "eth0",
"valuesAreSuspect": "true",
"transmittedTotalPacketsDelta": 31,
"receivedOctetsDelta": 1332
}],
"measurementsForVfScalingVersion": 2.1
}
}
}
|
...
Sample 2 - Sample data for general VES 5.3 use cases; downloadable: VES-5.3-measurement-mockup.txt
| Code Block | ||||||||
|---|---|---|---|---|---|---|---|---|
| ||||||||
{
"event":
{
"measurementsForVfScalingFields":
{
"measurementInterval":5,
"meanRequestLatency":0,
"additionalMeasurements":
[
{
"arrayOfFields":
[
{"name":"jvm.maxHeapSize", "value":"1040064" },
{"name":"jvm.peak.cpu", "value":"2.00" },
{"name":"jvm.usedMemory", "value":"284089"},
{"name":"jvm.freeMemory","value":"755975"},
{"name":"jvm.average.cpu","value":"0.41"}
],
"name":"virc-jvm"
},
{
"arrayOfFields":
[
{"name":"average.caps","value":"0.00"},
{"name":"mrf.numOfPlayedAnns","value":"0"},
{"name":"numOfSipServletErrors","value":"0"},
{"name":"numOfBlockOnMaxForwards","value":"0"},
{"name":"numOfBlockOnInvalidNumber","value":"0"},
{"name":"mrf.numOfFailuresOnNoAvailableMrf","value":"0"},
{"name":"mrf.numOfNotFoundAnns","value":"0"},
{"name":"dns.numOfSuccessfulDnsQueries","value":"0"},
{"name":"numOfCallAttempts","value":"0"},
{"name":"dns.numOfFailedDnsQueries","value":"0"}
],
"name":"virc-cc-generic"
},
{
"arrayOfFields":
[
{"name":"ccf.numOfAbnormalRfSessions","value":"0"},
{"name":"ccf.average.tps","value":"0.00"},
{"name":"ccf.low.tps","value":"0.00"},
{"name":"ccf.numOfDroppedAcrDueToBufferOverflow","value":"0"},
{"name":"ccf.numOfResentAcr","value":"0"},
{"name":"ccf.numOfFailedNotResentAcr","value":"0"},
{"name":"ccf.peak.tps","value":"0.00"},
{"name":"ccf.numOfActiveRfSessions","value":"14"}
],
"name":"virc-cc-ccf"
},
{
"arrayOfFields":
[
{"name":"inb.numOfCallReattemptsOnBgcfRouteAdvance","value":"0"},
{"name":"inb.average.call.block.latency","value":"0.00"},
{"name":"inb.numOfCallReattemptsOnBgcfNoAnswer","value":"0"},
{"name":"inb.average.call.setup.latency","value":"0.00"},
{"name":"inb.numOfCancelledCalls","value":"0"},
{"name":"inb.numOfBlockOnNpaNxxXxxxScreening","value":"0"},
{"name":"inb.numOfIncomingCalls","value":"0"},
{"name":"inb.numOfBlockOnOperatorCallScreening","value":"0"},
{"name":"inb.numOfAnsweredCalls","value":"0"},
{"name":"inb.numOfBlockOnFailedSiDerivation","value":"0"},
{"name":"inb.numOfBlockOnOrigCarrierScreening","value":"0"},
{"name":"inb.concurrentCalls","value":"0"},
{"name":"inb.numOfBlockOnSrvOrigDestScreening","value":"0"},
{"name":"inb.numOfFailedCalls","value":"0"},
{"name":"inb.average.caps","value":"0.00"},
{"name":"inb.numOfBlockOnNoAvailableBgcf","value":"0"},
{"name":"inb.peak.caps","value":"0.00"}
],
"name":"virc-cc-inbound"
},
{
"arrayOfFields":
[
{"name":"outb.numOfCallReAttemptsOnBgcfNoAnswer","value":"0"},
{"name":"outb.numOfBlockOnAdcScreening","value":"0"},
{"name":"outb.dom.numOfFailedCalls","value":"0"},
{"name":"outb.average.call.block.latency","value":"0.00"},
{"name":"outb.average.caps","value":"0.00"},
{"name":"outb.dom.numOfAnsweredCalls","value":"0"},
{"name":"outb.dom.numOfCancelledCalls","value":"0"},
{"name":"outb.numOfDestNumberPrefixManipulations","value":"0"},
{"name":"outb.transit.numOfCancelledCalls","value":"0"},
{"name":"outb.numOfBlockOnOrigCarrierScreening","value":"0"},
{"name":"outb.numOfBlockOnCspDetermination","value":"0"},
{"name":"outb.numOfNationalNumberManipulations","value":"0"},
{"name":"outb.numOfBlockOnCountryCodeScreening","value":"0"},
{"name":"outb.average.call.setup.latency","value":"0.00"},
{"name":"outb.numOfCallReAttemptsOnBgcfRouteAdvance","value":"0"},
{"name":"outb.numOfInterCarrierCallReattempts","value":"0"},
{"name":"outb.dom.numOfIncomingCalls","value":"0"},
{"name":"outb.numOfCliAnonymization","value":"0"},
{"name":"outb.numOfBlockOnLeadingZeroScreening","value":"0"},
{"name":"outb.concurrentCalls","value":"0"},
{"name":"outb.numOfBlockOnNoAvailableBgcf","value":"0"},
{"name":"outb.peak.caps","value":"0.00"},
{"name":"outb.numOfBlockOnFailedDcgDetermination","value":"0"},
{"name":"outb.transit.numOfAnsweredCalls","value":"0"},
{"name":"outb.numOfBlockOnFailedSiDerivation","value":"0"},
{"name":"outb.transit.numOfIncomingCalls","value":"0"},
{"name":"outb.numOfIntraCarrierCallReattempts","value":"0"},
{"name":"outb.transit.numOfFailedCalls","value":"0"},
{"name":"outb.numOfBlockOnSrvOrigDestScreening","value":"0"}
],
"name":"virc-cc-outbound"
},
{
"arrayOfFields":
[
{"name":"out.request.update","value":"0"},
{"name":"out.request.info","value":"0"},
{"name":"in.request.publish","value":"0"},
{"name":"in.request.ack","value":"0"},
{"name":"in.request.options","value":"0"},
{"name":"out.request.ack","value":"0"},
{"name":"in.request.invite","value":"0"},
{"name":"in.request.refer","value":"0"},
{"name":"out.request.subscribe","value":"0"},
{"name":"in.request.subscribe","value":"0"},
{"name":"out.request.message","value":"0"},
{"name":"in.request.info","value":"0"},
{"name":"out.request.notify","value":"0"},
{"name":"in.request.prack","value":"0"},
{"name":"in.response.1xx","value":"0"},
{"name":"in.request.notify","value":"0"},
{"name":"out.response.5xx","value":"0"},
{"name":"out.response.6xx","value":"0"},
{"name":"in.response.3xx","value":"0"},
{"name":"in.response.2xx","value":"0"},
{"name":"active.sip.sessions","value":"0"},
{"name":"in.request.cancel","value":"0"},
{"name":"out.request.bye","value":"0"},
{"name":"out.request.prack","value":"0"},
{"name":"out.response.1xx","value":"0"},
{"name":"out.response.2xx","value":"0"},
{"name":"in.request.register","value":"0"},
{"name":"in.request.bye","value":"0"},
{"name":"out.response.3xx","value":"0"},
{"name":"out.response.4xx","value":"0"},
{"name":"out.request.publish","value":"0"},
{"name":"out.request.options","value":"0"},
{"name":"out.request.invite","value":"0"},
{"name":"out.request.refer","value":"0"},
{"name":"out.request.register","value":"0"},
{"name":"in.request.update","value":"0"},
{"name":"in.response.5xx","value":"0"},
{"name":"in.response.4xx","value":"0"},
{"name":"in.request.message","value":"0"},
{"name":"out.request.cancel","value":"0"},
{"name":"in.response.6xx","value":"0"}
],
"name":"virc-cc-sip"
}
],
"concurrentSessions":0,
"measurementsForVfScalingVersion": 2
},
"commonEventHeader":
{
"reportingEntityName":"ircc0002vm049cca001",
"startEpochMicrosec":1496235300000000,
"lastEpochMicrosec":1496235600000000,
"eventId":"3409",
"sourceName":"ircc0002vm049cca001",
"sequence":0,
"priority":"Normal",
"domain":"measurementsForVfScaling",
"internalHeaderFields":
{
"collectorTimeStamp":"Wed, 05 31 2017 01:00:01 GMT"
},
"eventType":"Mfvs_ircc_cca",
"eventName":"Mfvs_ircc",
"version" : 3.0
}
}
}
|
| Anchor | ||||
|---|---|---|---|---|
|
The output TCA alert format is constructed as shown in Table 1. The description field below is copied from the Control Loop Event Message section here.
...
JSON Field | JSON sub-field | Populate Using | Description | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| { | |||||||||||||
| closedLoopControlName | closedLoopControlName included in the DCAE configuration Policy | The unique ID for the Control Loop. It is created by the CLAMP platform during Control Loop design. The DCAE Micro service that publishes this event structure MUST include this ID. | |||||||||||
| version | version included in the DCAE configuration Policy | The version of the Control Loop event message. Should be '1.0.2'. | |||||||||||
| requestID | Generate a UUID for this output message | For the control loop, when an instance of the Control Loop occurs, this unique ID must be created. The same ID must be forwarded for both the ONSET and the ABATED control loop messages. | |||||||||||
| closedLoopAlarmStart | commonEventHeader.startEpochMicrosec from the received VES measurementsForVfScaling message | When the alarm was first detected. | |||||||||||
| closedLoopEventClient | Concatenate name of this DCAE instance and name for this TCA instance, separated by "." | For monitoring/logging/auditing purposes, if there is an instance ID of the DCAE micro service this field should be populated with it. | |||||||||||
| target_type | "VNF" | The type of the target: VM or VNF. Future PNF(?). | |||||||||||
| target | "generic-vnf.vnf-idname" or "vserver.vserver-name" | This is the name of the field within the A&AI sub-tag that indicates the actual entity Node details. There should be a matching node field within the A&AI subtag holding this value. | |||||||||||
| AAI | { | Contains the A&AI Node-Attribute list. | |||||||||||
generic-vnf.vnf-idname | commonEventHeader.reportingEntityNamesourceName from the received VES measurementsForVfScaling message (value for the data element used in A&AI) | NEW: Updated on 10/23/17 to use vnf-name instead of vnf-id based on
| |||||||||||
| generic-vnf.in-maint | value | If the A&AI enrichment query added in JIRA
| |||||||||||
| generic-vnf.is-closed-loop-disabled | value | ||||||||||||
| generic-vnf.orchestration-status | value | ||||||||||||
| generic-vnf.prov-status | value | ||||||||||||
| generic-vnf.resource-version | value | ||||||||||||
| generic-vnf.service-id | value | ||||||||||||
| generic-vnf.vnf-nameid | value | ||||||||||||
| generic-vnf.vnf-type | value | ||||||||||||
| } | |||||||||||||
| from | "DCAE" | The ONAP platform component publishing this message. If DCAE, then it should be 'DCAE'. | |||||||||||
| policyScope | policyScope included in the DCAE configuration Policy | The version of the Policy driving the DCAE Micro service. Should be a part of the configuration policy setup by CLAMP and passed by DCAE controller. | |||||||||||
| policyName | policyName included in the DCAE configuration Policy | The version of the Policy driving the DCAE Micro service. Should be a part of the configuration policy setup by CLAMP and passed by DCAE controller. | |||||||||||
| policyVersion | policyVersion included in the DCAE configuration Policy | The version of the Policy driving the DCAE Micro service. Should be a part of the configuration policy setup by CLAMP and passed by DCAE controller. | |||||||||||
| closedLoopEventStatus | "ONSET" |
...
"controlLoopSchemaType": "VNF"
and the A&AI enrichment query fails or A&AI enrichment is disabled by setting the preference "enableAAIEnrichment" : false
Sample 3 - Sample Control Loop Event Message for vFW demo, "controlLoopSchemaType": "VNF"; no enrichmentAnchor SampleControlLoopMsgvFW SampleControlLoopMsgvFW
| Expand |
|---|
{ , |
...
"controlLoopSchemaType": "VM"
and the A&AI enrichment query failsfails or A&AI enrichment is disabled by setting the preference "enableAAIEnrichment" : false
Sample 4 - Sample Control Loop Event Message for vDNS/vLB demo, "controlLoopSchemaType": "VM", no enrichmentAnchor SampleControlLoopMsgvDNS SampleControlLoopMsgvDNS
| Expand |
|---|
{ "closedLoopEventClient": "DCAE_INSTANCE_ID.dcae-tca", "policyVersion": "1.0.0.5", "policyName": "vLoadBalancer", "policyScope": "resource=SampleResource,service=SampleService,type=SampleType,closedLoopControlName=SampleClosedLoop", "target_type": "VM", "AAI": { "vserver.vserver-name": "dfw1lb01lb01" }, "closedLoopAlarmStart": 1484855074322720, "closedLoopEventStatus": "ONSET", "closedLoopControlName": "CL-DNS-LOW-TRAFFIC-SIG-d925ed73-8231-4d02-9545-db4e101f88f8", "version": "1.0.2", "target": "vserver.vserver-name", "requestID": "8e9bc97e-795c-4195-820a-45a53a1db947", "from": "DCAE" } |
...
In both cases, the value of the fields, generic-vnf.vnf-id and name and vserver.vserver-name, is constructed from the incoming VES measurementsForVfScaling message received from the upstream collector in the field: commonEventHeader.reportingEntityNamesourceName.
Policy Interface (Control Loop Event structure)
In the Data Flow Diagram, v1 - Flow 3c, the Policy Interface described here is shown in Flow 3c. This is how the application gets its policy config, whether it comes at deployment and instantiation time via an ASDC blueprint (Flow 2a), or, in later releases, based on being reconfigured by CLAMP via Policy, Flow 3a+3b.
...
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
{
"domain": "measurementsForVfScaling",
"metricsPerEventName": [
{
"eventName": "vFirewallBroadcastPackets",
"controlLoopSchemaType": "VNF",
"policyScope": "DCAE",
"policyName": "DCAE.Config_tca-hi-lo",
"policyVersion": "v0.0.1",
"thresholds": [
{
"closedLoopControlName": "ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a",
"version": "1.0.2",
"fieldPath": "$.event.measurementsForVfScalingFields.vNicUsageArray[*].receivedTotalPacketsDelta",
"thresholdValue": 300,
"direction": "LESS_OR_EQUAL",
"severity": "MAJOR",
"closedLoopEventStatus": "ONSET"
},
{
"closedLoopControlName": "ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a",
"version": "1.0.2",
"fieldPath": "$.event.measurementsForVfScalingFields.vNicUsageArray[*].receivedTotalPacketsDelta",
"thresholdValue": 700,
"direction": "GREATER_OR_EQUAL",
"severity": "CRITICAL",
"closedLoopEventStatus": "ONSET"
}
]
},
{
"eventName": "vLoadBalancer",
"controlLoopSchemaType": "VM",
"policyScope": "DCAE",
"policyName": "DCAE.Config_tca-hi-lo",
"policyVersion": "v0.0.1",
"thresholds": [
{
"closedLoopControlName": "CL-LBAL-TRAFFIC-SIG-FB480F95-A453-6F24-B767-FD703241AB1A",
"version": "1.0.2",
"fieldPath": "$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedBroadcastPacketsAccumulated",
"thresholdValue": 300,
"direction": "GREATER_OR_EQUAL",
"severity": "CRITICAL",
"closedLoopEventStatus": "ONSET"
}
]
},
{
"eventName": "Measurement_vGMUX",
"controlLoopSchemaType": "VNF",
"policyScope": "DCAE",
"policyName": "DCAE.Config_tca-hi-lo",
"policyVersion": "v0.0.1",
"thresholds": [
{
"closedLoopControlName": "ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e",
"version": "1.0.2",
"fieldPath": "$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value",
"thresholdValue": 0,
"direction": "EQUAL",
"severity": "MAJOR",
"closedLoopEventStatus": "ABATED
},
{
"closedLoopControlName": "ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e",
"version": "1.0.2",
"fieldPath": "$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value",
"thresholdValue": 0,
"direction": "GREATER",
"severity": "CRITICAL",
"closedLoopEventStatus": "ONSET"
}
]
}
]
} |
SME(s):
...
Business Sponsor(s):
provide contact information for the sponsor of this microservice. This should be the individual or organizations that are providing funding and/or are otherwise responsible for the lifecycle of the application code.
Contact | Role | Notes |
|---|---|---|
| ONAP TSC | Technical Steering Committee | This microservice was specifically developed for ONAP R0 and later enhanced for ONAP R1. |
Known Service Use Case(s):
provide a short description of any known service use cases to include the VFs involved, the end services impacted, etc.
...