Summary

The Validation Service is configured with a set of explicitly defined Rules which are applied to entities in order to determine Data Integrity.
Whenever an entity instance is validated, a specific set of rules are determined and sequentially executed on a subset of the received entity.
Each rule is defined such that it detects a specific Data Integrity violation.
For each entity that is validated a single Validation Result is created containing a set of violations derived by the rules that have failed validation.

Each rule requires input argument(s). The rule's input values are read from attributes (properties) of the individual entity instance. An expression is applied to these attribute values to determine the success or failure of the validation. The rule expression can be a simple value check or instead use complex dynamic programming language constructs.

The format of the Rules configuration allows rules to be defined once and then used by multiple entity types.

Rules

A rule is a named function that accepts one or more arguments and returns a Boolean value indicating whether validation was successful. If the evaluation is false then this indicates that the set of argument value(s) is invalid.

Error text can be expanded with runtime values.  To use this functionality, the rule must return a tuple containing a Boolean and a list of arguments (see example below)

Configuration

Rules are categorized firstly by event type (e.g. AAI event, POA event), then by entity type or pre-configured index.  The rules are defined in one or more text files using the suffix *.groovy which are stored by event type.  For example, POMBA deployments use event type "poa-event" and rules for this event are stored in [validation-base-dir]/bundleconfig/etc/rules/poa-event/

To correctly define a rule, the following properties are required:

• a rule name that is unique within the validation service
• a set of named attributes (arguments)
• a validation expression, written in the Groovy programming language, that uses the named attributes and evaluates to produce a Boolean value
• meta-data which will appear in the violation details, including
  • category
  • severity
  • errorText

Examples

Simple Rule

entity {
  name 'POA-EVENT'
  indexing {
    indices 'default-rules'
  }
  validation {
    useRule {
      name 'vnf-name'
      attributes 'context-list.sdc.vfList[*].name'
    }
  }
}


rule {
    name        'vnf-name'
    category    'INVALID_NAME'
    description 'Invalid naming convention'
    errorText   'Invalid name - attribute does not match xxxxxnnnvbc (where x = alphanumeric and n = numeric)'
    severity    'MINOR'
    attributes  'name'
    validate    'name != null && name.matches("[a-z,0-9]{5}[0-9]{3}vbc")'
}

Complex Rule

The following example defines a rule that :

• accepts two attributes
• uses expandable error text
• uses a triple-quoted validate section to allow multiple lines
• defines multiple closures

entity {
  name 'POA-EVENT'
  indexing {
    indices 'default-rules'
  }
  validation {
    useRule {
      name 'NDCB-AAI-attribute-comparison'
      attributes 'context-list.ndcb.vfList[*].vfModuleList[*]', 'context-list.aai.vfList[*].vfModuleList[*]'
    }
  }
}


rule {
  name        'NDCB-AAI-attribute-comparison'
  category    'Attribute Mismatch'
  description 'Verify that all attributes in Network-Discovery are the same as in AAI'
  errorText   'Error found with attribute "{0}"; value "{1}" does not exist in Network-Discovery'
  severity    'ERROR'
  attributes  'ndcbItems', 'aaiItems'
  validate    '''
        Closure<java.util.Map> getAttributes = { parsedData ->
          java.util.Map attributeMap = new java.util.HashMap()

          def isAttributeDataQualityOk = { attribute ->
            attribute.findResult{ k, v -> if(k.equals("dataQuality") ) {return v.get("status")}}.equals("ok")
          }

          def addToMap = { attrKey, attrValue ->
            java.util.Set values = attributeMap.get("$attrKey")
            if(values == null) {
              values = new java.util.HashSet()
              attributeMap.put("$attrKey", values)
            }
            values.add("$attrValue")
          }

          def addAttributeToMap = { attribute ->
            if(isAttributeDataQualityOk(attribute)) {
              String key, value
              attribute.each { k, v ->
                if(k.equals("name")) {key = "$v"}
                if(k.equals("value")) {value = "$v"}
              }
              addToMap("$key", "$value")
            }
          }

          def processKeyValue = { key, value ->
            if(value instanceof java.util.ArrayList) {
              if(key.equals("attributeList")) {
                value.each {
                  addAttributeToMap(it)
                }
              }
            } else if(!(value instanceof groovy.json.internal.LazyMap)) {
              // only add key-value attributes, skip the rest
              addToMap("$key", "$value")
            }
          }

          if(parsedData instanceof java.util.ArrayList) {
            parsedData.each {
              it.each { key, value -> processKeyValue(key, value) }
            }
          } else {
            parsedData.each { key, value -> processKeyValue(key, value) }
          }
          return attributeMap
        }

        def slurper = new groovy.json.JsonSlurper()
        java.util.Map ndcb = getAttributes(slurper.parseText(ndcbItems.toString()))
        java.util.Map aai = getAttributes(slurper.parseText(aaiItems.toString()))

        boolean result = true
        List<String> details = new ArrayList<>();
        ndcb.any{ ndcbKey, ndcbValueList ->
          def aaiValueList = aai.get("$ndcbKey")
          aaiValueList.each{ aaiValue ->
            if(!ndcbValueList.any{ it == "$aaiValue" }) {
              result = false
              details.add("$ndcbKey")
              details.add("$aaiValue")
            }
          }
          if(result == false) {
            // break out of 'any' loop
            return true
          }
        }
        return new Tuple2(result, details)
        '''
}

Data-Dictionary rule

The following example defines a rule that uses the data-dictionary interfaced defined here.

By default, the URI template is configured as "/commonModelElements/{0}~{1}~1.0/validateInstance"

With the arguments used for calling validate() below, the resulting URL would be: [ddict-host:port]/commonModelElements/instance~vfModuleNetworkType~1.0/validateInstance

And the body would be:  {"type" : "some-value"}

entity {
  name 'POA-EVENT'
  indexing {
    indices 'default-rules'
  }
  validation {
    useRule {
      name 'Data-Dictionary validate VF type'
      attributes 'context-list.ndcb.vfList[*].vfModuleList[*].networkList[*].type'
    }
  }
}


rule {
    name        'Data-Dictionary validate VF type'
    category    'INVALID_VALUE'
    description 'Validate all VF type values against data-dictionary'
    errorText   'VF type [{0}] failed data-dictionary validation: {1}'
    severity    'ERROR'
    attributes  'typeList'
    validate    '''
        boolean success = true
        List<String> details = new ArrayList<>()
        typeList.any {
            if(!success) {
                // break out of 'any' loop
                return false
            }
            def result = org.onap.aai.validation.ruledriven.rule.builtin.DataDictionary.validate("instance", "vfModuleNetworkType", "type", "$it")
            if(!result.isEmpty()) {
                success = false
                details.add("$it")
                details.add("$result")
            }
        }
        return new Tuple2(success, details)
        '''
}

Entity Configuration

The entity configuration element defines which rules are applied to a specific entity type. The configuration is comprised of the following properties:

The validation comprises a set of useRule elements. Each specifies a rule to be applied to the entity.

useRule Configuration

This element is repeated within the validation element as illustrated in the example below. The following properties may be defined:

Example entity

entity {
	type 'complex'
	validation {
		useRule {
			name 'CLLI'
			attributes 'physical-location-id'
		}
		useRule {name 'complex is related to 1 oam-network' }
		useRule {
			name 'if a customer is related to an oam-network then oam-network.network-name must match naming convention'
			attributes 'relationship-list.relationship[*]'
		}
	}
}

Indexing

The POMBA solution intends to use runtime entity values to determine which rules to execute.  Currently, these attributes are defined as model-version-id and model-invariant-id.

In the following example, three entities are defined

entity {
   name 'POA-EVENT'
   indexing { indices 'version-1,invariant-1' }
   validation {
      useRule { name 'rule-1' }
   }
}


entity {
   name 'POA-EVENT'
   indexing { indices 'version-2,invariant-2' }
   validation {
      useRule { name 'rule-2' }
   }
}


entity {
   name 'POA-EVENT'
   indexing { indices 'default-rules' }
   validation {
      useRule { name 'default-rule' }
   }
}

Rules are determined based on the incoming event's values

Configuration

The event type and attributes must be pre-configured. The attributes are defined as JSON path expression within an event entity.

Using the following configuration

rule.indexing.events=POA-EVENT
rule.indexing.exclude.oxm.validation=POA-EVENT
rule.indexing.key.attributes=$.poa-event.modelVersionId,$.poa-event.modelInvariantId
rule.indexing.default.key=default-rules

And this event 

{
   "event-header": { ... }
   "entity": {
      "poa-event": {
         "modelVersionId": "version-1",
         "modelInvariantId": "invariant-1"
      }
      "entity-payload": { ... }
   }
}

Would result with rule-1 being executed on the entity payload.

Entity attributes

The attributes property value is a comma-separated list of strings. Each string identifies an attribute value to be read from the entity instance object.

There are two options for defining how the attribute values are read from the JSON representation of the entity.

Option A - define JSON path expressions in the rule section

The first option is to directly specify a JSON path within the rule. For example:

attributes 'relationship-list.relationship[*].related-to'

This definition instructs the rule to read multiple related-to values from the entity. The dot notation is used to navigate the JSON child object hierarchy. The [*] notation indicates that there are multiple values. The syntax is fully described here.

When the attribute value (in this case a collection) is referenced within the validate expression the leading parts (using the dot notation) are stripped. A sample valid expression is:

validate 'related-to != null && related-to.contains("complex")'

Option B - use named identifiers in the rule section

The second option is to use an attribute identifier e.g. field1. This requires the useRule section to define the actual path to the attribute value(s).
There are two benefits to this approach. Firstly, the validate expression can be shortened (i.e. by using a shorter string for the attribute). Secondly, the rule can then be applied to multiple different entity attributes.

When defining a rule it is important to specify attribute identifiers that can be directly replaced within the validate expression. Avoid using reserved words such as "null".
Only the following text characters are valid for an attribute name: any alphanumeric (a-z A-Z 0-9) and the set of special characters .*[]-_