R6 Discussion on API support for Native Policy CRUD (W.I.P)

In essence, native/raw policy is custom policy type implementation which can be native policy rules (e.g. drools rules, xacml rules or apex rules) with or without configurable property values. If a set of native policy rules does not contain any configurable property, it becomes policy implementation which is directly executable in PDP engines. If it contains one or more configurable properties (e.g. marked as ${xxx}), it becomes a rule template. Its corresponding policy type defines those configurable properties and their values can be provided by particular TOSCA policy created off this policy type. In this scenario, executable policy implementation will be generated by combining TOSCA policy and this corresponding policy type implementation. The relationship among policy type, policy type implementation, TOSCA policy and policy implementation can be illustrated as below:

Taking drools rules as one example, current usecases.drl used in PDP-D is a default policy type implementation for operational policy. It is both a policy type implementation and policy implementation since it is a set of rules without any configurable property values to be populated and it can be directly run in PDP-D. However, some policy authors might prefer to compose their own drools rules in a different way even if they aim to implement the same or different use cases. Current policy framework does not provide an interface for such policy authors to bring in their own native rules. Hence, we need to come up with a solution in Frankfurt to bridge the gap.

Regarding this solution, I have two design options in my mind which will be explained in the following. In both options, we want to use PAP to deploy such native policies to appropriate PDP engines. Given drools rules will import some java classes and use them in action part to perform particular operations as well as in condition part to indicate the java models for I/O message events, we want to separate java class implementation and rule authoring. Java classes being used in drl rules need to be implemented separately and packaged into JARs which are deployed to internal nexus repo before implementing/deploying native policies.

Option 1: Everything being provided through policy design API (i.e. drl rules, java dependencies, drools controller configuration)

In this design option, native policy author needs to provide all necessaries in the payload of POST custom policy type impl call. The payload will use application/json or application/yaml as content-type. All the necessaries include native policy rules with or without configurable properties (e.g. drl with ${xxx}), java dependency descriptors (i.e. groupId, artifactId, version) and other configuration required by corresponding PDP engine to load/execute these new rules (e.g. drools controller configuration which includes controller-name, dmaap server, dmaap topics, java classes for serialization/deserialization, filter, custom serializer, etc.).

Policy type, type implementation and policy creation

Policy type, policy type implementation and policy are organized as hierarchical resources in RESTful path. Policy type implementation payload should include which policy type this implementation is associated with. Likewise, policy payload should include which policy type and which type implementation this policy is created off. If policy type implementation is not present in endpoint path, it means to use default type implementation for associated policy type. Basically, the RESTful resource hierarchy can be expressed as

/policytypes/{policyTypeName}/versions/{policyTypeVersion}/policytypeimpls/{policyTypeImplName}/versions/{policyTypeImplVersion}/policies/{policyName}/versions/{policyVersion}

Some endpoint paths are exemplified in the following table:

policy_type_impls:
  - onap.policies.controlloop.operational.sampleusecase1.Impl:
      version: 1.0.0
      type: onap.policies.controlloop.operational.Sampleusecase1
      type_version: 1.0.0
      pdp: drools
      description: Implementation of sample use case1 drools rules
      policy_body: "<policy body>"
      properties:
        dependencies:
          - groupId: org.onap.policy.sampledomain
            artifactId: policy-sample-usecase1
            version: 1.0.0-SNAPSHOT
        controller:
          name: sample-usecase1
          config_body: "<config body>"

  - onap.policies.controlloop.operational.sampleusecase2.Impl:
      version: 1.0.0
      type: onap.policies.controlloop.operational.Sampleusecase2
      type_version: 1.0.0
      pdp: drools
      description: Implementation of sample use case2 drools rules
      policy_body: "<policy body>"
      properties:
        dependencies:
          - groupId: org.onap.policy.sampledomain
            artifactId: policy-sample-usecase2
            version: 1.0.0-SNAPSHOT
        controller:
          name: sample-usecase2
          config_body: "<config body>"

tosca_definitions_version: tosca_simple_yaml_1_0_0
topology_template:
  policies:
    - operational.sampleusecase1:
        type: onap.policies.controlloop.operational.Sampleusecase1
        type_version: 1.0.0
        type_impl: onap.policies.controlloop.operational.sampleusecase1.Impl
        type_impl_version: 1.0.0
        version: 1.0.0 
        metadata: 
          policy-id: operational.sampleusecase1
        properties: null
    - operational.sampleusecase2:
        type: onap.policies.controlloop.operational.Sampleusecase2
        type_version: 1.0.0
        type_impl: onap.policies.controlloop.operational.sampleusecase2.Impl
        type_impl_version: 1.0.0
        version: 1.0.0
        metadata:
          policy-id: operational.sampleusecase2
        properties:
          ClosedLoopControlName: sample-usecase-1

tosca_definitions_version: tosca_simple_yaml_1_0_0
policy_types:
  onap.policies.controlloop.operational.Sampleusecase1:
    derived_from: onap.policies.controlloop.Operational
    description: a sample use case1 specific policy type 
    version: 1.0.0
  onap.policies.controlloop.operational.Sampleusecase2:
    derived_from: onap.policies.controlloop.Operational
    description: a sample use case2 specific policy type
    version: 1.0.0
    properties:
      ClosedLoopControlName:
        type: string
        required: true
        description: the controlloop name for sample use case2

Note that the preceding policy type implementation list contains two elements. One is for sample use case1 rules whereas the other is for sample use case2 rules. Here, sample use case 1 and 2 are two example use cases which use drools rules for decision making. Let's assume sample use case1 rules do not have configurable properties while sample use case 2 rules contain configurable properties (e.g. ${ClosedLoopControlName}), which justifies that in the corresponding policies created off, we can see "properties: null" for use case 1 and "properties: ClosedLoopControlName: xxx" for use case 2 as shown above.

Policy deployment

With policy type, type implementation and policy created, next step is that policy author needs to deploy this policy to make it run in appropriate PDP. Deploying this policy is as easy as calling policy deployment API exposed by PAP. In this design option, the payload of deployment API will include only policy ID and version (the same as current). If version is not present, the latest version will be used. Content-type would be either application/json or application/yaml. For example,

Content-type: application/json

{
  "policies" : [
    {
      "policy-id": "operational.sampleusecase1",
      "policy-version": 1
    },
    {
      "policy-id": "operational.sampleusecase2"
    }
  ]
}

Flow explanation

Policy type, type implementation and policy are all stored in policy DB. As a follow-up after POST policy type impl and policy API calls, native policy author needs to call PAP policy deployment API to deploy native policies to PDP engine (e.g. drools). PAP will pull the policy and corresponding type implementation from DB then send them to appropriate PDP engine. Taking drools as one example, after PDP-D receives native policy and its type implementation from PAP, it will first pull necessary dependency artifact JARs from internal nexus based on what specified in "properties.dependencies" then copy them to local m2 repo. Next, PDP-D will package native rules into a new JAR and copy to local m2 as well. At last, PDP-D will instantiate a new drools controller instance based on what provided in "properties.controller" then invoke telemetry API to spin up the new drools controller. 

Sequence diagram

Option 2: Native policy rules being provided through policy design API whereas peripheral configuration (i.e. java dependencies, drools controller configuration) being provided through PAP policy deployment API

In this design option, only native policy rules are being provided in POST policy type implementation payload via custom content-type (e.g. application/drl, application/xacml, application/apex). Other dependencies and configuration are being provided in PAP policy deployment API call. The basic idea of this design option is, in policy design phrase, native policy author just needs to focus on composing native policy rules to realize predefined decision making logic, without worrying about how to setup/configure peripheral stuffs like dmaap server/topics and new drools controller instance. After finishing up policy rule handcrafting and later in policy deployment phrase, policy author can start considering about how to setup/configure other things to make sure native rules can be running correctly in appropriate PDP engine. It implies that when PAP policy deployment API is invoked, necessary configurations required by PDP engine to correctly execute native rules need to be provided in the payload of this policy deployment POST call. Along with native policy and its type implementation, they will all be sent to PDP engine for execution.

Policy type, type implementation and policy creation

Endpoint paths will remain the same in this option except that content-type will be customized to PDP engine specific one.

Policy payload will remain the same as that in option 1. 

Policy deployment

In this design option, payload of policy deployement API needs to include not only policy ID and version but also peripheral configurations required by PDP to load/execute newly created native policy rules. Taking drools rules as one example, peripheral configurations would include, dependency descriptors (i.e. event/util java models), drools controller configurations (i.e. controller name, dmaap server and topics, java class for mapping messages coming under each topic, message filter, custom serializer if any, etc.). Again, content-type could be application/json or application/yaml. Below is one example of the payload:

Content-type: applicaiton/json

{
  "policies": [
    {
      "policy-id": "operational.sampleusecase1",
      "policy-version": 1,
      "dependencies": [
        {
          "groupId": "org.onap.policy.sampledomain",
          "artifactId": "policy-sample-usecase1",
          "version": "1.0.0-SNAPSHOT"
        }
      ],
      "controller": {
        "name": "sample-usecase1",
        "config_body": "<config body>"
      }
    },
    {
      "policy-id": "operational.sampleusecase2",
      "dependencies": [
        {
          "groupId": "org.onap.policy.sampledomain",
          "artifactId": "policy-sample-usecase2",
          "version": "1.0.0-SNAPSHOT"
        }
      ],
      "controller": {
        "name": "sample-usecase2",
        "config_body": "<config body>"
      }
    }
  ]
}

Flow explanation

TBA

Sequence diagram