In essence, a native /raw policy is a custom policy type implementation which can be native policy rules (e.g. drools /rule implementation for a specific PDP engine such as: drools DRL rules, xacml rules XML policies or apex rules) with or without configurable property values. If a set of native policy rules does not contain any configurable property, it becomes policy implementation which is directly executable in PDP engines. If it contains one or more configurable properties (e.g. marked as ${xxx}), it becomes a rule template. Its corresponding policy type defines those configurable properties and their values can be provided by particular TOSCA policy created off this policy type. In this scenario, executable policy implementation will be generated by combining TOSCA policy and this corresponding policy type implementation. The relationship among policy type, policy type implementation, TOSCA policy and policy implementation can be illustrated as below:
Taking drools rules as one example, current usecases.drl used in PDP-D is a default policy type implementation for operational policy. It is both a policy type implementation and policy implementation since it is a set of rules without any configurable property values to be populated and it can be directly run in PDP-D. However, some policy authors might prefer to compose their own drools rules in a different way even if they aim to implement the same or different use cases. Current policy framework does not provide an interface for such policy authors to bring in their own native rules. Hence, we need to come up with a solution in Frankfurt to bridge the gap.
Regarding this solution, I have two design options in my mind which will be explained in the following. In both options, we want to use PAP to deploy such native policies to appropriate PDP engines. Given drools rules will import some java classes and use them in action part to perform particular operations as well as in condition part to indicate the java models for I/O message events, we want to separate java class implementation and rule authoring. Java classes being used in drl rules need to be implemented separately and packaged into JARs which are deployed to internal nexus repo before implementing/deploying native policies.
Option 1: Everything being provided through policy design API (i.e. drl rules, java dependencies, drools controller configuration)
In this design option, native policy author needs to provide all necessaries in the payload of POST custom policy type impl call. The payload will use application/json or application/yaml as content-type. All the necessaries include native policy rules with or without configurable properties (e.g. drl with ${xxx}), java dependency descriptors (i.e. groupId, artifactId, version) and other configuration required by corresponding PDP engine to load/execute these new rules (e.g. drools controller configuration which includes controller-name, dmaap server, dmaap topics, java classes for serialization/deserialization, filter, custom serializer, etc.).
Policy type, type implementation and policy creation
Policy type, policy type implementation and policy are organized as hierarchical resources in RESTful path. Policy type implementation payload should include which policy type this implementation is associated with. Likewise, policy payload should include which policy type and which type implementation this policy is created off. If policy type implementation is not present in endpoint path, it means to use default type implementation for associated policy type. Basically, the RESTful resource hierarchy can be expressed as
/policytypes/{policyTypeName}/versions/{policyTypeVersion}/policytypeimpls/{policyTypeImplName}/versions/{policyTypeImplVersion}/policies/{policyName}/versions/{policyVersion}
JSON policies.
Taking drools rules as one example, current usecases.drl used in PDP-D is a default DRL implementation for Control Loop operational policies. It is both a TOSCA Policy Type implementation and a native drools rule implementation since it is a set of rules that directly run in the PDP-D supporting the TOSCA onap.policies.Operational Policy Type. However, some policy authors might prefer to compose their own drools rules for certain features supported by the usecases.drl such as: changing the treatment of incoming event messages, adding custom timeout logic, or adding extra processing logic. Current policy framework does not provide an interface for such policy authors to bring in their own native rules. Hence, we need to come up with a solution in Frankfurt to bridge the gap.
Taking XACML as another example, some policy authors may want to add custom guard XACML policies that do not require TOSCA
1. Native Rule Development Guidelines
1.1 Drools Native Rules
1.1.1 DRL development
use existing nexus(s) that hold the released artifacts currently supported by the runtime PDP-D engines.
develop in IDE of choice (eg Eclipse/intelliJ) to ensure it compile
1.1.2 JAR development
use development best practices/governance to test/deploy new and/or updated java artifacts to the nexus repo for drools PDP-D.
1.2 XACML XML
Use editor of choice - existing XACML github jars etc. to validate/test
Pam
1.3 APEX JSON
Use Apex command line tool to build the JSON.
2. Policy Lifecycle API CRUD Enhancements
In order to designate between native policy/rule types, the REST header "Content-Type" is configured for each PDP engines specific content.
PDP Engine | Content-Type | Description |
---|---|---|
PDP-D: Drools | application/vnd.onap.drools+text application/vnd.onap.drools.mvn+xml | Drools DRL text files. Question: Does Drools have a custom content-type already?? Maven XML dependency specification for a java artifact containing drools rules. Does maven have a custom content-type?? |
PDP-X: XACML | application/xacml+xml; version=3.0 | Per http://docs.oasis-open.org/xacml/xacml-rest/v1.0/cos01/xacml-rest-v1.0-cos01.html |
PDP-A: Apex | application/vnd.onap.apex+json | Apex JSON policy files. TBC with Apex team |
2.1 PDP-D Content-Types
chenfei
2.2 PDP-X Content-Types
pam
2.3 PDP-A Content-Types
chenfei
2.4 Endpoint Details
The Policy Lifecycle API will need to support new endpoints that consume these PDP specific Content-Type's as well as the ability to save them to the database.Some endpoint paths are exemplified in the following table:
HTTP Method | Endpoint | Content-Type | Description | Example |
---|---|---|---|---|
POST |
/ |
natives/{ |
application/yaml
application/json
Create new custom policy type implementation(s) for an existing policy type version
Retrieve custom policy type implementation(s) for an existing policy type version
application/yaml
application/json
Retrieve a particular version of policy type implementation for an existing policy type version
Update a particular version of policy type implementation for an existing policy type version
Delete a particular version of policy type implementation for an existing policy type version
application/yaml
application/json
application/yaml
application/json
Create new polic(ies) off existing policy type and type implementation
Retrieve polic(ies) created off existing policy type and type implementation
application/yaml
application/json
Retrieve a particular version of policy created off existing policy type and type implementation
Update a particular version of policy created off existing policy type and type implementation
Delee a particular version of policy created off existing policy type and type implementation
application/yaml
application/json
application/yaml
application/json
policy-id} | *see above | Creates a native policy using the given id. Returns the id and version. | 200
| |||
GET | /natives | application/json application/yaml | Returns a list of ID/Version of natives policies | 200
| ||
GET | /natives/{policy-id} | Get a list of versions | ||||
chenfei finish |
3. PAP Enhancements
PDP Engines must now register with the PAP the native Content-Type's they support in order for policies to be deployed by the PAP engine to the PDP's. This will require an additional parameter in the Group Deploy/Undeploy to list the supported Content-Type's for the PDP engine. The proposal is to add a field "supportedContentTypes".
3.1 Example of PDP Register
TODO - add example register from Drools PDP. Chenfei.
Code Block |
---|
show drools registration |
Question: Do we need a version attached to the native content-type? Might be easier to keep it around.
3.2 Example PDP Group Deploy
Code Block | ||||
---|---|---|---|---|
| ||||
{
"groups": [
{
"name": "defaultGroup",
"description": "The default group that registers all supported policy types and pdps.",
"pdpGroupState": "ACTIVE",
"properties": {},
"pdpSubgroups": [
{
"pdpType": "apex",
"supportedPolicyTypes": [
{
"name": "onap.policies.controlloop.operational.Apex",
"version": "1.0.0"
}
],
"supportedContentTypes": [
{
| ||||
Code Block | ||||
| ||||
policy_type_impls: - onap.policies.controlloop.operational.sampleusecase1.Impl: version: 1.0.0 type: onap.policies.controlloop.operational.Sampleusecase1 type_version: 1.0.0 pdp: drools description: Implementation of sample use case1 drools rules policy_body: "<policy body>" properties: dependencies: - groupId"name": org"application/vnd.onap.policy.sampledomainapex+json", artifactId: policy-sample-usecase1 "version": "1.0.0-SNAPSHOT" controller: name: sample-usecase1} config_body: "<config body>" - onap.policies.controlloop.operational.sampleusecase2.Impl: version: 1.0.0 ], type: onap.policies.controlloop.operational.Sampleusecase2 type_version: 1.0.0 pdp"policies": drools[], description: Implementation of sample use case2 drools rules policy_body: "<policy body>" "currentInstanceCount": 0, properties: dependencies: "desiredInstanceCount": 1, - groupId: org.onap.policy.sampledomain artifactId"properties": policy-sample-usecase2{}, version: 1.0.0-SNAPSHOT controller"pdpInstances": [ name: sample-usecase2 config_body: "<config body>" | ||||
Code Block | ||||
| ||||
tosca_definitions_version: tosca_simple_yaml_1_0_0 topology_template: policies: { - operational.sampleusecase1: type: onap.policies.controlloop.operational.Sampleusecase1 type_version: 1.0.0 type_impl: onap.policies.controlloop.operational.sampleusecase1.Impl "instanceId": "apex_35", type_impl_version: 1.0.0 version: 1.0.0 metadata"pdpState": "ACTIVE", policy-id: operational.sampleusecase1 properties: null - operational.sampleusecase2: "healthy": "HEALTHY", type: onap.policies.controlloop.operational.Sampleusecase2 type_version: 1.0.0 type_impl: onap.policies.controlloop.operational.sampleusecase2.Impl"message": "Pdp Heartbeat" type_impl_version: 1.0.0 } version: 1.0.0 metadata:] policy-id: operational.sampleusecase2 }, properties: { ClosedLoopControlName: sample-usecase-1 | ||||
Code Block | ||||
| ||||
tosca_definitions_version: tosca_simple_yaml_1_0_0 policy_types: onap.policies.controlloop.operational.Sampleusecase1: derived_from: onap.policies.controlloop.Operational"pdpType": "drools", description: a sample use case1 specific policy type version: 1.0.0 onap.policies.controlloop.operational.Sampleusecase2: "supportedPolicyTypes": [ derived_from: onap.policies.controlloop.Operational description: a sample use case2 specific policy type version: 1.0.0 properties:{ ClosedLoopControlName: type: string required: true description: the controlloop name for sample use case2 |
Note that the preceding policy type implementation list contains two elements. One is for sample use case1 rules whereas the other is for sample use case2 rules. Here, sample use case 1 and 2 are two example use cases which use drools rules for decision making. Let's assume sample use case1 rules do not have configurable properties while sample use case 2 rules contain configurable properties (e.g. ${ClosedLoopControlName}), which justifies that in the corresponding policies created off, we can see "properties: null" for use case 1 and "properties: ClosedLoopControlName: xxx" for use case 2 as shown above.
Policy deployment
With policy type, type implementation and policy created, next step is that policy author needs to deploy this policy to make it run in appropriate PDP. Deploying this policy is as easy as calling policy deployment API exposed by PAP. In this design option, the payload of deployment API will include only policy ID and version (the same as current). If version is not present, the latest version will be used. Content-type would be either application/json or application/yaml. For example,
Content-type: application/json
Code Block | ||
---|---|---|
| ||
{ "policies" : [ { "name": "onap.policies.controlloop.Operational", "version": "1.0.0" } ], "supportedContentTypes": [ { "name": "application/vnd.onap.drools+text", "policy-idversion": "operational1.0.sampleusecase10", "policy-version": 1 }, { "policy-id": "operational.sampleusecase2" }{ ] } |
Flow explanation
Policy type, type implementation and policy are all stored in policy DB. As a follow-up after POST policy type impl and policy API calls, native policy author needs to call PAP policy deployment API to deploy native policies to PDP engine (e.g. drools). PAP will pull the policy and corresponding type implementation from DB then send them to appropriate PDP engine. Taking drools as one example, after PDP-D receives native policy and its type implementation from PAP, it will first pull necessary dependency artifact JARs from internal nexus based on what specified in "properties.dependencies" then copy them to local m2 repo. Next, PDP-D will package native rules into a new JAR and copy to local m2 as well. At last, PDP-D will instantiate a new drools controller instance based on what provided in "properties.controller" then invoke telemetry API to spin up the new drools controller.
Sequence diagram
Option 2: Native policy rules being provided through policy design API whereas peripheral configuration (i.e. java dependencies, drools controller configuration) being provided through PAP policy deployment API
In this design option, only native policy rules are being provided in POST policy type implementation payload via custom content-type (e.g. application/drl, application/xacml, application/apex). Other dependencies and configuration are being provided in PAP policy deployment API call. The basic idea of this design option is, in policy design phrase, native policy author just needs to focus on composing native policy rules to realize predefined decision making logic, without worrying about how to setup/configure peripheral stuffs like dmaap server/topics and new drools controller instance. After finishing up policy rule handcrafting and later in policy deployment phrase, policy author can start considering about how to setup/configure other things to make sure native rules can be running correctly in appropriate PDP engine. It implies that when PAP policy deployment API is invoked, necessary configurations required by PDP engine to correctly execute native rules need to be provided in the payload of this policy deployment POST call. Along with native policy and its type implementation, they will all be sent to PDP engine for execution.
Policy type, type implementation and policy creation
Endpoint paths will remain the same in this option except that content-type will be customized to PDP engine specific one.
...
application/drl
application/xacml
application/apex
...
Create new custom policy type implementation(s) for an existing policy type version
Retrieve custom policy type implementation(s) for an existing policy type version
...
application/drl
application/xacml
application/apex
...
Retrieve a particular version of policy type implementation for an existing policy type version
Update a particular version of policy type implementation for an existing policy type version
Delete a particular version of policy type implementation for an existing policy type version
...
application/drl
application/xacml
application/apex
...
application/yaml
application/json
...
Create new polic(ies) off existing policy type and type implementation
Retrieve polic(ies) created off existing policy type and type implementation
...
application/yaml
application/json
...
Retrieve a particular version of policy created off existing policy type and type implementation
Update a particular version of policy created off existing policy type and type implementation
Delee a particular version of policy created off existing policy type and type implementation
...
application/yaml
application/json
...
application/yaml
application/json
...
Policy payload will remain the same as that in option 1.
Policy deployment
In this design option, payload of policy deployement API needs to include not only policy ID and version but also peripheral configurations required by PDP to load/execute newly created native policy rules. Taking drools rules as one example, peripheral configurations would include, dependency descriptors (i.e. event/util java models), drools controller configurations (i.e. controller name, dmaap server and topics, java class for mapping messages coming under each topic, message filter, custom serializer if any, etc.). Again, content-type could be application/json or application/yaml. Below is one example of the payload:
Content-type: applicaiton/json
Code Block | ||
---|---|---|
| ||
{ "policies": [ { "policy-id": "operational.sampleusecase1", "policy-version": 1, "dependencies": [ { "name": "application/vnd.onap.drools.mvn+xml" "version": "1.0.0" } ], "policies": [], "currentInstanceCount": 0, "desiredInstanceCount": 1, "properties": {}, "pdpInstances": [ { "instanceId": "dev-policy-drools-0", "pdpState": "ACTIVE", "healthy": "HEALTHY" } ] }, { "pdpType": "xacml", "supportedPolicyTypes": [ { "name": "onap.policies.controlloop.guard.FrequencyLimiter", "version": "1.0.0" }, { "name": "onap.policies.controlloop.guard.MinMax", "version": "1.0.0" }, { "name": "onap.policies.controlloop.guard.Blacklist", "version": "1.0.0" }, { "name": "onap.policies.controlloop.guard.coordination.FirstBlocksSecond", "version": "1.0.0" }, { "name": "onap.Monitoring", "version": "1.0.0" }, { "name": "onap.policies.monitoring.cdap.tca.hi.lo.app", "version": "1.0.0" }, "groupId": "org.onap.policy.sampledomain", "artifactId": "policy-sample-usecase1", { "versionname": "1.0.0-SNAPSHOT" onap.policies.monitoring.dcaegen2.collectors.datafile.datafile-app-server", } ], "controller": { "nameversion": "sample-usecase1",1.0.0" "config_body": "<config body>" } }, { "policy-id": "operational.sampleusecase2", "dependencies": [ { "groupId": "org.onap.policy.sampledomain", "artifactId": "policy-sample-usecase2", "version": "1.0.0-SNAPSHOT" } ], "controller": { "name": "sample-usecase2", "config_body": "<config body>" } } ] } |
Flow explanation
The flow will be almost the same as in option 1. The only difference is, PAP will not only send policy and its type implementation but also peripheral configs to PDP as those configs come from deployment API payload instead of embedded in policy type implementation.
Sequence diagram
"name": "onap.policies.monitoring.docker.sonhandler.app",
"version": "1.0.0"
},
{
"name": "onap.policies.optimization.AffinityPolicy",
"version": "1.0.0"
},
{
"name": "onap.policies.optimization.DistancePolicy",
"version": "1.0.0"
},
{
"name": "onap.policies.optimization.HpaPolicy",
"version": "1.0.0"
},
{
"name": "onap.policies.optimization.OptimizationPolicy",
"version": "1.0.0"
},
{
"name": "onap.policies.optimization.PciPolicy",
"version": "1.0.0"
},
{
"name": "onap.policies.optimization.QueryPolicy",
"version": "1.0.0"
},
{
"name": "onap.policies.optimization.SubscriberPolicy",
"version": "1.0.0"
},
{
"name": "onap.policies.optimization.Vim_fit",
"version": "1.0.0"
},
{
"name": "onap.policies.optimization.VnfPolicy",
"version": "1.0.0"
}
],
"supportedContentTypes": [
{
"name": "application/xacml+xml; version=3.0",
"version": "1.0.0"
}
],
"policies": [],
"currentInstanceCount": 1,
"desiredInstanceCount": 1,
"properties": {},
"pdpInstances": [
{
"instanceId": "dev-policy-policy-xacml-pdp-558c478477-g85jl",
"pdpState": "ACTIVE",
"healthy": "HEALTHY"
}
]
}
]
}
]
} |
3.3 Deploy/Undeploy API
Simple vs Fully - what changes are needed if any?? Chenfei
4. PDP Changes
Each PDP will need to be able to support native policies being deploy/undeployed to it as done today.
4.1 Drools PDP
Drools PDP will need to be able to ingest a DRL directly into the controller. TBD - how to find/select the right controller. Or instantiate
4.2 XACML PDP
XACML PDP will need to be able to ingest a XACML XML directly. One suggestion is to create an application specifically for the XACML natives rules by default. The opportunity exists where a policy designer could create a specific application that supports native XACML policies (with or without TOSCA Policy Types as an option) and uses the grouping of PDPs to differentiate itself from the default XACML native rule application. The XACML PDP should also be enhanced to support configuring of applications in order to provide flexibility to the policy designers as to where all of its possible policy types and content types are deployed.
With regards to the Decision API supported by XACML, that api can be enhanced to support XACML XML requests/responses directly.
TODO: Show these scenarios Pam will do this.
4.3 Apex PDP
Apex PDP will need to be able to ingest custom Apex JSON policies. TBC with that team - may already be well-supported.