Currently, when policies are updated via one of the PAP APIs, the entire list of policies is transmitted to the PDPs.  The communication between PAP and the PDPs should be enhanced so that only the changes need be transmitted.

After receiving the request from REST layer, PAP does the basic validation checks and then generate a PDP_UPDATE message with full list of policies for that particular PdpSubGroup and sends it to the PDPs through DMaaP. And PDPs on the other side scans through the list and identifies the policies that needs to be deployed or undeployed. Thereby making PDPs share the work which ideally PAP should be doing as an administrator component.

The aim is to send only the requested & validated policies that needs to be deployed or undeployed in PDPs. So that PDPs can remove the scanning logic and focus more on the "execution" than "administration". And it also reduces the data transmitting over DMaaP (network) and making the layer more robust and resilient.


Important Note:

  • The change needs to be designed in a way so that it doesn't break the currently working contract between PAP & PDPs. And allow sometime for PDPs to adapt and move to the newer (simpler) way of managing policies.
  • The change shouldn't affect the PDP registration flow. Because there we would like to continue sending the FULL list of policies that are currently supported in PdpSubGroup in which the PDP is getting registered.


Current thoughts/suggestions:

  1. Enhance PDP_UPDATE message to contain two new lists (deploy and undeploy), while maintaining its current full list of policies. When the PDPs support delta policies we can then remove the original full list.
  2. Enhance PDP_UPDATE message to contain one new list of policies to be updated, and a boolean value to determine whether to deploy/undeploy the policies while maintaining the full list of policies. When the PDPs support the delta policies we can remove the original list.


 

Current Status

At present, PAP has been updated so that when policies are updated, lists of delta policies  are sent to the various PDPs via the PDP_UPDATE message. Similar to before, PAP does the basic checks for validation when a request is received from the REST layer and then generates a PDP_UPDATE message for that particular PdpSubGroup and sends it to the PDPs through DMaaP. This time however, PAP sends a list of "policiesToBeDeployed" and another list of "policiesToBeUndeployed" along with the full list of policies, for now. The goal is to remove sending the full list of policies when all PDPs have been updated to accept delta policies. 

This eliminates the need for PDPs on the other side to scan through the full list of policies as before to determine what policies need to be deployed and undeployed. This is administration work and is now handled on the administration side i.e. PAP. 

APEX-PDP and DROOLS-PDP are the only PDPs which can handle delta policies at the moment. Note: Apex sends back a list of deployed policies via the PDP_STATUS message to PAP while DROOLS-PDP sends back a full list of policies to PAP via the PDP_STATUS message. 


  • No labels

6 Comments

  1. Jim Hahn

    Probably want option 1, as there may be both deploys & undeploys in a single PDP_UPDATE.

    1. Wayne Dunican

      I agree. Thank you for the feedback.

  2. Ram Krishna Verma

    Just checked the policy batch deployment api, seems we are taking both deployment and undeployment in a single API call. So it makes sense to generate a single PDP_UPDATE with 2 new lists for deployment and undeployment.

    However, the deployment list will contain the full tosca policies content. And the undeployment list can only contain the name and version of policies to be undeployed.

    1. Wayne Dunican

      Okay - sounds good. Thanks Ram.

  3. Jorge Hernandez

    Nice write up describing the problem Wayne Dunican . 

    I think number 1 may be easier to start with (as it gets a more complicated with a single list in #2 with all policies and deploys and undeploy flags, note that in PDPs undeploys are inferred by policy omission in in the pdp-update, and they are not explicitly mentioned in the message, and instead of a boolean, it may require a 3-value operation field with deploy, undeploy, and NA values).

    #1 also seems as the base for the final functionality as some times the full blown of policies must be sent from PAP, for example at PDP initialization and if we need to resync PDP and PAP for some reason, so in some specific cases the full blown list of policies must be sent by the PAP, for regular operation though in the final version the full blown policy list can be omitted, once all PDPs are migrated to delta updates.

    1. Wayne Dunican

      Thanks Jorge Hernandez- Ram Krishna Vermahelped with this write up also. 

      Option 1 changes have been made and are now up for review.