Persist the cache of policy deployment status in the DB so other PAPs can read it.

JIRA:  POLICY-2648 - Getting issue details... STATUS


To satisfy the requirement to generate notifications when policies are fully deployed/undeployed, it is necessary to track the policy deployment/undeployment status for each PDP.  With this in mind, two options present themselves:

  1. Create a new JPA data structure to manage the information; or
  2. Augment the existing PdpGroup data structure to maintain the additional information.  This could be done by changing the list of policies to a list of policy-info structures, where each object contained:
    • policy name and version
    • flag indicating whether the policy is being deployed or undeployed
    • list of PDPs for which PAP is still awaiting the outcome of the PDP-UPDATE message


An advantage of approach #2 is that it's easier to transactionalize updates to the PdpGroup along with the list of waiting PDPs.  One downside is the extra amount of data that must be retrieved when a PdpGroup is extracted from the DB.

One complicating factor of #2 is that the same class, PdpGroup is used as both the data store structure and the PAP REST API structure.  If the same structure were to continue to be used, then the extra data fields would have to be handled (e.g., discarded on input from REST).  On the other hand, it could provide a mechanism for clients of the REST API to determine the actual PDP deployment status of each policy.

  • No labels

3 Comments

  1. Jim Hahn

    I'm currently pursuing option #2 by modifying the Pdp class, adding lists of policies awaiting deployment & undeployment, as well as a list of policies that failed undeployment.  (Note: no failure list is needed for deployment, as failures immediately trigger an undeployment of the policy.)  I believe this can be done without breaking any of the interfaces.  Rules/pseudocode for use:

    • REGISTRATION: When a new PDP is registered with a subgroup:
      • The awaitingDeployment list will be populated with all policies in the associated PdpSubgroup
      • A deployment notification is generated indicating a change in the success vs awaiting counts
    • DEREGISTRATION: When a PDP is removed from a subgroup:
      • A deployment notification is generated for any policy listed in the Pdp's awaitingDeployment list that does not appear within any other Pdp's awaitingDeployment list
      • An undeployment notification is generated for any policy listed in the Pdp's awaitingUndeployment or failedUndeployment lists that does not appear within any other Pdp's awaitingUndeployment list (the other Pdps' failedUndeployment list is not examined)
      • For any policy that appeared within the Pdp's awaitingUndeployment list:
        • If it does not appear within any other's Pdp's awaitingUndeployment list then remove it from all Pdp's failedUndeployment list
    • DEPLOY: When a new policy is deployed to a subgroup:
      • The policy is added to the awaitingDeployment list of each Pdp associated with the PdpSubgroup
      • The policy is also removed from each Pdp's awaitingUndeployment and failedUndeployment lists, if it happens to appear there
    • UNDEPLOY: When an existing policy is undeployed from a subgroup:
      • The policy is added to the awaitingUndeployment list of each Pdp associated with the PdpSubgroup
      • The policy is also removed from each Pdp's awaitingDeployment and failedUndeployment lists, if it happens to appear there
    • DEPLOY SUCCESS: When a policy is successfully deployed to a PDP and it was still in that Pdp's awaitingDeployment list:
      • The policy is removed from the list
      • If the policy no longer appears in any of the Pdps' lists for that subgroup, then a "deploy successful" notification is generated
    • UNDEPLOY SUCCESS: When a policy is successfully undeployed from a PDP and it was still in that Pdp's awaitingUndeployment list:
      • The policy is removed from the list
      • If the policy no longer appears in any of the Pdps' awaitingUndeployment lists for that subgroup, then an undeployment notification is generated
        • The policy is also removed from any Pdps' failedUndeployment lists
    • DEPLOY FAILED: When a PDP fails to deploy a policy and it was still in that Pdp's awaitingDeployment list:
      • The steps associated with the "UNDEPLOY" rule are performed
      • A "failed deployment" notification is also generated
    • UNDEPLOY FAILED: When a PDP fails to undeploy a policy and it was still in that Pdp's awaitingUndeployment list:
      • The policy is removed from the awaitingUndeployment list
      • The policy is added to the Pdp's failedUndeployment list
      • A "failed undeployment" notification is generated
      • If the policy no longer appears within any of the Pdps' awaitingUndeployment lists, then it is also removed the Pdps' failedUndeployment lists
      • Note: the policy is not re-added to the PdpSubgroup

  2. Jim Hahn

    Feedback was that option #1 is preferable, to isolate the impacts.

    With that in mind, rather than creating a complex structure just to track the state of policy deployments, the simple proposal would be to create a single table whose records contain:

    • pdp-group
    • pdp-subgroup
    • pdp-name
    • policy-type id & version
    • policy id & version
    • status - enumeration: deployed, awaiting deployment, failed deployment, undeployed, awaiting undeploment, failed undeployment
    • primary key: pdp-name, policy-id, policy-version
    • secondary key: pdp-group

    Note: once all records for a given policy have been marked "undeployed" or "failed undeployment", those records will be deleted, as the corresponding policy no longer appears within the PdpGroup records.

    Performance concern: Do we have a guesstimate of how many records there could be for a given group?  If it's many, then we may want to eliminate records that are marked "deployed", as any PDP that appears in the PdpSubGroup containing the policy would be implicitly deployed, unless there's a record for one of the other status values (i.e., awaiting, failed).  However, that would make some of the cases more complicated, as the list of successfully deployed PDPs would now have to be queried from the PdpGroup (which may defeat the purpose, because now all policy IDs within the group will have to be loaded, as part of the group structure).  Proposal is to keep it simple for now.

    This requires an additional rules:

    • DELETE GROUP: When a PdpGroup is deleted:
      • There will be no entries for deployed policies, only undeployed policies (potentially)
      • "Undeploy" records will not be deleted at this time, but will be deleted later when all PDPs report on the result
    • DELETE SUBGROUP: When a PdpSubGroup is deleted:
      • Similar logic as DELETE GROUP

    1. Ram Krishna Verma

      Looks good Jim. Few points from my side:

      • May be we can add "timestamp" in the table to give us the idea of what happened when.
      • Regarding performance, I feel we can keep all the records (deployed, failed, undeployed etc) to keep it simple, give us some audit capability and tracing present/past actions. In order to avoid the table growing too big, we can have some purging based on size or time. Also, we can put some indexing to make the query faster. 
      • Primary key with only (pdp-group, pdp-name) will also work if pdp-name is same as pdpInstanceId.