You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 11 Next »

Requirements on PDP - implement the pub-sub for multiple policies per each component

  1. maintain the database of subscribers with the
    1. list of generic policy-filters (each policy-filter is the resource from the request json to /decision/v1 API) Policy Design and API Flow for Model Driven Control Loop
    2. subscriber_topic for Message Router of DMaaP that uniquely identifies the component instance like “policies_DCAE_tca_<service-component-name>”
      1. <service-component-name> uniquely identifies the component in DCAE
  2. on policy push/delete from PAP
    1. select all subscribers that match to the pushed/deleted policies by any policy-filter
    2. for each affected subscriber retrieve all the latest snapshot of policies
    3. notify each subscriber_topic separately with the latest snapshot of policies by sending the message to Message Router of DMaaP with the topic=subscriber_topic
  3. there might be a need for some logic to identify the stale subscriptions by checking with the Message Router of DMaaP on the timestamps of the latest delivered/undelivered message per ach topic PDP has the subscription on

Requirements on Message Router of DMaaP

  1. persistently (up to 7 days) deliver all the policy-update notifications per subscriber_topic
  2. component will listen for the subscriber_topic of the policy-update notification from MR of DMaaP with long-collect-polling time like 15 seconds to grab the push notification

Requirements on component

Less requirements on component - policy-handler to do the subscription with PDP for the component

  1. on startup – get the subscriber_topic=“policies_DCAE_tca_<service-component-name>” from config-binding-service

  2. listen for subscriber_topic of policy-update notification from MR of DMaaP with long-collect-polling time like 15 seconds to grab the push notification

  3. on receiving the policy-update pushed notification from DMaaP, handle the policy-update that contains the full snapshot of all the policies that match to the component

    1. calculate the delta between the current set of policies and the received shapshot of policies

More requirements on component (initial proposal) - too much

  1. on startup – subscribe to PDP with the policy-filter the component is interested in and the subscriber_topic=“policies_DCAE_tca_<service-component-name>” that that uniquely identifies the component instance
  2. listen for subscriber_topic of policy-update notification from MR of DMaaP with long-collect-polling time like 15 seconds to grab the push notification

  3. on receiving the policy-update pushed notification from DMaaP, handle the policy-update that contains the full snapshot of all the policies that match to the component

    1. calculate the delta between the current set of policies and the received shapshot of policies

  4. n stop, unsubscribe subscriber_topic from PDP


See inside the diagram for more details, data structures, and flow steps

uml for diagram on policy flow between PDP and DCAE Component 
@startuml r5_proposed_policy_update_flow
allowmixing
scale 4096*4096


skinparam title {
    FontSize 24
    FontColor Blue
    FontStyle Bold
    BorderRoundCorner 15
    BorderThickness 2
}

skinparam roundcorner 20

skinparam component {
    BackgroundColor Snow
}

skinparam note {
    FontColor Black
    BackgroundColor azure
}

title = R5 El Alto proposal for \n policy flows between the new PDP and DCAE component \n PDP does the pub-sub for policies per component \n %date[yyyy-MM-dd HH:mm]%

package "<&dollar> Policy-Engine" as policy_engine #88ff88 {
    component "<&dollar> new Policy-Access-Point (<b>new PAP</b>)" as PAP
    component "<&dollar> new Policy-Decision-Point (<b>new PDP</b>)\n==pub-sub==\n maintains the table of policy-update subscribers \nwith <&crop> policy-filters and subscriber_topic\n--subscribe--\n[0.3.2] insert-update the record for the <b>subscriber_topic</b>\n[0.3.3] return policies that match to policy-filters\n--unsubscribe--\n[99.2] delete the record for the <b>subscriber_topic</b>\n==[1] on policy push/delete from PAP==\n[1.0] iterate through all subscribers and policy-filters on each subscriber\n[1.1] select all added/updated/removed policies \nthat match to any <&crop> policy-filter on the subscriber_topic\n[1.2] notify each subscriber_topic separately \nwith full policy_body of each updated/added policy \nand policy-id of each removed policy" as PDP

    database "<&list> <b>policy-update subscribers</b> (table in database)\n..unique topic per each component instance..\n<b>subscriber_topic</b> TEXT <&key> PK -- "policies_DCAE_tca_<service-component-name>"\n..\nONAPName TEXT -- "DCAE"\nONAPComponent TEXT -- "tca"\nONAPInstance TEXT -- "<service-component-name>"\n..list of policy-filters on subscriber_topic (component)..\n<&list> <&crop> <b>policy-filters</b> JSON -- list of the <b>"resource"</b> objects of /decision/v1: \n[{"policy-id": ["onap.scaleout.tca", "onap.restart.tca"]}, ...]" as policy_update_subscribers
}

package "<&dollar> DCAE" as DCAE {
    package "<&signpost> DCAE-Controller" as DCAE_Controller #eeffee {
        package "<&cog> deployment" #eeeeff {
            component "<&aperture> <b>deployment_handler</b>\n--\n[0] install component through cloudify for the blueprint+inputs\n--\n[99] uninstall component through cloudify" as deployment_handler
            package "<b>cloudify</b>" as cloudify_server #00ffff {
                control cloudify
                component "<&aperture> <b>k8s_plugin decorated with @policies_gather</b>\n--[0] install component--\n[0.1] <b>@policies_gather</b>: gather policy-filters assigned to component in blueprint+inputs\n[0.2] <b>@policies_gather</b>: save multiple policy-filters for component into consul-kv as \n<service-component-name>:policies/filters/<filter_id>:{"policy-id": ["onap.scaleout.tca", "onap.restart.tca"]}\n[0.3] <b>k8s_plugin</b>: install and start component\n--[99] uninstall component--\n[99.1] <b>@policies_gather</b>: delete records from consul-kv\n[99.2] <b>k8s_plugin</b>: stop and uninstall component" as k8s_plugin
            }
        }
        database "<&target> <b>consul-kv</b>\n--multiple policy-filters records (<b>"resource"</b> objects of /decision/v1 API in PDP) per component--\n<service-component-name>:<b>policies/filters</b>/<filter_id>:{"policy-id": ["onap.scaleout.tca", "onap.restart.tca"]}\n<service-component-name>:<b>policies/subscriber_topic</b>='policies_DCAE_tca_<service-component-name>'\n--config of policy_engine_client in DCAE - populated on install of DCAE platform--\n<b>policy_engine_client</b>: {url, headers, tls-settings, timeout-settings, etc.}" as consul_kv
        component "<&aperture> <b>config_binding_service</b>" as config_binding_service
    }
    component "<&target> <b>DCAE component like TCA</b>\n--[0.3] on startup - subscribe--\n[0.3.1] get policy_engine_client config, subscriber_topic and \nall policy-filters along with the config from CBS\n[0.3.2] subscribe with PDP for policy-updates on all <&crop> policy-filters and \n<b>subscriber_topic</b>="policies_DCAE_tca_<service-component-name>"\n[0.3.3] use the policies that match to policy-filters\n==policy-update receiver==\nsubscribe-listen for policy-update notifications from <b>MR of DMaaP</b> on \n<b>subscriber_topic</b>="policies_DCAE_tca_<service-component-name>"\n with long-collect-polling time like 15 seconds to grab the push notification\n--\n[1.2] on receiving the push-notification of policy-update, \nhandle and consume the added/updated/removed policies\n==[99.2] on shutdown - unsubscribe==\n[99.2] unsubscribe <b>subscriber_topic</b> from PDP" as dcae_component #eeeeff
}

component "<&rss> <b>Message Router of DMaaP</b>\n--\n [1.2] persistently delivers \nthe policy-update notification \nto each <b>subscriber_topic</b> \nwith full policy_body \nof each updated/added policy \nand policy-id of each removed policy" as DMaaP #ff8888

note "--request to subscribe--\n[0.3.2] http POST /decision/<b>subscription</b>/v1/<<b>subscriber_topic</b>>\n for policy-updates with all the <&crop> policy-filters and \n <b>subscriber_topic</b>='policies_DCAE_tca_<service-component-name>'\n--response to subscribe--\n[0.3.3] http return policies that match to any of the policy-filters" as message_sub_to_PDP

deployment_handler .right.> cloudify : [0] install component
deployment_handler .> cloudify : [99] uninstall \ncomponent
cloudify .down.> k8s_plugin : [0] install \ncomponent
cloudify .down.> k8s_plugin : [99] uninstall \ncomponent
k8s_plugin .down.> consul_kv : [0.2] save multiple policy-filters \nper component
k8s_plugin .down.> consul_kv : [99.1] delete records

k8s_plugin .> dcae_component : [0.3]  start  component
dcae_component ..> config_binding_service : [0.3.1] get policy_engine_client config, subscriber_topic and policy-filters along with the config
config_binding_service .> consul_kv : [0.3.1] get policy_engine_client config, \nsubscriber_topic \nand policy-filters \nalong with the config

PAP .down.> PDP : [1] push/delete policies

dcae_component .up.> message_sub_to_PDP : [0.3.2] sub
message_sub_to_PDP ..> PDP : [0.3.2] sub
PDP ..> policy_update_subscribers : [0.3.2] subscribe\n insert/update \nsubscriber record

PDP ..> message_sub_to_PDP : [0.3.3] policies
message_sub_to_PDP ..> dcae_component : [0.3.3] policies

PDP .down.> policy_update_subscribers : [1.0] iterate through \nall subscribers

PDP ..> DMaaP : [1.2] push policy-update \n for DCAE component instance
DMaaP ..> dcae_component : [1.2] push policy-update for DCAE component instance

k8s_plugin .> dcae_component : [99.2] stop component

' note "[99.2] http DELETE /decision/<b>subscription</b>/v1/<<b>subscriber_topic</b>>" as message_unsub_to_PDP
' dcae_component .down.> message_unsub_to_PDP
' message_unsub_to_PDP ..> PDP

dcae_component .> PDP : [99.2] http DELETE /decision/<b>subscription</b>/v1/<<b>subscriber_topic</b>>

PDP ..> policy_update_subscribers : [99.2] unsubscribe \n delete <b>subscriber_topic</b>

left footer
    <&thumb-up> no middleman (DCAE-Control) for policy update flow
    <&thumb-up> minimal number of messages and volume of data
    <&thumb-up> Message Router of DMaaP persistently delivers the push-notification of the policy-update per each component instance globally identified by subscriber_topic

    <b>requirements on new PDP -- see the diagram</b>
    <&task> maintain pub-sub table per subscriber_topic with policy-filters
    <&task> notify about the policy added/updated/removed through Message Router of DMaaP

    <b>requirements on DCAE Component -- see the diagram</b>
    <&task> on start, get the
        + policy_engine_client (when have <scn>:policies),
        + subscriber_topic (globally unique for the component instance like "policies_DCAE_tca_<service-component-name>")
        + and all the policy-filters assigned to the component instance from Config-Binding-Service (consul-kv)
    <&task> on start, subscribe directly with PDP for the policy-updates on the subscriber_topic on all the policy-filters assigned to the component instance
    <&task> on stop, unsubscribe subscriber_topic from PDP
    <&task> provide handler for the push-notification of the polciy-update received from Message Router of DMaaP
    <&task> listen for the policy-update notification on subscriber_topic coming through the Message Router of DMaaP with long polling collect-time

    <b>requirements on Config-Binding-Service - new data from consul-kv</b>
    <&task> get from consul-kv and return the
        + policy_engine_client (when have <scn>:policies),
        + subscriber_topic (globally unique for the component instance like "policies_DCAE_tca_<service-component-name>")
        + and all the policy-filters assigned to the component instance from Config-Binding-Service (consul-kv)

    <b>requirements on @policies_gather -- see the diagram</b>
    <b>requirements on k8s_plugin -- see the diagram</b>

    <b>requirements on DCAE install</b>
    <&task> put policy_engine_client config into consul-kv

endfooter

@enduml
  • No labels