You are viewing an old version of this page. View the current version.
Compare with Current
View Page History
« Previous
Version 15
Next »
- maintain the database of subscribers with the
- list of generic policy-filters (each policy-filter is the resource from the request json to /decision/v1 API) Policy Design and API Flow for Model Driven Control Loop
- subscriber_topic for Message Router of DMaaP that uniquely identifies the component instance like “policies_DCAE_tca_<service-component-name>”
- <service-component-name> uniquely identifies the component in DCAE
- on policy push/delete from PAP and on the insert-update of the subscriber record
- select all subscribers that match to the pushed/deleted policies by any policy-filter
- for each affected subscriber retrieve all the latest snapshot of policies
- notify each subscriber_topic separately with the latest snapshot of policies by sending the message to Message Router of DMaaP with the topic=subscriber_topic
- there might be a need for some logic to identify the stale subscriptions by checking with the Message Router of DMaaP on the timestamps of the latest delivered/undelivered message per ach topic PDP has the subscription on
Requirements on Message Router of DMaaP
- persistently (up to 7 days) deliver all the policy-update notifications per subscriber_topic
- component will listen for the subscriber_topic of the policy-update notification from MR of DMaaP with long-collect-polling time like 15 seconds to grab the push notification
Requirements on component
Less requirements on component - policy-handler to do the subscription with PDP for the component
on startup – get the subscriber_topic=“policies_DCAE_tca_<service-component-name>” from config-binding-service
listen for subscriber_topic of policy-update notification from MR of DMaaP with long-collect-polling time like 15 seconds to grab the push notification
on receiving the policy-update pushed notification from DMaaP, handle the policy-update that contains the full snapshot of all the policies that match to the component
calculate the delta between the current set of policies and the received shapshot of policies
See inside the diagram for more details, data structures, and flow steps
@startuml r5-1_proposed_policy_update_flow
allowmixing
scale 4096*4096
skinparam title {
FontSize 24
FontColor Blue
FontStyle Bold
BorderRoundCorner 15
BorderThickness 2
}
skinparam roundcorner 20
skinparam component {
BackgroundColor Snow
}
skinparam note {
FontColor Black
BackgroundColor azure
}
title = R5-1 El Alto proposal for \n policy flows between the new PDP and DCAE component \n PDP does the pub-sub for policies per component \n %date[yyyy-MM-dd HH:mm]%
package "<&dollar> Policy-Engine" as policy_engine #88ff88 {
component "<&dollar> new Policy-Access-Point (<b>new PAP</b>)" as PAP
component "<&dollar> new Policy-Decision-Point (<b>new PDP</b>)\n==pub-sub==\n maintains the table of policy-update subscribers \nwith <&crop> policy-filters and subscriber_topic\n--subscribe--\n[0.3.2] insert-update the record for the <b>subscriber_topic</b>\n[0.3.3] notify the subscriber_topic \nwith the latest snapshot of policies that match to any of policy-filters\n--unsubscribe--\n[99.2] delete the record for the <b>subscriber_topic</b>\n==[1] on policy push/delete from PAP==\n[1.0] select all subscribers that match to the pushed/deleted policies \n by any <&crop> policy-filter\n[1.1] for each affected subscriber retrieve all the latest snapshot of policies \n[1.2] notify each subscriber_topic separately \nwith the latest snapshot of policies" as PDP
database "<&list> <b>policy-update subscribers</b> (table in database)\n..unique topic per each component instance..\n<b>subscriber_topic</b> TEXT <&key> PK -- "policies_DCAE_tca_<service-component-name>"\n..\n action TEXT -- "configure"\nONAPName TEXT -- "DCAE"\nONAPComponent TEXT -- "tca"\nONAPInstance TEXT -- "<service-component-name>"\n..list of policy-filters on subscriber_topic (component)..\n<&list> <&crop> <b>policy-filters</b> JSON -- list of the <b>"resource"</b> objects of /decision/v1: \n[{"policy-id": ["onap.scaleout.tca", "onap.restart.tca"]}, ...]" as policy_update_subscribers
}
package "<&dollar> DCAE" as DCAE {
package "<&signpost> DCAE-Controller" as DCAE_Controller #eeffee {
package "<&cog> deployment" #eeeeff {
component "<&aperture> <b>deployment_handler</b>\n--\n[0] install component through cloudify for the blueprint+inputs\n--\n[99] uninstall component through cloudify" as deployment_handler
component "<&aperture> <b>policy_handler</b>\n--[0.3] on startup of component - /subscribe--\n[0.3.1] get <b>subscriber_topic</b> and all policy-filters from req from plugin\n[0.3.2] subscribe with PDP for policy-updates on all <&crop> policy-filters and \n<b>subscriber_topic</b>="policies_DCAE_tca_<service-component-name>"\n--\n==[99.2] on shutdown of component - /unsubscribe==\n[99.2] unsubscribe <b>subscriber_topic</b> from PDP" as policy_handler
package "<b>cloudify</b>" as cloudify_server #00ffff {
control cloudify
component "<&aperture> <b>k8s_plugin decorated with @policies_gather</b>\n--[0] install component--\n[0.0] <b>k8s_plugin</b>: save config for component into consul-kv under <service-component-name>\n[0.1] <b>@policies_gather</b>: gather policy-filters assigned to component in blueprint+inputs\n[0.2] <b>@policies_gather</b>: save <service-component-name>:<b>subscriber_topic</b>='policies_DCAE_tca_<service-component-name>' into consul-kv\n[0.3] <b>@policies_gather</b>: subscribe the component for policy-updates through policy_handler\n pass <b>subscriber_topic</b>="policies_DCAE_tca_<service-component-name>" \nand multiple policy-filters for component as [{"policy-id": ["onap.scaleout.tca", "onap.restart.tca"]}]\n[0.4] <b>k8s_plugin</b>: install and start component\n--[99] uninstall component--\n[99.0] <b>@policies_gather</b>: delete records from consul-kv\n[99.1] <b>k8s_plugin</b>: delete records from consul-kv\n[99.2] <b>k8s_plugin</b>: stop and uninstall component" as k8s_plugin
}
}
database "<&target> <b>consul-kv</b>\n--config for component--\n<service-component-name>={...config...}\n<service-component-name>:<b>subscriber_topic</b>='policies_DCAE_tca_<service-component-name>'" as consul_kv
component "<&aperture> <b>config_binding_service</b>" as config_binding_service
}
component "<&target> <b>DCAE component like TCA</b>\n--[0.4] on startup - run policy-update receiver--\n[0.4.1] get subscriber_topic along with the config from CBS\n[0.4.2] listen for policy-update notifications from <b>MR of DMaaP</b> on \n<b>subscriber_topic</b>="policies_DCAE_tca_<service-component-name>"\n with long-collect-polling time like 15 seconds to grab the push notification\n--\n[0.3.3] and [1.2] on receiving the push-notification of policy-update, \n[1.3] calculate the delta - compare the collection of existing policies \n versus the latest snapshot of policies and figure out \nwhat policies are added/updated/removed \n[1.4] handle and consume the added/updated/removed policies" as dcae_component #eeeeff
}
component "<&rss> <b>Message Router of DMaaP</b>\n--\n [0.3.3] and [1.2] persistently delivers \nthe policy-update notification \nto each <b>subscriber_topic</b> \nwith the latest snapshot of policies" as DMaaP #ff8888
PAP .down.> PDP : [1] push/delete policies
PDP .down.> policy_update_subscribers : [0.3.2] subscribe\n insert/update \nsubscriber record
PDP .down.> policy_update_subscribers : [1.0] iterate through \nall subscribers
PDP ..> policy_update_subscribers : [99.2] unsubscribe \n delete <b>subscriber_topic</b>
PDP .down.> DMaaP : [0.3.3] and [1.2] push policy-update \n for DCAE component instance
DMaaP .up.> dcae_component : [0.3.3] and [1.2] push policy-update for DCAE component instance
dcae_component .right.> config_binding_service : [0.4.1] get subscriber_topic along with the config
config_binding_service .right.> consul_kv : [0.4.1] get config and subscriber_topic
policy_handler ..> PDP : [0.3.2] POST /v1/decision/<b>subscription</b>/<<b>subscriber_topic</b>>
policy_handler ..> PDP : [99.2] DELETE /v1/decision/<b>subscription</b>/<<b>subscriber_topic</b>>
deployment_handler .down.> cloudify : [0] install component
deployment_handler .down.> cloudify : [99] uninstall \ncomponent
cloudify .down.> k8s_plugin : [0] install \ncomponent
cloudify .down.> k8s_plugin : [99] uninstall \ncomponent
k8s_plugin .down.> consul_kv : [0.0] and [0.2] save config and <b>subscriber_topic</b> for component
k8s_plugin .down.> consul_kv : [99.0] and [99.1] delete records
k8s_plugin .up.> policy_handler : [0.3] subscribe
k8s_plugin .up.> policy_handler : [99.2] unsubscribe
k8s_plugin .left.> dcae_component : [0.4] start component
k8s_plugin .left.> dcae_component : [99.2] stop component
' note "[99.2] http DELETE /decision/<b>subscription</b>/v1/<<b>subscriber_topic</b>>" as message_unsub_to_PDP
' dcae_component .down.> message_unsub_to_PDP
' message_unsub_to_PDP ..> PDP
left footer
<&thumb-up> no middleman (DCAE-Control) for policy update flow
<&thumb-up> minimal number of messages and volume of data
<&thumb-up> Message Router of DMaaP persistently delivers the push-notification of the policy-update per each component instance globally identified by subscriber_topic
<b>requirements on new PDP -- see the diagram</b>
<&task> maintain pub-sub table per subscriber_topic with policy-filters
<&task> notify about the policy added/updated/removed through Message Router of DMaaP
<b>requirements on DCAE Component -- see the diagram</b>
<&task> on start, get
+ subscriber_topic (globally unique for the component instance like "policies_DCAE_tca_<service-component-name>")
<&task> provide handler for the push-notification of the polciy-update received from Message Router of DMaaP
<&task> listen for the policy-update notification on subscriber_topic coming through the Message Router of DMaaP with long polling collect-time
<&task> calc delta of policy-update and consume it
<b>requirements on Config-Binding-Service - new data from consul-kv</b>
<&task> get from consul-kv and return the
+ subscriber_topic (globally unique for the component instance like "policies_DCAE_tca_<service-component-name>")
<b>requirements on @policies_gather -- see the diagram</b>
<b>requirements on k8s_plugin -- see the diagram</b>
<b>requirements on policy_handler -- see the diagram</b>
endfooter
@enduml
More requirements on component (initial proposal) - too much
- on startup – subscribe to PDP with the policy-filter the component is interested in and the subscriber_topic=“policies_DCAE_tca_<service-component-name>” that that uniquely identifies the component instance
- listen for subscriber_topic of policy-update notification from MR of DMaaP with long-collect-polling time like 15 seconds to grab the push notification
on receiving the policy-update pushed notification from DMaaP, handle the policy-update that contains the full snapshot of all the policies that match to the component
calculate the delta between the current set of policies and the received shapshot of policies
- n stop, unsubscribe subscriber_topic from PDP
See inside the diagram for more details, data structures, and flow steps
@startuml r5_proposed_policy_update_flow
allowmixing
scale 4096*4096
skinparam title {
FontSize 24
FontColor Blue
FontStyle Bold
BorderRoundCorner 15
BorderThickness 2
}
skinparam roundcorner 20
skinparam component {
BackgroundColor Snow
}
skinparam note {
FontColor Black
BackgroundColor azure
}
title = R5 El Alto proposal for \n policy flows between the new PDP and DCAE component \n PDP does the pub-sub for policies per component \n %date[yyyy-MM-dd HH:mm]%
package "<&dollar> Policy-Engine" as policy_engine #88ff88 {
component "<&dollar> new Policy-Access-Point (<b>new PAP</b>)" as PAP
component "<&dollar> new Policy-Decision-Point (<b>new PDP</b>)\n==pub-sub==\n maintains the table of policy-update subscribers \nwith <&crop> policy-filters and subscriber_topic\n--subscribe--\n[0.3.2] insert-update the record for the <b>subscriber_topic</b>\n[0.3.3] return policies that match to policy-filters\n--unsubscribe--\n[99.2] delete the record for the <b>subscriber_topic</b>\n==[1] on policy push/delete from PAP==\n[1.0] select all subscribers that match to the pushed/deleted policies \n by any <&crop> policy-filter\n[1.1] for each affected subscriber retrieve all the latest snapshot of policies \n[1.2] notify each subscriber_topic separately \nwith the latest snapshot of policies" as PDP
database "<&list> <b>policy-update subscribers</b> (table in database)\n..unique topic per each component instance..\n<b>subscriber_topic</b> TEXT <&key> PK -- "policies_DCAE_tca_<service-component-name>"\n..\n action TEXT -- "configure"\nONAPName TEXT -- "DCAE"\nONAPComponent TEXT -- "tca"\nONAPInstance TEXT -- "<service-component-name>"\n..list of policy-filters on subscriber_topic (component)..\n<&list> <&crop> <b>policy-filters</b> JSON -- list of the <b>"resource"</b> objects of /decision/v1: \n[{"policy-id": ["onap.scaleout.tca", "onap.restart.tca"]}, ...]" as policy_update_subscribers
}
package "<&dollar> DCAE" as DCAE {
package "<&signpost> DCAE-Controller" as DCAE_Controller #eeffee {
package "<&cog> deployment" #eeeeff {
component "<&aperture> <b>deployment_handler</b>\n--\n[0] install component through cloudify for the blueprint+inputs\n--\n[99] uninstall component through cloudify" as deployment_handler
package "<b>cloudify</b>" as cloudify_server #00ffff {
control cloudify
component "<&aperture> <b>k8s_plugin decorated with @policies_gather</b>\n--[0] install component--\n[0.1] <b>@policies_gather</b>: gather policy-filters assigned to component in blueprint+inputs\n[0.2] <b>@policies_gather</b>: save multiple policy-filters for component into consul-kv as \n<service-component-name>:policies/filters/<filter_id>:{"policy-id": ["onap.scaleout.tca", "onap.restart.tca"]}\n[0.3] <b>k8s_plugin</b>: install and start component\n--[99] uninstall component--\n[99.1] <b>@policies_gather</b>: delete records from consul-kv\n[99.2] <b>k8s_plugin</b>: stop and uninstall component" as k8s_plugin
}
}
database "<&target> <b>consul-kv</b>\n--multiple policy-filters records (<b>"resource"</b> objects of /decision/v1 API in PDP) per component--\n<service-component-name>:<b>policies/filters</b>/<filter_id>:{"policy-id": ["onap.scaleout.tca", "onap.restart.tca"]}\n<service-component-name>:<b>policies/subscriber_topic</b>='policies_DCAE_tca_<service-component-name>'\n--config of policy_engine_client in DCAE - populated on install of DCAE platform--\n<b>policy_engine_client</b>: {url, headers, tls-settings, timeout-settings, etc.}" as consul_kv
component "<&aperture> <b>config_binding_service</b>" as config_binding_service
}
component "<&target> <b>DCAE component like TCA</b>\n--[0.3] on startup - subscribe--\n[0.3.1] get policy_engine_client config, subscriber_topic and \nall policy-filters along with the config from CBS\n[0.3.2] subscribe with PDP for policy-updates on all <&crop> policy-filters and \n<b>subscriber_topic</b>="policies_DCAE_tca_<service-component-name>"\n[0.3.3] use the policies that match to policy-filters\n==policy-update receiver==\nsubscribe-listen for policy-update notifications from <b>MR of DMaaP</b> on \n<b>subscriber_topic</b>="policies_DCAE_tca_<service-component-name>"\n with long-collect-polling time like 15 seconds to grab the push notification\n--\n[1.2] on receiving the push-notification of policy-update, \n[1.3] calculate the delta - compare the collection of existing policies \n versus the latest snapshot of policies and figure out \nwhat policies are added/updated/removed \n[1.4] handle and consume the added/updated/removed policies\n==[99.2] on shutdown - unsubscribe==\n[99.2] unsubscribe <b>subscriber_topic</b> from PDP" as dcae_component #eeeeff
}
component "<&rss> <b>Message Router of DMaaP</b>\n--\n [1.2] persistently delivers \nthe policy-update notification \nto each <b>subscriber_topic</b> \nwith the latest snapshot of policies" as DMaaP #ff8888
note "--request to subscribe--\n[0.3.2] http POST /decision/<b>subscription</b>/v1/<<b>subscriber_topic</b>>\n for policy-updates with all the <&crop> policy-filters and \n <b>subscriber_topic</b>='policies_DCAE_tca_<service-component-name>'\n--response to subscribe--\n[0.3.3] http return policies that match to any of the policy-filters" as message_sub_to_PDP
deployment_handler .right.> cloudify : [0] install component
deployment_handler .> cloudify : [99] uninstall \ncomponent
cloudify .down.> k8s_plugin : [0] install \ncomponent
cloudify .down.> k8s_plugin : [99] uninstall \ncomponent
k8s_plugin .down.> consul_kv : [0.2] save multiple policy-filters \nper component
k8s_plugin .down.> consul_kv : [99.1] delete records
k8s_plugin .> dcae_component : [0.3] start component
dcae_component ..> config_binding_service : [0.3.1] get policy_engine_client config, subscriber_topic and policy-filters along with the config
config_binding_service .> consul_kv : [0.3.1] get policy_engine_client config, \nsubscriber_topic \nand policy-filters \nalong with the config
PAP .down.> PDP : [1] push/delete policies
dcae_component .up.> message_sub_to_PDP : [0.3.2] sub
message_sub_to_PDP ..> PDP : [0.3.2] sub
PDP ..> policy_update_subscribers : [0.3.2] subscribe\n insert/update \nsubscriber record
PDP ..> message_sub_to_PDP : [0.3.3] policies
message_sub_to_PDP ..> dcae_component : [0.3.3] policies
PDP .down.> policy_update_subscribers : [1.0] iterate through \nall subscribers
PDP ..> DMaaP : [1.2] push policy-update \n for DCAE component instance
DMaaP ..> dcae_component : [1.2] push policy-update for DCAE component instance
k8s_plugin .> dcae_component : [99.2] stop component
' note "[99.2] http DELETE /decision/<b>subscription</b>/v1/<<b>subscriber_topic</b>>" as message_unsub_to_PDP
' dcae_component .down.> message_unsub_to_PDP
' message_unsub_to_PDP ..> PDP
dcae_component .> PDP : [99.2] http DELETE /decision/<b>subscription</b>/v1/<<b>subscriber_topic</b>>
PDP ..> policy_update_subscribers : [99.2] unsubscribe \n delete <b>subscriber_topic</b>
left footer
<&thumb-up> no middleman (DCAE-Control) for policy update flow
<&thumb-up> minimal number of messages and volume of data
<&thumb-up> Message Router of DMaaP persistently delivers the push-notification of the policy-update per each component instance globally identified by subscriber_topic
<b>requirements on new PDP -- see the diagram</b>
<&task> maintain pub-sub table per subscriber_topic with policy-filters
<&task> notify about the policy added/updated/removed through Message Router of DMaaP
<b>requirements on DCAE Component -- see the diagram</b>
<&task> on start, get the
+ policy_engine_client (when have <scn>:policies),
+ subscriber_topic (globally unique for the component instance like "policies_DCAE_tca_<service-component-name>")
+ and all the policy-filters assigned to the component instance from Config-Binding-Service (consul-kv)
<&task> on start, subscribe directly with PDP for the policy-updates on the subscriber_topic on all the policy-filters assigned to the component instance
<&task> on stop, unsubscribe subscriber_topic from PDP
<&task> provide handler for the push-notification of the polciy-update received from Message Router of DMaaP
<&task> listen for the policy-update notification on subscriber_topic coming through the Message Router of DMaaP with long polling collect-time
<b>requirements on Config-Binding-Service - new data from consul-kv</b>
<&task> get from consul-kv and return the
+ policy_engine_client (when have <scn>:policies),
+ subscriber_topic (globally unique for the component instance like "policies_DCAE_tca_<service-component-name>")
+ and all the policy-filters assigned to the component instance from Config-Binding-Service (consul-kv)
<b>requirements on @policies_gather -- see the diagram</b>
<b>requirements on k8s_plugin -- see the diagram</b>
<b>requirements on DCAE install</b>
<&task> put policy_engine_client config into consul-kv
endfooter
@enduml
