Table of Contents
This is a potential draft of a project proposal template. It is not final or to be used until the TSC approves it.
Project Name:
- Proposed name for the project:
DCAE - Proposed name for the top level repository name: dcaegen2
- DCAE Gen2 VES Collector
- Repo:
- dcaegen2/collectors/ves
DCAE Gen2 SNMP Trap CollectorRepo:dcaegen2/collectors/snmptrap - Repo:
- DCAE Gen2 Controller
- Repo:
- dcaegen2/controller
- dcaegen2/controller/dispatcher
- dcaegen2/controller/inventory
- dcaegen2/controller/sch
- Repo:
- DCAE Gen2 Persistent Storage
- Repo:
- dcaegen2/storage/pgaas
- dcaegen2/storage/esaas dcaegen2/storage/mdbaas
- Note: For the 4Q17 release, one of major goals for DCAE is to evolve from the "old" controller that is currently in gerrit.onap.org with a new controller that follows the Common Controller Framework. The switch will maintain external (to other ONAP component) compatibility, but NOT backwards compatible internally. That is, subcomponents built for the old controller will not work with the new controller or vice versa. We are proposing to us a new top level naming “dcaegen2” for repos of subcomponents that are compatible with the new controller because it appears to be the cleanest approach to avoid confusion. The existing "dcae" top level still hosts repos of subcomponent projects that are compatible with the old controller. Eventually we will phase out the "dcae" tree.
- Note: For the 4Q17 release, one of major goals for DCAE is to evolve from the "old" controller that is currently in gerrit.onap.org with a new controller that follows the Common Controller Framework. The switch will maintain external (to other ONAP component) compatibility, but NOT backwards compatible internally. That is, subcomponents built for the old controller will not work with the new controller or vice versa. We are proposing to us a new top level naming “dcaegen2” for repos of subcomponents that are compatible with the new controller because it appears to be the cleanest approach to avoid confusion. The existing "dcae" top level still hosts repos of subcomponent projects that are compatible with the old controller. Eventually we will phase out the "dcae" tree.
DCAE Gen2 AnalyticsRepo: dcaegen2/apod/analytics-ms
DCAE Gen2 UtilitiesRepo:dcaegen2/utils
DemoRepo: dcaegen2/demo
HolmesRepos: ??
Project description:
DCAE is the umbrella name for a number of projects, components collectively fulfilling the role of Data Collection Events and Analytics for , Analytics, and Events generation for ONAP. The architecture of DCAE targets flexible, plug-able, micros-service oriented, and model based component deployment and service composition.
The DCAE team proposes the following projects for 4Q17 R1 ONAP release.
- Collectors:
- VES Collector: VNF Event Stream collector
- SNMP Trap collector:
- Controller:
- The new ONAP DCAE (gen2) controller follows the paradigm of Common Controller Framework for command and control. Hence the core function of the DCAE Controller code resides within the Common Controller Framework project. This project is for adaptation of the Common Controller Framework for DCAE use.
- Storage: this group of projects provides data storage solutions for DCAE. The common usages include supporting DCAE's own functions; storage for data analytics; persistent data storage; etc. For this release we target three popular database technologies.
- Postgres as a Service (PGaaS)
- Elastic Search as a Service (ESaaS)
- MariaDB as a Service (MDBaaS)
- Analytics: this group of projects include various analytics related components and applications. At this point, we have only placed placeholders for repos for analytics, due to the fact that the DCAE requirements (i.e. what kind of analytics are needed for supporting the vCPE and VoLTE use cases) are still unknown.
- Utilities: this project contains various utility for DCAE development, testing, and operations.
- Demo: this project contains additional, scripts, and miscellaneous data and files for supporting demo use cases
- Holmes: this is an OpenO project that performs similar/related role to DCAE. Further information is needed for identifying path for normalization.
(Notes to DCAE team members: previously we also had DMaaP topics and Call Flow proposals. After consulting with other project teams, DMaaP topic related work is better suited under the DMaaP project; and call flow related work overlaps with the CLAMP team work. It would seem that these two do not need separate project proposals under DCAE. In addition, there were items related to code that is done outside of ONAP, e.g. collectd, VES agent. ONAP will use them but do they need separate projects/repos under DCAE?)
Scope:
For ONAP 4Q17 release, the proposed DCAE projects are centered around three high level objectives: 1. normalization with Open-O DCAE functions; 2. supporting ONAP TOSCA modeling approach; 3. supporting ONAP use cases including CLAMP, vCPE, VoLTE, and vFW/vDNS.
Under these high level objectives, the scopes of DCAE projects cover the following functional items:
DCAE also support multi-site collection and analytics operations which are essential for large ONAP deployments.
DCAE components generally fall into two categories: DCAE Platform Components and DCAE Services Components. DCAE Platform consists of components that are needed for any deployments. They form the foundation for deploying and managing DCAE Service components, which are deployed on-demand based on specific collection and analytics needs.
DCAE Platform
DCAE Platform consists of a growing list of components. For R1 release this includes the Cloudify Manager, Consul, Dispatcher, Policy Handler, Service Changing Handler, Service Infrastructure (Docker host and CDAP cluster), and CDAP Broker. Their roles and functions are described below:
- Cloudify Manager: Cloudify, through its army of plugins, is capable of relationship topology base resource orchestration in many levels and cross different technologies. It is the lifecycle management engine of DCAE. Various resource deployment, change, allocation, configuration, etc, operations are all done through Cloudify. [Note: This component is part of the Common Controller SDK, or ccsdk project. How DCAE uses it will be described more below regarding deployment.]
- Consul: Consul is a service discovery technology for distributed fault detection and KV store. DCAE uses Consul for service and configuration registry. [Note: This component is part of the Common Controller SDK, or ccsdk project. How DCAE uses it will be described more below regarding deployment. ]
- Service infrastructure: DCAE platform supports two kinds of infrastructures, the Docker container hosts and CDAP/Hadoop clusters. The former is for running containerized applications and services. And the latter is for running CDAP-Hadoop based big data analytics.
- Dispatcher: Dispatcher is a NB API provider for the DCAE Services. Service related triggers, such as deploying/undeploying services, changing configurations, etc all arrive at the Dispatcher, which then enriches the request, and invokes the right Blueprints and calling Cloudify Manager plugins to complete the necessary changes in virtual resources.
- Inventory: Inventory tracks DCAE related resource information such as various Blueprints and templates that are used by Cloudify Manager to deploy and configure components, as well as inventory information extracted from A&AI that is related to but not really part of DCAE, such as the relationships between virtual network resources and their physical infrastructures.
- PGaaS: Inventory is backed by a PostgreSQL database for data storage.
- Policy Handler and Service Changing Handler: They are the interfacing modules for specific external components such as Policy, SDC, etc.
- CDAP Broker: CDAP Broker interfaces between CDAP and Cloudify Manager, supporting carrying out various Cloudify CDAP operations onto the CDAP.
The DCAE platform is expected to be deployed through the ONAP Operations Manager (OOM). There are two ways to deploy a DCAE system: 1. all DCAE components (both Platform and Service components) deployed by the OOM; or 2. the OOM deploys only the core DCAE platform functions, namely the Lifecycle Manager (Cloudify) and the Service & Config Registry (Consul). Then this platform core subsequently deploys all other DCAE Platform components. Note that the platform core is based on the same software tools that are part of the Common Controller SDK just like OOM. Because of the multi-level responsibility structure, option 2 is more suitable for larger DCAE systems that span cross multiple sites and security and infrastructure technology boundaries. We will use the option 2 as example in the DCAE Service component deployment description too.
DCAE Services
DCAE provides data collection and analysis services to the ONAP. Different analysis services will require different collection components and analytics components. They are distinct from DCAE Platform components because they are dynamically deployed and managed based on service needs. More over they are deployed and managed by the DCAE Platform. These are the DCAE Service components.
Under DCAE, the collectors and analytics, either as Docker containers or as CDAP applications, will be on-boarded as Blueprints to the DCAE Platform. At deployment time, as triggered by API calls through Dispatcher and other interfacing Platform modules, the Cloudify Manager completes the deployment of the required DCAE Service resources onto the Docker Host or CDAP Cluster that is determined by the DCAE Platform. The illustration below depicts that the DCAE is deploying two services (yellow and red). The required DCAE Service components are deployed onto the DCAE Platform. In this example, the yellow service is completed by a VNF Event Streaming collector and a Threshold Crossing Analytics; while the red service consists of a SNMP Trap collector, followed by a data normalization step and a Correlation analytics.
In addition, the DCAE Platform will also configure the data paths between the Service components, i.e. by setting up DMaaP topics and configurations.
The figure below further illustrates how performance measurement and fault management data, i.e. VNF events, SNMP traps, and alert events, traverse through the DCAE Service components, and eventually depart from DCAE to reach downstream components such as ONAP Policy, or other external systems such as ticketing.
* Portal/GUI
(This section is for addressing related TSC comments, not as part of DCAE project.)
Dedicated DCAE Portal/GUI is not part of the DCAE project under the current scope. In future if such portal is deemed necessary, it may be developed under DCAE, or under Portal or VID project. At present time, certain aspects of DCAE operation status can be displayed by utilizing a combination of native GUI/Portal of the open source software tools used by DCAE (e.g. CDAP GUI, Cloudify portal, etc), or CLAMP cockpit for a more service level end-to-end view in which DCAE is only a part, or CLI-style and RestAPI interaction for status probing.
Project Scope:
Because of the large potential scope for DCAE, the components proposed for 4Q17 R1 are prioritized as follows:
- DCAE Platform: This is top priority because everything DCAE depends on it. All platform components listed above fall under this priority.
- DCAE Services: collectors and analytics that are needed for supporting the 4Q17 use cases and Open-O harmonization. We have identified the VES and SNMP Trap collectors; Threshold Crossing and Data Normalization Analytics. We expect the list to grow and be finalized as the ONAP R1 use cases and the control loops for the use cases are fully defined.
- Other collectors, analytics, and functions that have been identified as valuable by the community will be included based on resource availabilities in this and future releases of ONAP.
- DCAE Portal and design studio
- Additional normalization Normalization of OpenECOMP/OpenO data collection needs and realization
- More TOSCA model based artifact (component, VNF, and data) on-boarding
- Catalog of artifact models
- VNF/PNF data collection
- New DCAE Controller that follows the Common Controller Framework, and supports DCAE component placement (e.g. edge/central) and federation
- Supporting VES standardized VNF data modeling and collectionMultiVIM interfacing
- Event triggered life cycle management
- Cloud computing Infrastructure event collection and analytics
- Predictive analytics
- AI/ML
...
How does this project fit into the rest of the ONAP Architecture?
DCAE performs a vital function within the ONAP architecture. DCAE collects performance metrics and fault data from the VNFs, PNFs, and computing infrastructure, performing local and global analytics, and generating events that are provided for downstream ONAP components (e.g. Policy) for further operations.DCAE follows the TOSCA model based ONAP Operations Manager and Common Controller Framework SDK architecture for component on-boarding, close-loop triggered or event (e.g. A&AI) triggered collector and analytics deployment, configuration, and scaling.
- What other ONAP projects does this project depend on?
- A&AI, Policy, Micro Services, Modeling, CLAMP, SDC, ONAP Controller, Common Controller FrameworkOOM, CCSDK, DMaaP, Common Services, MultiVIM, Integration, Holmes
- How does this align with external standards/specifications?
- TOSCA
- VES (OPNFV)
- Are there dependencies with other open source projects?
- CDAP, Cloudify, Consul, Hadoop, Elastic Search, PostgreSQL, MariaDB
Resources:
...
Project Technical Lead
- Current
- Vijay Venkatesh Kumar
- (AT&T)
- Past
- Lusheng
Ji - Ji (AT&T)
Primary Contact Person
John F. MurrayNames, gerrit IDs, and company affiliations of
...
the
...
committers
...
| Name | Gerrit ID | Company | Time Zone | DCAE |
|---|
...
| Component Focus | |
|---|---|
| Lusheng Ji | wrider |
...
| AT&T |
...
| lji@research.att.com | New Jersey, USA EST/EDT |
...
| Vijay Venkatesh Kumar | vv770d | AT&T |
...
| New Jersey, USA EST/EDT | collectors controller | |||
| Tony Hansen | TonyLHansen | AT&T | tony@att.com | New Jersey, USA EST/EDT |
...
| database, storage, analytics | |||||
| Mike Hwang | researchmike | AT&T | mhwang@research.att.com | New Jersey, USA EST/EDT | controller |
| Yan Yang | yangyan | China Mobile | yangyanyj@chinamobile.com |
...
| Beijing, China. UTC +8 |
...
| collectors | |||
| Xinhui Li | xinhuili | VMware | lxinhui@vmware.com |
...
| Beijing, China. UTC +8 |
...
| collectors |
Contributors
Names and affiliations of any other contributors
Company | Name | DCAE Component/Repo Focus | Commitment (%) |
...
AT&T | Alok Gupta | ag1367@att.com | VES | |
| Gayathri Patrachari | Collector | 50% | ||
| Alexei Nekrassov | nekrassov@att.com | Analytics |
...
| 50% | ||
| Tommy Carpenter | tommy@research |
...
...
| CDAP Broker, Config binding service | >50% | |
| Jack Lucas | jflucas@research.att |
...
...
Names and affiliations of any other contributors
| Deployment Handler, Cloudify Manager | >50% | |||
| Alexander V Shatov | alexs@research.att.com | Policy Handler | >50% | |
| David Ladue | dl3158@att.com | SNMP Trap Collector | 50% | |
| Gokul Singraju | gs244f@att.com | VES | ||
Intel | Maryam Tahhan | VES | ||
| Tim Verrall | VES | |||
| China Mobile | Yuan Liu | |||
| Huawei | Avinash S | |||
| Actual contribution TBD for below | ||||
| AT&T | Jerry Robinson | |||
| BOCO | Jingbo Liu | liujingbo@boco.com.cn | ||
| Zhangxiong Zhou | ||||
| Deutsch Telcom | Mark Fiedler | |||
Futurewei | Xin Miao | xin.miao@huawei.com | ||
| IBM | Yusuf Mirza | |||
| David Parfett | ||||
| Mathew Thomas | ||||
| Janki Vora | ||||
| Amandeep Singh | ||||
...
Company
...
Name
...
AT&T
...
Alok Gupta
...
Jerry Robinson
...
Lusheng Ji
...
Tech Mahindra
...
Sandeep Singh
...
Abhinav Singh
...
Gokul Singraju
...
Futurewei
...
Xin Miao
...
Deutsch Telcom
...
Mark Fiedler
...
Intel
...
Orange | Vincent Colas | |||
Olivier Augizeau | ||||
| Pawel Pawlak | ||||
Reliance Jio | Aayush Bhatnagar | Aayush.Bhatnagar@ril.com | ||
| Yog Vashishth | yog.vashishth@ril.com | |||
| Adityakar Jha |
...
Intel
...
Maryam Tahhan
| Adityakar.Jha@ril.com | ||||
Tech Mahindra | Sandeep Singh | |||
| Abhinav Singh |
...
| VMware | Sumit Verdi |
...
- Project Roles (include RACI chart, if applicable)
Other Information
- link to seed code (if applicable)
- Vendor Neutral
- if the proposal is coming from an existing proprietary codebase, have you ensured that all proprietary trademarks, logos, product names, etc., have been removed?
The current seed code has been already scanned and cleanup to remove all proprietary trademarks, logos, etc. except openecomp to be replaced by onap
Subsequent modification to the existing seed code should continue to follow the same scanning and clean up principles.
- if the proposal is coming from an existing proprietary codebase, have you ensured that all proprietary trademarks, logos, product names, etc., have been removed?
- Meets Board policy (including IPR)
Use the above information to create a key project facts section on your project page
Key Project Facts
Project Name:
- JIRA project name: Data Data Collection , Analytics , and Events
- JIRA project prefix: DCAE-dcaegen2
Repo name: Lifecycle State: Primary Contact: Project Lead: mailing list tag [Should match Jira Project Prefix]
Committers:
Vijay Venkatesh Kumar vv770d@att.com
Serban Jora sj2381@att.com
Tony Hansen tony@att.com
Avinash S avinash.s@huawei.com
Lifecycle State:
- Mature code that needs enhancement/integration: VES collector, TCA analytics, Dispatcher, Service Changing Handler, Inventory PGaaS, CDAP infrastructure, CDAP Broker
- Incubation: Policy Handler, SNMP Trap collector, additional use case specific collectors/analytics/Blueprints, ESaaS,
Primary Contact: John F. Murray (AT&T), Lusheng Ji (AT&T)
Project Lead: Lusheng Ji (AT&T)
Mailing list tag: dcaegen2
Repo structure and names:
Category | Components Repository name | Maven Group ID | Components Description |
Platform | dcaegen2/platform/blueprints | org.onap.dcaegen2.platform.blueprints | Blueprint for DCAE controller |
dcaegen2/platform/plugins | org.onap.dcaegen2.platform.plugins | Plugin for DCAE controller | |
dcaegen2/platform/cdapbroker | org.onap.dcaegen2.platform.cdapbroker | CDAP Broker | |
dcaegen2/platform/cli | org.onap.dcaegen2.platform.cli | Cli tool for onboarding through new dcae controller | |
dcaegen2/platform/deployment-handler | org.onap.dcaegen2.platform.deployment-handler | Deployment handler | |
dcaegen2/platform/servicechange-handler | org.onap.dcaegen2.platform.servicechange-handler | Service change handler | |
dcaegen2/platform/inventory-api | org.onap.dcaegen2.platform.inventory-api | DCAE inventory API service | |
dcaegen2/platform/policy-handler | org.onap.dcaegen2.platform.policy-handler | Policy handler | |
dcaegen2/platform/configbinding | org.onap.dcaegen2.platform.configbinding | Configbinding api service | |
dcaegen2/platform/registrator | org.onap.dcaegen2.platform.registrator | Registrator | |
ccsdk/storage/pgaas | org.onap.ccsdk.storage.pgaas | Postgres as a service | |
ccsdk/storage/esaas | org.onap.ccsdk.storage.esaas | Elastic Search as a service | |
Service | dcaegen2/collectors/ves | org.onap.dcaegen2.collectors.ves | VNF Event Streaming collector |
dcaegen2/collectors/snmptrap | org.onap.dcaegen2.collectors.snmptrap | SNMP Trap collector (TBD) | |
dcaegen2/analytics/tca | org.onap.dcaegen2.analytics.tca | Threshold crossing analytics | |
deployments | dcaegen2/deployments | org.onap.dcaegen2.deployments | For hosting configurations and blueprints for different deployments |
utils | dcaegen2/utils | org.onap.dcaegen2.utils | For hosting utility/tools code used cross components |
...
Link to approval of additional submitters:
...