Overview

Project NameData movement as a platform
Target Release NameBeijing  Release
Project Lifecycle StateIncubation (Refer to ONAP Charter, section 3.3 Project Lifecycle for further information)
Participating Company AT&T, Tech Mahindra

Scope

What is this release trying to address?

AAF integration with Message Router, Data Router, and Bus Controller will be added to the DMaap and some enhancements to the DMaap client.

Use Cases

The existing Amsterdam use cases are still going to be supported and additional use cases related to DR and Bus Controller  will be supported for the Beijing Release.

Minimum Viable Product

For Beijing release, the minimum viable product we are targeting is  integrating AAF  with the Message Router, sending and receiving messages using Data router, and provisioning topics and feeds via Bus Controller.

Functionalities

List the functionalities that this release is committing to deliver by providing a link to JIRA Epics and Stories. In the JIRA Priority field, specify the priority (either High, Medium, Low). The priority will be used in case de-scoping is required. Don't assign High priority to all functionalities.

Epics

Key Summary T Created Updated Due Assignee Reporter P Status Resolution
Refresh

Stories

Key Summary T Created Updated Due Assignee Reporter P Status Resolution
Refresh


Longer term roadmap

DMaaP is a premier platform for high performing and cost effective data movement services that transports and processes data from any source to any target with the format, quality, security, and concurrency required to serve the business and customer needs.

Release Deliverables


Deliverable NameDeliverable Description

AAF integration with Data Router

DMaap Git repository
Data Router source codeDMaap Git repository
Data Router libraryONAP Nexus repository
Data Router API descriptionONAP wiki
Data Router Release NotesONAP wiki
Data Router DocumentationDMaap Git repository
Bus Controller APIDMaaP Git repo
Bus Controller ContainerONAP Nexus repo
Bus Controller source codeDMaaP Git repo

Sub-Components

As part of the Beijing release, the following components will be added to DMaaP:

  • Data Router - for file transport
  • Bus Controller Provisioning API - for provisioning of topics (MR), feeds (DR) and permissions (AAF)
  • Bus Controller GUI - human interface to Bus Controller Provisioning API

Architecture

High level architecture diagram


Platform Maturity

Refering to CII Badging Security Program and Platform Maturity Requirements, fill out the table below by indicating the actual level , the targeted level for the current release and the evidences on how you plan to achieve the targeted level.

AreaActual LevelTargeted Level for current ReleaseHow, EvidencesComments
Performance01Baseline performance criteria will be defined
  • 0 -- none
  • 1 – baseline performance criteria identified and measured
  • 2 & 3 – performance improvement plans created & implemented
Stability1172 hours soak test with random transactions
  • 0 – none
  • 1 – 72 hours component level soak w/random transactions
  • 2 – 72 hours platform level soak w/random transactions
  • 3 – 6 months track record of reduced defect rate
Resiliency22K8S auto detection and recovery
  • 0 – none
  • 1 – manual failure and recovery (< 30 minutes)
  • 2 – automated detection and recovery (single site)
  • 3 – automated detection and recovery (geo redundancy)
Security01
  • 0 – none
  • 1 – CII Passing badge + 50% Test Coverage
  • 2 – CII Silver badge; internal communication encrypted; role-based access control and authorization for all calls
  • 3 – CII Gold
Scalability11Level 1 single site horizontal scaling
  • 0 – no ability to scale
  • 1 – single site horizontal scaling
  • 2 – geographic scaling
  • 3 – scaling across multiple ONAP instances
Manageability11Using EELF common framework for logging
  • 1 – single logging system across components; instantiation in < 1 hour
  • 2 – ability to upgrade a single component; tracing across components; externalized configuration management
Usability12API documentation and swagger API provided
  • 1 – user guide; deployment documentation; API documentation
  • 2 – UI consistency; usability testing; tutorial documentation


  • API Incoming Dependencies

List the API this project is expecting from other projects.
Prior to Release Planning review, Team Leads must agreed on the date by which the API will be fully defined. The API Delivery date must not be later than the release API Freeze date.

Prior to the delivery date, it is a good practice to organize an API review with the API consumers.

API NameAPI DescriptionAPI Definition DateAPI Delivery dateAPI Definition link (i.e.swagger)
AAFAuthentication and Authorization FrameworkTBDMarch 08 2010TBD

  • API Outgoing Dependencies

API this project is delivering to other projects.

API NameAPI DescriptionAPI Definition DateAPI Delivery dateAPI Definition link (i.e.swagger)
PublishingAPI used to post a message to the TopicSept 11 2017March 08 2018DMaap API
SubscribingAPI used to subscribe a message from the TopicSept 11 2017March 08 2018DMaap API
ProvisiningAPI for creating,deleting and listing the administrative objectsSept 11 2017March 08 2018DMaap API (Bus Controller API)
Admin APIAPI for the admin accessSept 11 2017March 08 2018DMaap API

  • Third Party Products Dependencies

NameDescriptionVersion
AJSCJava services container1.0.0
KafkaDistributed messaging platform0.8.2.1
Zoo KeeperCentralized service for maintaining configuration information3.4.6
SpringSpring Framework1.4
DockerContainer engine1.12

  • Testing and Integration Plans

  1. Unit test: Unit tests has atleast 30% code coverage.
  2. Functional test: Leverage the robot framework infrastructure to provide the functional test.
  3. Integration test: Support integration team to provide the end to end integration test.
  4. All the above should be automation tests run on the LF Jenkins Infrastructure.


  • Gaps

No gaps identified so far

  • Known Defects and Issues

Provide a link toward the list of all known project bugs.

Key Summary T Created Updated Due Assignee Reporter P Status Resolution
Refresh

  • Risks

No known risks so far.

  • Resources

Updated  the Resources Committed to the Release .

  • Release Milestone

The milestones are defined at the Release Level and all the supporting project agreed to comply with these dates.

  • Team Internal Milestone

  • This section may be used to document internal milestones that the team agreed on.

    Also, in the case the team has made agreement with other team to deliver some artifacts on a certain date that are not in the release milestone, provide these agreements and dates in this section.

    It is not expected to have a detailed project plan.

    Date

    Project

    Deliverable

    3/29/2018DMaaPDMaap - Buscontroller

Other Information

  • Vendor Neutral

If this project is coming from an existing proprietary codebase, ensure that all proprietary trademarks, logos, product names, etc. have been removed. All ONAP deliverables must comply with this rule and be agnostic of any proprietary symbols.

  • Free and Open Source Software

FOSS activities are critical to the delivery of the whole ONAP initiative. The information may not be fully available at Release Planning, however to avoid late refactoring, it is critical to accomplish this task as early as possible.
List all third party Free and Open Source Software used within the release and provide License type (BSD, MIT, Apache, GNU GPL,... ).
In the case non Apache License are found inform immediately the TSC and the Release Manager and document your reasoning on why you believe we can use a non Apache version 2 license.

Each project must edit its project table available at Project FOSS.


Charter Compliance

The project team comply with the ONAP Charter.

7 Comments

  1. Evguenia (Jane) Freider

    Hi, I read a bunch of docs on DMaaP and watched a presentation on Message Router architecture and usage. I've noticed that DME2 is shown as a part of MR architecture. Although DME2 description seems to imply it is only needed if a remote communication needs to be supported across data centers perhaps... So is DME2 a required component of DMaaP/MR ? If so, Is it packaged within MR docker container or does it have to be deployed separately? Thanks a lot, Jane F.

  2. sunil unnava

    Frieder, We are not using the DME2  in the MR now. For deploying the MR, one need to deploy only Kafka and Zookeeper

    1. Evguenia (Jane) Freider

      Thanks Sunil. Then the design diagram on the MR Architecture page is misleading as it shows DME2 clearly. I also have a related question - as I understand Beijing now has Data Bus Controller which sits on top of MR and DR and ultimately provides a kind of wrapper API. The attached picture says that Bus Controller saves data into DCAE Persistent Storage. This sounds very strange to me. Does it mean that DMaaP can't be used stand-alone and requires DCAE always?? I'd appreciate a clarification on this. Thanks, Jane F.

  3. sunil unnava

    DMaaP MR can be used without the Bus Controller. Bus Controller is used to provision the AAF permissions and creating the topic etc.

    1. Dominic Lunanuova

      To add a few more words:  

      • in Beijing, the DMaaP transport service available is the Message Router (MR - for messages).  A future release should also offer Data Router (DR - for files).  
      • app usage of these services is focused on the pub/sub API, but before that can be attempted, the service needs to be provisioned.  
      • The Bus Controller API is provided to apps so they don't need to learn multiple DMaaP provisioning APIs and also to unify the authorization model.
        • The data persistence represented is only for provisioning data.
        • The value of this API may be unclear in a Release before geo-redundancy is supported, DR is included, and full AAF integration is understood.  But apps will be better positioned for these future complexities by using the Bus Controller API to provision now.

      1. Evguenia (Jane) Freider

        Thanks Dominic. So are you saying DR is not in Beijing release? I was under the impression it is.

        If understand you correctly you are saying messages are not really persisted, just the topics info is ... Note, in the Bus Controller description I saw that data persistence is optional, meaning topics are stored in a cache of some kind then?

        1. Dominic Lunanuova

          re DR in Beijing:  AFAIK, DR was descoped.

          re prov data persistence in Bus Controller: correct, when persistence is not configured, prov data is stored in a memory cache which would be lost if the container is restarted.  This is handy for when the overhead of Postgres install and config is an issue.