Project Name:
- Proposed name for the project:
DMaaP – Data movement as a platform
- Proposed name for the repository:
DMaaP
Project description:
DMaaP is a premier platform for high performing and cost effective data movement services that transports and processes data from any source to any target with the format, quality, security, and concurrency required to serve the business and customer needs.
...
3. Data Processing - the low latency and high throughput data transformation, aggregation, and analysis. The processing will be elastically scalable and fault-tolerant across data centers. The Data processing needs to provide the ability to process both batch and near real-time data.
Scope:
DMaaP has four components:
...
- It is based on Pub/Sub architecture, not Point to Point. Adding new subscribers (targets) for existing published feed requires no work on the publisher (Source side)
- Agnostic to source and sink of data residing in an RDBMS, NoSQL DB, other DBMS, Flat Files, etc.
- Does not tightly couple publisher and subscriber endpoints.
- Ability to track completed file transmissions (via log records)
- Using HTTP as File transfer Protocol allows customers to utilize the language and frameworks that they are most comfortable with.
- Does not require a contract (connectivity configuration) between two servers (not server specific Point to Point). It’s simply a contract between two applications.
- Automatically retries delivery to subscriber for 24 hours. If it still fails, it will be marked as permanently failed.
- User authentication for each publish request.
- Authorization for each end point (IP of publisher) determines if allowed to publish or not.
- Supports low latency for transmission of large files.
- Feed can be defined with a guaranteed delivery option. This includes dealing with subscribers that are down for an extended amount of time.
Data Router:
Architecture Alignment:
How does this project fit into the rest of the ONAP Architecture?
- What other ONAP projects does this project depend on?
- AAF
- AAF
- Are there dependencies with other open source projects?
- AJSC (container framework based on Spring, developed by "AT&T Common platform" TeamJSC ( Java Services Container )
DME ( Direct Messaging Engine , developed by "AT&T Common platform" Team )
AAF ( Application Authorization Framework )
Apache Kafka
Zookeeper.
- Apache Tomcat
Release 1:
Release 1 provides the following features:
- Pub/sub messaging metaphor to broaden data processing opportunities.
- A single solution for most event distribution needs to simplify our solutions.
- Horizontal scalability: Add servers to the cluster to add capacity. (Our initial installations are expected to handle at least 100,000 1KB msgs/sec per data center.)
- Durability: Hardware failure in the cluster should not impact service, and messages should never be lost.
- High throughput: Consumers must be able to distribute topic load across multiple systems.
- Easy integration via RESTful HTTP API
Resources:
- Primary Contact Person - Ram Bhanu Ramesh (ATT), Ram Koya (ATT), John Murray(ATT), DOMINIC LUNANUOVA (ATT)
- Names, gerrit IDs, and company affiliations of the committers :
- Names and affiliations of any other contributors
Project Roles (include RACI chart, if applicable)
Name Gerrit ID Company Email Time Zone Bhanu Ramesh AT&T bg6954@att.com USA, EST Ram Koya AT&T Dallas, USA
CST/CDTJohn Murray AT&T Bedminster, USA EST/EDT Dominic Lunanuova AT&T MiddleTown, USA EST/EDT
Other Information:
- link to seed code (if applicable): https://github.com/att/dmaap-framework
- Vendor Neutral
- The current seed code has been already scanned ( Using Fossology and Blackduck) and cleaned up to remove all proprietary trademarks, logos.
- Subsequent modification to the existing seed code should continue to follow the same scanning and clean up principles
Key Project Facts
Project Name:
- JIRA project name: Data Movement as a Platform
- JIRA project prefix: DMAAP-
Repo name: DMaaP
Lifecycle State: incubation
Primary Contact: Bhanu Ramesh (PTL Delegate) / Ram Koya
Project Lead: Ram Koya Bhanu Ramesh (PTL Delegate) / Ram Koya
mailing list tag [Should match Jira Project Prefix]
Committers:
Ram Koya rk541m@att.com
Varuneshwar Gudisena vg411h@att.com
Habib Madani habib.madani@huawei.com
Xinhui Li lxinhui@vmware.com
Contributors:
Ramdas Sawant rs873m@att.com
Bhanu Ramesh bg6954@att.com
Ramkumar Sembaiyan rs857c@att.com
Ramdas Sawant rs873m@att.com
Vikram Singh vs215k@att.com
Sai Gandham sg481n@att.com
Dominic Lunanuova dgl@research.att.com
Catherine Lefèvre cl664y@intl.attHabib Madani habib.madani@huawei.com
Link to approval of additional submitters: