Project Name:
- Proposed name for the project:
DMaaP – Data movement as a platform - Proposed name for the repository:
com.att.dmaap
Project description:
DMaaP is a premier platform for high performing and cost effective data movement services that transports and processes data from any source to any target with the format, quality, security, and concurrency required to serve the business and customer needs.
...
3. Data Processing - the low latency and high throughput data transformation, aggregation, and analysis. The processing will be elastically scalable and fault-tolerant across data centers. The Data processing needs to provide the ability to process both batch and near real-time data.
Scope:
Message Router will include the following functionality:
...
- It is based on Pub/Sub architecture, not Point to Point. Adding new subscribers (targets) for existing published feed requires no work on the publisher (Source side)
- Agnostic to source and sink of data residing in an RDBMS, NoSQL DB, other DBMS, Flat Files, etc.
- Does not tightly couple publisher and subscriber endpoints.
- Ability to track completed file transmissions (via log records)
- Using HTTP as File transfer Protocol allows customers to utilize the language and frameworks that they are most comfortable with.
- Does not require a contract (connectivity configuration) between two servers (not server specific Point to Point). It’s simply a contract between two applications.
- Automatically retries delivery to subscriber for 24 hours. If it still fails, it will be marked as permanently failed.
- User authentication for each publish request.
- Authorization for each end point (IP of publisher) determines if allowed to publish or not.
- Supports low latency for transmission of large files.
- Feed can be defined with a guaranteed delivery option. This includes dealing with subscribers that are down for an extended amount of time.
Architecture Alignment:
- How does this project fit into the rest of the ONAP Architecture?
DMaaP has four components:
- Message Router (MR) -Message Router is a reliable, high-volume pub/sub messaging service with a RESTful HTTP API. It is intended to be deployed by Platform Service providers so that it is available to Platform clients as a web service. The service is initially built over Apache Kafka, with some important additional features which focus on integration with other Platform services such as provisioning and authorization.
- Data Router (DR) - The Data Routing System project is intended to provide a common framework by which data producers can make data available to data consumers and a way for potential consumers to find feeds with the data they require. The interface to DR is exposed as a RESTful web service known as the DR Publishing and Delivery API
- Data Movement Director (DMD) - A client to DMaaP platform to publish & subscribe data.
- Data Bus Controller - Provisioning API of the Data Movement Platform.
- Message Router (MR) -
...
- How does this align with external standards/specifications?
- APIs/Interfaces
- Information/data models
- Are there dependencies with other open source projects?
AJSC, DME, AAF, Apache Kafka and Zookeeper.
Resources:
- Primary Contact Person - Ram Koya (ATT), John Murray(ATT), DOMINIC LUNANUOVA (ATT)
- Names, gerrit IDs, and company affiliations of the committers :
- Names and affiliations of any other contributors
Project Roles (include RACI chart, if applicable)
Name Gerrit ID Company Email Time Zone Ram Koya AT&T Dallas, USA
CST/CDTJohn Murray AT&T Bedminster, USA EST/EDT Dominic Lunanuova AT&T MiddleTown, USA EST/EDT
Other Information:
- link to seed code (if applicable)
- Vendor Neutral
- if the proposal is coming from an existing proprietary codebase, have you ensured that all proprietary trademarks, logos, product names, etc., have been removed?
- Meets Board policy (including IPR)
Use the above information to create a key project facts section on your project page
Key Project Facts
Project Name:
- JIRA project name:
- JIRA project prefix:
...