Versions Compared

Old Version 26

changes.mady.by.user Varun Gudisena

Saved on

compared with

New Version 27

changes.mady.by.user Varun Gudisena

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

2. Data Transport - the transport of data intra & inter data centers. The transport will support both file base based and message base based data movement. The Data Transport process needs to provide the ability to move data from any system to any system with minimal latency, guaranteed delivery and highly available solution that supports a self-subscription model that lowers initial cost and improves time to market.

...

  • It is based on Pub/Sub architecture, not Point to Point. Adding new subscribers (targets) for existing published feed requires no work on the publisher (Source side)
  • Agnostic to source and sink of data residing in an RDBMS, NoSQL DB, other DBMS, Flat Files, etc.
  • Does not tightly couple publisher and subscriber endpoints.
  • Ability to track completed file transmissions (via log records) 
  • Using HTTP as File transfer Protocol allows customers to utilize the language and frameworks that they are most comfortable with.
  • Does not require a contract (connectivity configuration) between two servers (not server specific Point to Point). It’s simply a contract between two applications.
  • Automatically retries delivery to subscriber for 24 hours. If it still fails, it will be marked as permanently failed.
  • User authentication for each publish request.
  • Authorization for each end point (IP of publisher) determines if allowed to publish or not.
  • Supports low latency for transmission of large files.
  • Feed can be defined with a guaranteed delivery option. This includes dealing with subscribers that are down for an extended amount of time.
  • Provide support for real time streaming of data.


In DMaaP Message Router, Restful web service is exposed to client to perform any needed action with Kafka. After getting the request it calls the Message router service layer which is created using AJSC. AJSC finally calls Kafka services and response is sent back. 

...

  • How does this align with external standards/specifications?
    • APIs/Interfaces - RESTful web service, Kafka

  • Are there dependencies with other open source projects?

    • AJSC (container framework based on Spring and developed by "AT&T Common platform" Team)

    • DME ( Direct Messaging Engine and developed by "AT&T Common platform" Team)

    • AAF ( Application Authorization Framework )

    • Apache Kafka

    • Zookeeper.

...