How the data movement is established between SDC and DMAAP?

 
1
0
-1

I'm new in exploring SDC .Here I want to understand the data movement between SDC and DMAAP.

I have the following assumptions and correct me if I'm wrong,

1.The communication between SDC and other ecomp components(MSO,DCAE,etc) is established using DMAAP.

2.In DMAAP, the data are shared using data-router and messages/notifications are shared using message router.   (This DMAAP is implemented over kafka)

3.The data router in DMAAP uses jetty server to accept REST inputs from producer and consumer.(Here the producer and consumer be SDC or other ECOMP components) 

4.Here in SDC,where will be the Rest inputs are created/initiated inorder to move data to Data router in DMAAP?.



  1. Dominic Lunanuova

    Just to clarify:

    in point 2, to avoid ambiguity, the last phrase should be "(The DMAAP Message Router is implemented over kafka)".

    in point 3.  The Data Router jetty server accepts REST inputs from a producer, but not a consumer.  The consumer must implement an http server which accepts REST inputs from Data Router.

  2. vignesh k

    Thank you, Dominic Lunanuova .Can u plz. clarify over the fourth one on which I get blocked.i.e.where/how the REST inputs are created in SDC ?.

  3. Dominic Lunanuova

    vignesh k  I'm not sure how to respond to your #4 because I'm not familiar with SDC.  But in reading SDC Dependencies  it says SDC Distribution Client "create topics in the Dmaap".  This can be accomplished using the DMaaP Bus Controller API.

    The other "information expected" listed is "and subscribers to the status topic".  I'm not sure how to parse this.  Will SDC Distrubution Client also provision subscribers (in which case the Bus Controller API is used), or will it behave as a subscriber to some status topic (in which case it uses the Message Router consumer API)?  Both these APIs are described on DMaaP API page.

    So, if I am not hitting on your question, perhaps you can clarify what REST inputs you are concerned about.

  4. vignesh k

    Hi sir Dominic Lunanuova,

    Actually what I try to understand is , how SDC initiates the process of distribution. In SDC Dependencies the outgoing direction from SDC to DMAAP denotes the creation of Topics,distributing service data (as artifacts),etc .But from which component in SDC code this distribution process is done.And what is the role of cambria API and UEB listener here.

    Can you plz. clarify over this .

    Currently I deployed message router(dmaap,kafka,zookeeper) and SDC(fe,be,etc) as dockers. I need to understand that how the distribution from SDC to DMAAP gets done.The above issue blocks to understand the relationship b/w SDC and DMAAP.

    Thanks in advance!

CommentAdd your comment...

1 answer

  1.  
    1
    0
    -1
    1. "creation of Topics"
      1. in Amsterdam, only unauthenticated Topics are supported.  Unauthenticated Topics are created upon first message publish transaction.  Therefore, the only provisioning step needed is for the publisher and subscriber to agree on the Topic name.  I suggest adopt a convention like "unauthenticated.SDC-NOTIF-TOPIC" to emphasize the fact that this Topic is unauthenticated (useful for keeping separate namespaces in future environments where authentication will also be supported).
      2. in Beijing, DMaaP will support authenticated Topics.  Instructions still to be developed, but high level provisioning steps for a component like SDC will include:
        1. obtain AAF credential
        2. Topic provisioning using the DMaaP Bus Controller API, which will create the authenticated topic on MR, and grant the associated pub/sub permissions in AAF
    2. "distributing service data (as artifacts)" - Note: I assume this means messages with content consisting of the artifacts you mention.
      1. Since SDC is distributing the data, it has the role of a Publisher.
      2. Both "cambria" and "UEB" are internal  names for prior versions of MR, so be careful about API compatibility with ONAP MR version, though conceptually you should be OK.  I will ask a member of DMaaP MR team to clarify this more.
    3. 'how the distribution from SDC to DMAAP gets done"
      1. Message Router docker container is running an HTTP service supporting its MR Producer API.
      2. SDC should implement an HTTP client which publishes on a Topic
        1. e.g. using curl as an example:
          curl -X POST -d @artifactFile http://<MR docker IP>:<API Port>/<API Path>/events/unauthenticated.SDC-NOTIF-TOPIC
      3. Other components which consume these messages need to implement a Subscriber on the appropriate Topic (using the Consumer API)
      4. When you get to using authenticated Topics, the only difference to a Publisher or Subscriber is to pass their AAF credentials in the API call using Basic Authentication.
    4. For a recorded demo providing a more indepth overview, see DMaaP
    1. vignesh k

      Hi Sir Dominic Lunanuova,

      Thank you for the information. I've gone through #1,#4 and I'm also able to do #3 as you directed. But I'm bit confused on how to start with #2 . I've glanced through the coding part in SDC. And I've the following assumptions and please correct me if I'm wrong,

      1.Initially,there is a distribution client in producer(sdc-distribution client in SDC) and consumer(asdc-controller in MSO) and this must be started inorder to communicate with each other.

      2.The ECOMP component can able to download the artifacts from SDC (as mentioned in SDC Distribution client AID) once they received notifications about the new service through DMAAP. (Assuming SDC distribution client gets started in both SDC and ECOMP component). 

      3.Since the MR(DMAAP) uses jetty web server there is a REST request from SDC to DMAAP for event notifications(And I'm clueless on which part in SDC this REST request arises).

      Thank you!

    2. Dominic Lunanuova

      hi vignesh k

      1. yes.  Also the Message Router must be started.  Do note that since Message Router queues messages until retrieved by Subscribers, the ECOMP component doesn't necessarily have to be running at the time that SDC publishes a message.  This is one of the advantages of using a message bus.
      2. yes.  I focused on Section 6.1 of the AID you cite and that supports your description.  To clarify more, the artifact download API is not a DMaaP function.   DMaaP MR is used to notify components about the availability of an artifact on SDC.
      3. The MR Producer/Consumer API is an HTTP REST interface.  (not positive it is Jetty, but that shouldn't matter to clients).  Client calls to this API were described in the prior answer.  Where those calls belong within the SDC design is outside the scope of DMaaP, except to offer an example pattern:
        1. Some SDC code might drop an artifact into a directory whenever it is "ready for distribution"
        2. A watcher process might monitor that directory, and send notification messages to announce the artifact availability.  THIS is the place for the Producer API call.
    3. vignesh k

      Thank you sir! Dominic Lunanuova

    CommentAdd your comment...