References

CPS-1361 - Getting issue details... STATUS


Brief

After registering cmHandles through /ncmpInventory/v1/ch API and monitoring the cps.data-updated-events topic, the generated event is getting larger by each new cmHandle. After several thousand cmHandles, this causes org.apache.kafka.common.errors.RecordTooLargeException to be thrown and event publishing fails. Side effect is that CPS starts to consume CPU constantly and service performance degrades for subsequent cmHandle registrations.

The number of cmHandles after which the issue occurs depends on when the message size hits the limit of the producer's message.max.bytes configuration (1MB by default).

What is trying to be achieved:

Current Output 
{
  "schema": "urn:cps:org.onap.cps:data-updated-event-schema:v1",
  "id": "55bcae1d-91cf-4864-bc81-5d3fc2fa3b8d",
  "source": "urn:cps:org.onap.cps",
  "type": "org.onap.cps.data-updated-event",
  "content": {
    "observedTimestamp": "2022-11-02T12:53:19.591+0000",
    "dataspaceName": "NCMP-Admin",
    "schemaSetName": "ncmp-dmi-registry-model",
    "anchorName": "ncmp-dmi-registry",
    "operation": "UPDATE",
    "data": {
      "dmi-registry:dmi-registry": {
        "cm-handles": [
          {
            "id": "B615E09903D6AFBD08AFEF6F869329F8",
            "dmi-service-name": "",
            "dmi-data-service-name": "http://some-data-service-name:8080",
            "dmi-model-service-name": "http://some-model-service-name:8080",
            "additional-properties": [
              {
                "name": "someKey",
                "value": "someValue"
              }
            ],
            "state": {
              "cm-handle-state": "READY",
              "last-update-time": "2022-11-02T12:44:30.593+0000",
              "data-sync-enabled": false,
              "datastores": {
                "operational": {
                  "sync-state": "NONE_REQUESTED"
                }
              }
            }
          },
          {
            "id": "452E73D23BF444023784946E09525192",
            "dmi-service-name": "",
            "dmi-data-service-name": "http://some-data-service-name:8080",
            "dmi-model-service-name": "http://some-model-service-name:8080",
            "additional-properties": [
              {
                "name": "someKey",
                "value": "someValue"
              }
            ],
            "state": {
              "cm-handle-state": "READY",
              "last-update-time": "2022-11-02T12:53:19.590+0000",
              "data-sync-enabled": false,
              "datastores": {
                "operational": {
                  "sync-state": "NONE_REQUESTED"
                }
              }
            }
          }
        ]
      }
    }
  }
}


Desired Output 
{
  "schema": "urn:cps:org.onap.cps:data-updated-event-schema:v1",
  "id": "55bcae1d-91cf-4864-bc81-5d3fc2fa3b8d",
  "source": "urn:cps:org.onap.cps",
  "type": "org.onap.cps.data-updated-event",
  "content": {
    "observedTimestamp": "2022-11-02T12:53:19.591+0000",
    "dataspaceName": "NCMP-Admin",
    "schemaSetName": "ncmp-dmi-registry-model",
    "anchorName": "ncmp-dmi-registry",
    "operation": "UPDATE",
    "data": {
      "dmi-registry:dmi-registry": {
        "cm-handles": [
          {
            "id": "B615E09903D6AFBD08AFEF6F869329F8",
            "dmi-service-name": "",
            "dmi-data-service-name": "http://some-data-service-name:8080",
            "dmi-model-service-name": "http://some-model-service-name:8080",
            "additional-properties": [
              {
                "name": "someKey",
                "value": "someValue"
              }
            ],
            "state": {
              "cm-handle-state": "READY",
              "last-update-time": "2022-11-02T12:44:30.593+0000",
              "data-sync-enabled": false,
              "datastores": {
                "operational": {
                  "sync-state": "NONE_REQUESTED"
                }
              }
            }
          }
        ]
      }
    }
  }
}


Resolution


Currently the code is behaving as expected so the solution will require a change in behaviour. The event payload carries the whole data tree as it is designed to contain the details of the Anchor. 

The solution will require only the affected cmhandles to be included in the event however it should also maintain current behaviour for other use cases. The current POC proposal can be seen below

https://gerrit.onap.org/r/c/cps/+/132305




  • No labels