CPS-505 - Getting issue details... STATUS


Requirements

Characteristics

#Interface(s) / RequirementCapabilitiesNotes
1

CPS-NCMP-I-01
Registration Performance

  1. NCMP should support discovery of up to 10k cmhandles
  2. NCMP should discovery cmhandles to READY state (with modules) at a rate of 4 per second assumptions :
    1. 100 cm handles per request
    2. 10 modules / cmhandle
    3. 95% module overlap across cmhandles
  1. This KPI is for an E2E use-case depending on the performance of components outside NCP e.g. the 'modelAdapter'. Some assumption about the performance of this external dependency should be defined too.
    1. This KPI should include a cloud environment definition such as available resources like CPU and memory and same for the Postgress DB configuration.
2

CPS-NCMP-I-01
Deletion Performance
CPS-1173 - Getting issue details... STATUS

  • Environment Requirements for :

    1. CPS and NCMP

    requests:
        cpu: 2000m
        memory: 2Gi
    limits:
        memory: 3Gi
        cpu: 3000m

    2. Postgres

    requests:
        cpu: 4000m
        memory: 1Gi
     limits:
        memory: 3Gi
        cpu: 6000m

  • Starting number of CM Handles: 20,000 and percentage deleted: 100
  • Number of deletion requests: 100 cm handles per request with a single thread.
  • De-Register CM Handles once all cm handles moved into READY state.
  • Configuration Options

    • CM Handle LCM Event enabled

       notifications:    
         enabled: true

    • CPS-Temporal Notification should be disabled for the NCMP Registry dataspace
      notification:
        data-updated:

          filters:
            enabled-dataspaces: "NON-EXISTING-DATASPACE"


3

CPS-NCMP-I-01
Cm Handle Inventory Query  Performance

CPS-1494 - Getting issue details... STATUS


#ParameterExpectationNotes
1Response Time 1 <40 second
<60 second (stretch)


2Query complexity1 or more attributes on 1 Fragment (jsonb) object 
3Response payload sizeTBD KB
4Maximum registered #cm handles 20,000This will effect the internal query time
5Test Environment
  1. CPS and NCMP

requests:
    cpu: 2000m
    memory: 2Gi
limits:
    memory: 3Gi
    cpu: 3000m

2. Postgres

requests:
    cpu: 4000m
    memory: 1Gi
 limits:
    memory: 3Gi
    cpu: 6000m


6Concurrent request1
7Request FrequencyLow no more than 1 per minute

Background

When a new CM handle is encountered by the DMI plugin, NCMP is notified via REST. Once NCMP picks up the new CM handle it needs to determine what modules exist for it in the database and figure out which modules are missing. DMI plugin is contacted for all modules for this new node and they are matched to what CPS has. Missing modules are then retrieved from DMI and entered into the database.



High Level Proposal of Work to be done

  1. Call dmi–plugin rest endpoint to retrieve all modules on new node (depends CPS-483 and CPS-531)
  2. Call CPS-Core rest endpoint to get all existing modules in cps-core (depends CPS-506)
  3. Calculate difference (delta) 
  4. Call dmi–plugin rest endpoint to retrieve missing modules CPS-483
  5. Add missing modules to cps-core to anchor (cm handle) (depends CPS-508)

Implementation Proposal

Within the DMI plugin there is a method where cm handles are sent to a registration method in NCMP.

From: DmiServiceImpl.registerCmHandlesWithNcmp(final String jsonData) To: NetworkCmProxyController.updateDmiPluginRegistration() 


Dmi Plugin Service
 @Override
    public void registerCmHandles(final List<String> cmHandles) {
        final CmHandleOperation cmHandleOperation = new CmHandleOperation();
        cmHandleOperation.setDmiPlugin(dmiPluginProperties.getDmiServiceName());
        final List<CreatedCmHandle> createdCmHandleList = new ArrayList<>();
        for (final String cmHandle: cmHandles) {
            final CreatedCmHandle createdCmHandle = new CreatedCmHandle();
            createdCmHandle.setCmHandle(cmHandle);
            createdCmHandleList.add(createdCmHandle);
        }
        cmHandleOperation.setCreatedCmHandles(createdCmHandleList);
        final String cmHandlesJson;
        try {
            cmHandlesJson = objectMapper.writeValueAsString(cmHandleOperation);
        } catch (final JsonProcessingException e) {
            log.error("Parsing error occurred while converting cm-handles to JSON {}", cmHandles);
            throw new DmiException("Internal Server Error.",
                    "Parsing error occurred while converting given cm-handles object list to JSON ");
        }
        final ResponseEntity<String> responseEntity = ncmpRestClient.registerCmHandlesWithNcmp(cmHandlesJson);
        if (!(responseEntity.getStatusCode() == HttpStatus.CREATED)) {
            throw new CmHandleRegistrationException(responseEntity.getBody());
        }
    }


DMI Plugin REST
/**
     * Register a cmHandle with NCMP using a HTTP call.
     * @param jsonData json data
     * @return the response entity
     */
    public ResponseEntity<String> registerCmHandlesWithNcmp(final String jsonData) {
        final var ncmpRegistrationUrl = buildNcmpRegistrationUrl();
        final var httpHeaders = new HttpHeaders();
        httpHeaders.setBasicAuth(cpsProperties.getAuthUsername(), cpsProperties.getAuthPassword());
        httpHeaders.set(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE);
        final var httpEntity = new HttpEntity<>(jsonData, httpHeaders);
        return restTemplate.postForEntity(ncmpRegistrationUrl, httpEntity, String.class);
    }

    private String buildNcmpRegistrationUrl() {
        return UriComponentsBuilder
            .fromHttpUrl(cpsProperties.getBaseUrl())
            .path(cpsProperties.getDmiRegistrationUrl())
            .toUriString();
    }


NCMP REST
 /**
     * Update DMI Plugin Registration (used for first registration also).
     * @param restDmiPluginRegistration the registration data
     */
    @Override
    public ResponseEntity<Void> updateDmiPluginRegistration(
        final @Valid RestDmiPluginRegistration restDmiPluginRegistration) {
        final DmiPluginRegistration dmiPluginRegistration =
            convertRestObjectToJavaApiObject(restDmiPluginRegistration);
        networkCmProxyDataService.updateDmiPluginRegistration(dmiPluginRegistration);
        return new ResponseEntity<>(HttpStatus.CREATED);
    }


NCMP Service
 @Override
    public void updateDmiPluginRegistration(final DmiPluginRegistration dmiPluginRegistration) {
        try {
            final List<PersistenceCmHandle> persistenceCmHandles =
                new ArrayList<>();
            for (final CmHandle cmHandle: dmiPluginRegistration.getCreatedCmHandles()) {
                final PersistenceCmHandle persistenceCmHandle = new PersistenceCmHandle();
                persistenceCmHandle.setDmiServiceName(dmiPluginRegistration.getDmiPlugin());
                persistenceCmHandle.setId(cmHandle.getCmHandle());
                persistenceCmHandle.setAdditionalProperties(cmHandle.getCmHandleProperties());
                persistenceCmHandles.add(persistenceCmHandle);
            }
            final PersistenceCmHandlesList persistenceCmHandlesList = new PersistenceCmHandlesList();
            persistenceCmHandlesList.setCmHandles(persistenceCmHandles);
            final String cmHandleJsonData = objectMapper.writeValueAsString(persistenceCmHandlesList);
            cpsDataService.saveListNodeData(NCMP_DATASPACE_NAME, NCMP_ANCHOR_NAME, NCMP_DATA_TOP_PATH,
                cmHandleJsonData);
        } catch (final JsonProcessingException e) {
            throw new DataValidationException(
                "Parsing error occurred while processing DMI Plugin Registration" + dmiPluginRegistration, e
                .getMessage(), e);
        }
    }


Return cmHandleJsonData from updateDmiPluginRegistration() in NetworkCmProxyDataService

Append deriving modules after line 127 in method updateDmiPluginRegistration() in NetworkCmProxyDataController

deriveModulesForCmHandle(cmHandleJsonData);


get cm handles from string
call dmi to get modules for this cmhandle
compare with cps and get delta
call dmi for delta modules
return 
Map<String, String> newYangResourcesModuleNameToContentMap


Alternative flow

DMI calls ncmp with new cm handle

ncmp sends back created

dmi then calls get new modules 

get cm handles from string
call dmi to get modules for this cmhandle
compare with cps and get delta
call dmi for delta modules
return 
Map<String, String> new


  • No labels