...
draw.io Diagram | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Example heap dump from OOME during CM-handle search
...
Code Block | ||
---|---|---|
| ||
curl --location 'http://localhost:8883/ncmp/v1/ch/searches' \
--header 'Content-Type: application/json' \
--data '{
"cmHandleQueryParameters": [
{
"conditionName": "hasAllProperties",
"conditionParameters": [ {"Color": "yellow"}, {"Size": "small"} ]
}
]
}' |
Implementation of CM-handle Search and ID Search
While it is difficult to give a complete explanation of CM-handle Search functionality, a workflow for a particular CM-handle Search will be given to illustrate. In this case, the following Rest request will be issued:
POST http://{{CPS_HOST}}:{{CPS_PORT}}/ncmp/v1/ch/searches
with the following request body:
Code Block | ||
---|---|---|
| ||
{ "cmHandleQueryParameters": [ { "conditionName": "hasAllModules", "conditionParameters": [ {"moduleName": "ietf-netconf"} ] }, { "conditionName": "hasAllProperties", "conditionParameters": [ {"Color": "yellow"}, {"Size": "small"} ] } ] } |
In this case, a search will be executed returning all CM-handles having the Yang module "ietf-netconf" and the public property Color="yellow" and the public property Size="small". These three condition parameters will be combined using logical AND (or set intersection), meaning only CM-handles satisfying all three criteria will be returned.
The relevant code is in cps-ncmp-service/src/main/java/org/onap/cps/ncmp/api/impl/NetworkCmProxyCmHandleQueryServiceImpl.java
executeModuleNameQuery: The module name search works by executing a search for Anchors with the given module references (a single DB query is run for all modules)
- cpsAnchorService.queryAnchorNames(NFP_OPERATIONAL_DATASTORE_DATASPACE_NAME, moduleNamesForQuery)
queryCmHandlesByPublicProperties: The properties search works by executing a separate Cps Path Query (thus separate DB query) for each property pair:
//public-properties[@name='Color' and @value='yellow']
- //public-properties[@name='Size' and @value='small']
- Note additional query parameters are supported, such as Cps Path Query and query by Trust Level. These would result in additional DB queries.
The results of each of these are sets of CM-handle IDs, which are combined using set intersection (Set::retainAll). After the final set of CM-handle IDs has been computed:
- In the case of ID search, the set will be returned as a list of IDs from the Rest controller:
- return ResponseEntity.ok(List.copyOf(cmHandleIds));
- In the case of CM-handle search, the DB will be queried again to find all CM-handles with the given IDs:
- return getNcmpServiceCmHandles(cmHandleIds);
Here is a diagram of the flow:
draw.io Diagram | ||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Proposed Solution
It is proposed to create an end-to-end streaming solution, from Persistence layer to Controller. A Proof of Concept will be constructed to document challenges and investigate performance characteristics.
...
draw.io Diagram | |||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
NOTE: Spring Data has stream support, and will page results given appropriate settings. For example, JdbcTemplate::queryForStream will page at 100 when following settings are used:
spring.jdbc.template.fetch-size=100
spring.datasource.hikari.auto-commit=false
Here is some source code showing how the streams API would be used:
...