...
- We want all cm-Handles where both things are true.
- If name is empty silently ignore, if value is empty (edge case test?)
- Those properties also need to exist.
- If query body does not follow supported structure return 400.
- Empty query will return all cm-Handles.
...
- Demo - with "and" behavior
- CI Test - two attributes
Issues/Decisions
...
Issue | Notes | Decision | |
---|---|---|---|
1 | Does request body need to declare "publicCmHandleProperties"? | Do we need to |
explicitly declare "publicCmHandleProperties"? Will there be another possible variation to this in the future?
|
|
|
|
|
|
|
Yes | |
2 | Are public properties always stored (in postgress) |
in the format of "name" : x, "value": y? (does NOT affect implementation) | In the fragment table there are example that follow this format
|
|
| Yes. |
But all this is hidden when using CpsPath type query |
3 |
Will there only ever be 1 |
KV pairs in public properties (in the DB)? | Yes |
, as each property is stored in a separate list-item Fragment But all this is hidden when using CpsPath type query |
4 |
Does the order matter? | Does
|
|
|
|
==
|
|
|
|
|
| No |
But all this is hidden when using CpsPath type query |
5 |
Should search be case sensitive? |
Analysis/Implementation Proposal
High level Jiras:
Jira server ONAP Jira serverId 425b2b0a-557c-3c0c-b515-579789cceedb key CPS-901 Jira server ONAP Jira serverId 425b2b0a-557c-3c0c-b515-579789cceedb key CPS-902 Jira server ONAP Jira serverId 425b2b0a-557c-3c0c-b515-579789cceedb key CPS-903 Jira server ONAP Jira serverId 425b2b0a-557c-3c0c-b515-579789cceedb key CPS-904
Possible High-Level Implementation Steps:
Match all xpaths that contain "public" in fragment table
Code Block | ||
---|---|---|
| ||
SELECT xpath, attributes #>> '{}' AS attributes
FROM fragment
WHERE fragment.xpath LIKE '%public%' |
Iterate over the list and retrieve all attributes
- Iterate over the attributes and collect those instances that match
- Return collected list of cm handles
depend on CPSPath functionality - may be case-sensitive https://docs.onap.org/projects/onap-cps/en/latest/cps-path.html | Yes, that is the currently implement as such CpsPath type query. Given it is mostly machine-to-machine type queries this is good enough and performs better | |||||||
6 | What format should the response take? | We have two options currently:
2. Return CM Handle Objects
Contact Tony Finnerty & kieran mccarthy regarding this. | It has been decided to implement this using a list of cm handles | |||||
7 | what if we have valid entries and one entry is empty? | We would return all cm handles and others that match would be included in this list anyway. → To avoid this should we first check the entries int map for empty entries as further processing is unnecessary in this case. | Discussed with Toine Siebelink and implementation changes are required for handling edges cases. we will handle empty and unknown properties separately. | |||||
8 | Regarding empty/missing property in request e.g. { "publicCmHandleProperties": { | Do we want to throw an exception if there is an empty property? Do we silently ignore them and move on? should we send two lists back... one with matching cm handles and one with those that failed processing? | We will throw a DataValidationException if an entry contains an empty property |
Analysis/Implementation Proposal
...
High level Jiras:
Jira server ONAP Jira serverId 425b2b0a-557c-3c0c-b515-579789cceedb key CPS-901 Jira server ONAP Jira serverId 425b2b0a-557c-3c0c-b515-579789cceedb key CPS-902 Jira server ONAP Jira serverId 425b2b0a-557c-3c0c-b515-579789cceedb key CPS-903 Jira server ONAP Jira serverId 425b2b0a-557c-3c0c-b515-579789cceedb key CPS-904
Possible High-Level Implementation Steps:
Match all xpaths using CPS Path Query
Iterate over the list and retrieve all attributes
- Iterate over the attributes and collect those instances that match
- Return collected list of cm handles
Interface Proposal
# | URI | Design Notes | Comment(s) | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | POST /ncmp/v1/data/ch/searches | Scenario : Request received to return all cm handles matching properties given Request Body
Response Body Example 1
|
Below is a sample yaml for OpenAPI.
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
executeSearchForMatchingPublicProperties:
post:
description: Execute search to get all cm handles for the given public properties
tags:
- network-cm-proxy
summary: Execute cm handle search using
operationId: executeSearchForMatchingPublicProperties
requestBody:
required: true
content:
application/json:
schema:
$ref: 'components.yaml#/components/schemas/PublicProperties'
responses:
200:
description: OK
content:
application/json:
schema:
type: array
items:
type: string
400:
$ref: 'components.yaml#/components/responses/BadRequest'
401:
$ref: 'components.yaml#/components/responses/Unauthorized'
403:
$ref: 'components.yaml#/components/responses/Forbidden'
404:
$ref: 'components.yaml#/components/responses/NotFound'
500:
$ref: 'components.yaml#/components/responses/InternalServerError' |
Testing
...
Gerrit Link
https://gerrit.onap.org/r/c/cps/+/127541
Data Used for Testing
# | xpath | attributes |
---|---|---|
1 | /dmi-registry/cm-handles[@id='PNFDemo']/public-properties[@name='Contact2'] | {"name": "Contact2", "value": "storeemail2@bookstore.com"} |
2 | /dmi-registry/cm-handles[@id='PNFDemo']/public-properties[@name='Contact'] | {"name": "Contact", "value": "newemailforstore@bookstore.com"} |
3 | /dmi-registry/cm-handles[@id='Bookstore5']/public-properties[@name='color'] | {"name": "color", "value": "won't match"} |
4 | /dmi-registry/cm-handles[@id='Bookstore4']/public-properties[@name='color'] | {"name": "color", "value": ""} |
5 | /dmi-registry/cm-handles[@id='Bookstore3']/public-properties[@name='color'] | {"name": "color", "value": "red"} |
6 | /dmi-registry/cm-handles[@id='Bookstore2']/public-properties[@name='Contact'] | {"name": "Contact", "value": "newemailforstore2@bookstore.com"} |
7 | /dmi-registry/cm-handles[@id='Bookstore1']/public-properties[@name='Contact'] | {"name": "Contact", "value": "newemailforstore@bookstore.com"} |
8 | /dmi-registry/cm-handles[@id='Bookstore6']/public-properties[@name='color'] | {"name": "color", "value": "12345"} |
9 | /dmi-registry/cm-handles[@id='Bookstore7']/public-properties[@name='color'] | {"name": "color", "value": "12345"} |
10 | /dmi-registry/cm-handles[@id='Bookstore8']/public-properties[@name='size'] | {"name": "size", "value": "large"} |
11 | /dmi-registry/cm-handles[@id='Bookstore8']/public-properties[@name='color'] | {"name": "color", "value": "red"} |
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
{
"dmiPlugin": "http://172.24.170.77:8783",
"createdCmHandles": [
{
"cmHandle": "Bookstore1",
"publicCmHandleProperties": {
"Contact": "newemailforstore@bookstore.com"
}
},
{
"cmHandle": "Bookstore2",
"publicCmHandleProperties": {
"Contact": "newemailforstore2@bookstore.com"
}
},
{
"cmHandle": "Bookstore3", | ||||||
Code Block | ||||||
| ||||||
/** * Retrieve public properties for given cm handle. * * @param publicProperties the public properties to match * @return lit of cm handles that match */ @Override public List<String> getCmHandlesForMatchingPublicProperties(final Map<String, String> publicProperties) { final String retrievePublicFragments = "SELECT xpath, attributes #>> '{}' AS attributes FROM fragment WHERE fragment.xpath LIKE '%public%';"; final List<Object[]> resultsAsObjects = entityManager.createNativeQuery(retrievePublicFragments).getResultList(); "publicCmHandleProperties": { final Set<String> cmHandlesThatMatch = new TreeSet<>(); "color": "red" final Set<String> allCmHandles = new TreeSet<>(); } for (final Object[] row : resultsAsObjects) {}, { final String xpath = (String) row[0]; "cmHandle": "Bookstore4", final String attributes = (String) row[1]; "publicCmHandleProperties": { allCmHandles.add(getCmHandle(xpath)); "color": "" try { } }, final String attributesAsJsonString = getAttributesAsJsonString(attributes); { "cmHandle": "Bookstore5", final JsonObject attributesAsJsonObject = new JsonParser().parse(attributesAsJsonString) "publicCmHandleProperties": { "color": .getAsJsonObject(); "won't match" } if (nameAndValueMatch(publicProperties, attributesAsJsonObject)) {}, { cmHandlesThatMatch.add(getCmHandle(xpath));"cmHandle": "Bookstore6", "publicCmHandleProperties": { } } catch (final JsonProcessingException jsonProcessingException) { "color": 12345 } jsonProcessingException.getMessage();}, { } } "cmHandle": "Bookstore7", if (publicProperties.get("name").isEmpty() && publicProperties.get("value").isEmpty())publicCmHandleProperties": { return List.copyOf(allCmHandles); "color": "12345" } return List.copyOf(cmHandlesThatMatch); } } private}, String getAttributesAsJsonString(final String attributes) throws JsonProcessingException { final ObjectMapper objectMapper = new ObjectMapper();"cmHandle": "Bookstore8", String attributesAsJsonString = objectMapper.writeValueAsString(attributes); "publicCmHandleProperties": { attributesAsJsonString = attributesAsJsonString.replace("\\", ""); attributesAsJsonString = attributesAsJsonString.replaceFirst("\color"",: "red");, attributesAsJsonString = attributesAsJsonString.substring(0, attributesAsJsonString.length() - 1); return attributesAsJsonString;"size": "large" } private boolean nameAndValueMatch(final Map<String, String> publicProperties, } } ] } final JsonObject attributesAsJsonObject) { return attributesAsJsonObject.get("value").getAsString().equals(publicProperties.get("value")) && attributesAsJsonObject.get("name").getAsString().equals(publicProperties.get("name")); } private String getCmHandle(final String xpath) { if (xpath.indexOf("\'") > - 1) { return xpath.substring(xpath.indexOf("\'") + 1, xpath.indexOf("]") - 1); } return xpath.substring(xpath.indexOf("\"") + 1, xpath.indexOf("]") - 1); } |
Interface Proposal
...
Request Body
|
URL Used for all Requessts
http://localhost:8883/ncmp/v1/data/ch/searches
Results - Happy Path
# | Scenario | Request | Response | Notes/Decisions |
---|---|---|---|---|
1 | Both properties match (Return CM Handles that Match) | { | [ | |
2 | Multiple Entries - both properties match (Return CM Handles that Match) | { | [ | |
3 | No properties given (Return All CM Handles which contain public properties) | { "publicCmHandleProperties": { | [ |
Results - Edge Cases
# | Scenario | Request | Response | Notes/Decisions | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | Value doesn't match (Return Empty Response) | { | [ ] | |||||||||||
2 | Unknown properties given (Return Empty Response) | { | [ ] | |||||||||||
3 | Empty value (valid) (Return CM Handles Returned that Match) | { "publicCmHandleProperties": { | [ | |||||||||||
4 | Empty property (invalid) (BAD_REQUEST) | { "publicCmHandleProperties": { | { | |||||||||||
5 | Multiple entries with one empty property (invalid) (BAD_REQUEST) | { | { | |||||||||||
6 | ||||||||||||||
7 | Apostrophe in cm handle (Exception) *** CPS can create a cm handle with a public property value that has an apostrophe but throws exception when queried *** | { | { | Will handle this as and when it arises | ||||||||||
8 | CPS stores Integers as Strings. | Given the public properties below
When stored in CPS Integers are stored as Strings.
Therefore the two requests below return both cm handles regardless of wanting Integer or String. { { | [ "Bookstore6", "Bookstore7" ] |
Code Block | ||
---|---|---|
| ||
{
"publicCmHandleProperties" : {
"Name-1" : "some-value",
"Name-2" : "other-value"
}
} |
Code Block | ||
---|---|---|
| ||
{
"publicCmHandleProperties": {
"name": "Contact",
"value": "newemailforstore@bookstore.com"
}
} |
Response Body
Code Block | ||
---|---|---|
| ||
[
"cmHandle1",
"cmHandle2",
...,
"cmHandleN"
] |
Below is a sample yaml for OpenAPI.
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
executeSearchForMatchingPublicProperties:
post:
description: Execute search to get all cm handles for the given public properties
tags:
- network-cm-proxy
summary: Execute cm handle search using
operationId: executeSearchForMatchingPublicProperties
requestBody:
required: true
content:
application/json:
schema:
$ref: 'components.yaml#/components/schemas/PublicProperties'
responses:
200:
description: OK
content:
application/json:
schema:
type: array
items:
type: string
400:
$ref: 'components.yaml#/components/responses/BadRequest'
401:
$ref: 'components.yaml#/components/responses/Unauthorized'
403:
$ref: 'components.yaml#/components/responses/Forbidden'
404:
$ref: 'components.yaml#/components/responses/NotFound'
500:
$ref: 'components.yaml#/components/responses/InternalServerError' |
Public Properties are stored in the fragment table in CPS as seen below:
PoC Gerrit Review
A PoC has been coded and can be found here: https://gerrit.onap.org/r/c/cps/+/127541
Both properties match
(CM Handles Returned that Match)
One property doesn't match
(Nothing Returned)
No properties given - all cm handles returned
(that contain public properties)
Return 400
(BAD_REQUEST)
Future Example (Out-of-scope)
...
Modules in the example are to demonstrate future intentions but is out of scope for this user story.
References
scope for this user story.
References
...
CPS Swagger: http://localhost:8883/swagger-ui/index.html?configUrl=%2Fv3%2Fapi-docs%2Fswagger-config
https://docs.onap.org/projects/onap-cps/en/latest/cps-path.html
https://www.freeformatter.com/xpath-tester.html#ad-output
//public-properties[@name='Contact' and @value='xyz']/ancestor::cm-handles
org.onap.cps.spi.impl.CpsDataPersistenceQueryDataNodeSpec#Query for attribute by cps path of type ancestor with #scenario.CPS Swagger: http://localhost:8883/swagger-ui/index.html?configUrl=%2Fv3%2Fapi-docs%2Fswagger-config