CPS-222: Spike: CPS data update options

Table of Contents:

Addresses: 

Parsing and validation of data fragment

Problem description

Initial implementation of data insertion expects full data tree to be provided as input then it's being validated and parsed
versus root defined model starting from top level defined container element. The root element of a  model is determined
automatically with no extra action required. 

When not full data tree but just a fragment is passed as input the validation and subsequent arsing of the data fails
because the structure of data no longer matches the model (if compared from roots). 

Solution proposal

ODL Yang Tools library provides an option to define a point within a model which will be used as root when data is validated
and parsed. It requires an appropriate DataSchemaNode instance to be provided to JsonParserStream instance as a custom root point.

It assumes the implementation/update of a following logic:

• Extracting the DataSchemaNode (from SchemaContext) matching the xpath requested
• Update the  YangUtils.parseJsonData(..) method to utilize additional input
• Update the DataNodeBuilder logic to accept non-empty xpath when building data from NormalizedNode instance 

Alternative solutions

The initial proposals described 2 simplified approached:

• Updating leaf values only (without validation)
• Updating full tree

Both require almost same effort (comparing to proposed above) with significantly less features delivered. Also these solutions 
are temporary and assume refactoring and proper implementation in future.

Also

According to initial DB schema design (see CPS Internal Relation DB Schema) there are SCHEMA_NODE table and associated
SCHEMA_NODE_ID reference field in FRAGMENT. The assumption these were designed to store the reference to DataSchemaNode object
associated with particular data node for usage as described above.

However the object itself is too complex to by stored as is (serialized). Storing the node identifier as reference to schema node object
is less reliable then just a path from the root (already stored in xpath). Also fetching data by xpath to identify the schema object 
new data belong to (for validation and parsing) seems not optimal and over-complicated.

Data Input / Output Consistency

In order to make custom root pointer solution (described above) work properly, it should point to parent of data node being updated.
At the same time the JSON data require to be presented as a single entry explicitly defining a type of a current node.

From other hand when same data node is requested for output (GET) it addressed by direct xpath and value returned is a data
unwrapped (as is: multiple top level elements).

The difference in same parameters meaning depending on action performed (while addressing same data fragment) makes the
exposed API inconsistent (for update case, acceptable for child addition case).

Below is example using a data fragment referencing ran-network2020-08-06.yang model 

// xPath: /ran-network/NearRTRIC[@idNearRTRIC='11']/GNBDUFunction[@idGNBDUFunction='1']/NRCellDU[@idNRCellDU='103594001']/attributes/pLMNInfoList[@mcc='310' and @mnc='410']
{ 
   "mcc": "310",
   "mnc": "410",
   "sNSSAIList": [{
       "sNssai": "10000100",
       "status": "INACTIVE",
       "configData": [{
           "configParameter": "maxNumberOfConns",
           "configValue": 5000
        }]
    ]}
}

// xPath: /ran-network/NearRTRIC[@idNearRTRIC='11']/GNBDUFunction[@idGNBDUFunction='1']/NRCellDU[@idNRCellDU='103594001']/attributes
{
   "pLMNInfoList": [{

       "sNSSAIList": [{
            "sNssai": "10000100",
            "status": "INACTIVE",
            "configData": [{
                "configParameter": "maxNumberOfConns",
                "configValue": 5000
            }]
        }]
    }]
}

Pointing to parent node in order to parse the child correctly is mainly an internal requirement. It means there could be 
dynamic update of both incoming parameters (as internal logic): 

• shortening the xpath by 1 entry to address parent
• wrapping incoming JSON with a required type pointer (using last entry of xpath)

Also the following should be taken into account:

• the incoming data validation/parsing is required for both data node adding and updating cases, so it's preferable to use same
  logic for both cases
• extra validation is necessary to prevent key attributes change for mapped list elements
• difference in parameters for add/update cases will require extra effort for service consumer to compose a proper request

Partial Data Storage

As it was initially implemented there is no validation for parent data node existence when persisting data nodes.

It is unnecessary when the whole data tree is persisted (A) because there is a data for all the levels with no gaps. 
With non-empty initial xpath it makes it possible to persist data nodes from lower levels of structure hierarchy without upper
level data nodes (B). 

From one hand it seems extra to persist upper level nodes if these nodes are never requested. From other hand it could lead to data inconsistency
if the parent node being added after child nodes (C). The case  however can be resolved by existing data check before inserting new entries.

Points to discuss/clarify

Addressing a data node through parent xpath in API for insert and update operations (options):

• using parent xpath for insert only, direct xpath for update 
• using parent xpath for insert and update operations

The naming and default value for an option (boolean type of) to instruct the descendants update together with the data node.
The option used for data extraction is called include-descendants, false by default. same to be defined for data update operation:
is it to have same name, what's the default value. 

Allowing data node persistence without a parent data nodes. Options:

• allow, requires implementation of data check for existing parents/children on data node insertion
• prohibit, requires the implementation of xpath being root node check for data nodes inserted without a parent

Remove SCHEMA_NODE table and references as extra from database or clarify the reason for this data to be stored.