...
There is work in progress to address this behavior using a custom algorithm using JDBC. The use of a graph database would implicitly fix such issues.
Updating data nodes
Batch updating a collection using updateDataNodesAndDescendants (plural)
In this scenario, 1,000 Open ROADM device nodes are already defined. A number of these existing data nodes will be updated using CpsDataService::updateDataNodeAndDescendantsupdateDataNodesAndDescendants (plural).
| Updated device nodes | 1 | 100 | 200 | 300 | 400 | 500 | 600 | 700 | 800 | 900 | 1,000 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Fragments updated | 86 | 8,600 | 17,200 | 25,800 | 34,400 | 43,000 | 51,600 | 60,200 | 68,800 | 77,400 | 86,000 |
Time (seconds) | 0.22 | 12.79 | 28.38 | 44.23 | 51.55 | 69.46 | 85.67 | 95.02 | 109.16 | 117.00 | 131.15 |
| Fragments | 391 | 672 | 606 | 583 | 667 | 619 | 602 | 634 | 630 | 662 | 656 |
...
There are a number of issues possibly affecting performance here. For example, after DataNodes are converted to FragmentEntity objects, they are being persisted using the Hibernate persist method, which will always write the changes, regardless of whether the data changes. There is an alternative method called merge which compares existing data to updated data. Even if this were implemented, there are additional issues, such as JSON encoding giving inconsistent ordering. For example, data leaves could be encoded as "{'x':'1', 'y':'2'}" or "{'y':'2','x':'1'}", depending on exact object type for storing leaves (HashMap, LinkedHashMap, ImmutableMap, etc.). There is an option for JsonObjectMapper to order the keys alphabetically during encoding.
WIP Comparison of updating using different approaches
There are multiple was of updating data nodes, each with different performance.
In these scenario, 1,000 Open ROADM device nodes are already defined. A number of these existing data nodes will be updated using CpsDataService::updateDataNodeAndDescendants (singular).
| Scenario | Time (seconds) | Parent Xpath | Sample Json | Remarks |
|---|---|---|---|---|
| Update 1 device node under top-level container | 0.6 | /openroadm-devices | { "openroadm-device": [ ... ] } | |
| Update a list of 100 device nodes under top-level container | 3 | /openroadm-devices | { "openroadm-device": [ ... ] } | |
| Update top-level container having a list of 100 device nodes | 22 | / | { "openroadm-devices": { "openroadm-device": [ ... ] } } |
To Do: this section is be completed pending further analysis
Updating data leaves
In this scenario, 1,000 Open ROADM device nodes are already defined. The data leaves of a number of these existing data nodes will be updated using CpsDataService::updateNodeLeaves.
...