CPS Core Performance
Test Environment
TODO: Laptop specs
The performance tests are written in Groovy (a JVM language). As all CPS Core operations are synchronous, the results here are to be considered as single-threaded operation only.
...
Test data used complies with OpenROADM Open ROADM YANG model. Specifically, openroadm-device nodes consisting of 86 fragments are created. For example, a test that creates 1000 device nodes will result in 86,000 fragments in the database. Some tests use up to 3000 device nodes (258,000 fragments - equivalent to around 20,000 CM-handles in NCMP), with four anchors replicating the data, meaning that the system has been tested up to 1 million fragments. Not all results are displayed on this page, but are included in the attached spreadsheet.
Storing data nodes
A varying number of OpenROADM Open ROADM device nodes will be stored using CpsDataService::saveData.
...
Updating data nodes
In this scenario, 1000 OpenROADM Open ROADM device nodes are already defined. A number of these existing data nodes will be updated using CpsDataService::updateDataNodeAndDescendants.
...
Updating data leaves
In this scenario, 1000 OpenROADM Open ROADM device nodes are already defined. The data leaves of a number of these existing data nodes will be updated using CpsDataService::updateNodeLeaves.
...
- Updating data leaves has linear time complexity.
- Raw performance is about 3000 fragments per second.
Deleting data nodes
In this scenario, 300 Open ROADM device nodes are already defined. A number of these data nodes will be deleted using CpsDataService::deleteDataNodes. The types of nodes will be varied, for example, deleting container nodes, list elements, or whole lists.
Test results
| N | 50 | 100 | 150 | 200 | 250 | 300 | Example xpath |
| Delete top-level container node | - | - | - | - | - | 0.630 | /openroadm-devices |
| Batch delete N/300 container nodes | 0.150 | 0.261 | 0.377 | 0.453 | 0.553 | 0.686 | /openroadm-devices/openroadm-device[@device-id='C201-7-1A-10']/org-openroadm-device |
| Batch delete N/300 lists elements | 0.132 | 0.248 | 0.338 | 0.449 | 0.545 | 0.670 | /openroadm-devices/openroadm-device[@device-id='C201-7-1A-49'] |
| Batch delete N/300 whole lists | 0.509 | 1.054 | 1.401 | 1.848 | 2.134 | 2.555 | /openroadm-devices/openroadm-device[@device-id='C201-7-1A-293']/org-openroadm-device/degree |
| Try batch delete N/300 non-existing | 0.250 | 0.535 | 0.667 | 0.951 | 1.145 | 1.318 | /path/to/non-existing/node[@id='27'] |
Observations
- Delete performance is linear on the amount of data being deleted.
- Raw performance of deleting containers of list elements is around 40,000 fragments per second.
- Of note, attempting to delete non-existing data nodes takes longer than actually deleting the equivalent amount of nodes with descendants - it is a slow operation.
- Deleting lists is much slower than deleting the parent container of the list.
Suggested improvement: For whole list deletion, add a condition to the WHERE clause in the SQL for deleting lists, to narrow the search space to children of the parent. For example:
WHERE (existing conditions)AND parent_id = (SELECT id FROM fragment WHERE xpath = '/parent-xpath')
This should narrow the performance gap in this case.
Reading data nodes
In these tests, a varying number of OpenROADM Open ROADM devices are created and retrieved.
...
This test uses CpsDataService::getDataNodesForMultipleXpaths with all descendants to retrieve a varying number of OpenROADM Open ROADM device nodes.
Test results
...