Page History

...

Test Environment

TODO: Laptop specs

The performance tests are written in Groovy (a JVM language). As all CPS Core operations are synchronous, the results here are to be considered as single-threaded operation only.

Test data

Test data used complies with Open ROADM OpenROADM YANG model. Specifically, openroadm-device nodes consisting of 86 fragments are created. For example, a test that creates 1000 device nodes will result in 86,000 fragments in the database. Some tests use multiple anchors with similar data, up to 3000 device nodes, meaning that the system has been tested up to 1 million fragments.

Storing data nodes

A varying number of openroadm OpenROADM device nodes will be stored using CpsServiceCpsDataService::saveData.

...

• Storing data nodes has linear time complexity (as expected).
• Raw performance is roughly 3200 3000 fragments per second for the given test setup.
• Performance can be improved by enabling write batching (CPS-1795)
• There are edge cases with exponential complexity.

Updating data nodes

In this scenario, 1000 devices OpenROADM device nodes are already defined. A number of these existing data nodes will be updated using CpsServiceCpsDataService::updateDataNodeAndDescendants.

...

• Updating data nodes has linear time complexity (as expected).
• Raw performance is roughly 650 600 fragments per second for the given model and test setup.
• Updating data nodes is 5 times slower than storing data nodes.

Updating data leaves

In this scenario, 1000 devices OpenROADM device nodes are already defined. The data leaves of a number of these existing data nodes will be updated using CpsServiceCpsDataService::updateNodeLeaves.

Example JSON payload for updating two leaves ('status' and 'ne-state') data leaves for one device:

{
    'openroadm-device': [
        {'device-id':'C201-7-1A-1', 'status':'fail', 'ne-state':'jeopardy'}
    ]
}

Test Results

...

• Updating data leaves has linear time complexity.
• Raw performance is about 3000 fragments per second.

Reading data nodes

In these tests, a varying number of OpenROADM devices are created and retrieved.

Reading top-level container node

In this test, CpsDataService::getDataNodes is used to retrieve the top-level container node.

Test results

Reading the top-level container node with no descendants:

The above data clearly indicates constant time.

Reading the top-level container node with all descendants:

Image Added

Observations

• Reading a single top-level container node with no descendants has constant time (as expected).
• Reading a single top-level container node with all descendants has linear time (as expected).
• Raw performance of reading with all descendants is roughly 100,000 fragments per second.

Reading data nodes for multiple xpaths

This test uses CpsDataService::getDataNodesForMultipleXpaths with all descendants to retrieve a varying number of OpenROADM device nodes.

Test results

Image Added

Special case: attempting to read multiple non-existing data nodes

In this case, we attempt to read many non-existing data nodes:

(The above case appears to be constant time, but theoretically must be linear - we'll call it negligible time.)

Observations

• Reading many data nodes with all descendants has linear time (as expected).
• Attempting to read many non-existing data nodes takes negligible time.
• Raw performance of reading many with all descendants is roughly 80,000 fragments per second.

Additional test cases: Reading container node versus list

Recently, functionality was added to enable reading whole lists (CPS-1696). Here we compare performance of reading a container node containing a list, versus reading the list (with all descendants).

As can be seen, it is slower reading a list than reading the parent node containing the list.

Suggested improvement: Add a condition to the WHERE clause in the SQL for reading lists, to narrow the search space to children of the parent. For example:

WHERE (existing conditions)

  AND parent_id = (SELECT id FROM fragment WHERE xpath = '/parent-xpath')

Cps Path Queries

Deleting data nodes

...