...
Although there should be measurements for design time and deployment time performance, here we focus on the run time performance metrics for policy execution in the PDPs, the most critical run time metrics for Policy. We propose to measure these metrics in a simulated environment and in a full ONAP deployment.
Measurements for the "Single Threaded" case are made using the Stability test, which uses timers to determine when to inject responses via REST. It also uses the REST API to look for "FINAL" states, thus it is limited to two threads. On the other hand, the measurements for the "Multi Threaded" case are made using the Performance test, which uses DMaaP notifications to determine when to inject responses via REST. It also uses DMaaP notifications to look for "FINAL" states, thus it is only limited by how many simultaneous consumers and connections are supported by DMaaP.
| No. | Metric | Description |
|---|---|---|
| 1 | Single Threaded Response Time | Measure the execution time for onset and abatement in each use case with only a single policy executing |
| 2 | Single Threaded CPU Usage | CPU Usage for each use case when executing alone |
| 3 | Single Threaded Memory Usage | Memory Usage for each use case when executing alone |
| 4 | Maximum Simultaneous Executions | Measure the maximum number of simultaneous policy executions that can be achieved whilst maintaining system stability and resource utilization |
| 5 | Multi - Threaded Response Time | Measure the execution time for onset and abatement in each use case when multiple threads are injecting ONSET events simultaneously |
| 6 | Multi Threaded CPU Usage | CPU Usage for each use case when multiple threads are injecting ONSET events simultaneously |
| 7 | Multi Threaded Memory Usage | Memory Usage for each use case when executing in multiple threads |
| 8 | Installation Footprint | The requirements of the ONAP Policy Framework installation in terms of VM resources including the amount of disk space required for the database and for logs. |
...