The Platform Maturity requirements for Beijing are on this page:
Platform Maturity Requirements (aka Carrier Grade)
The ambition for the ONPA Policy Framework is to reach level 1 in performance in Beijing, see JIRA POLICY-392
For Performance Level 1, we have to define and measure our performance metrics:
Level 1: baseline performance criteria identified and measured (such as response time, transaction/message rate, latency, footprint, etc. to be defined on per component
I don't think we have to state what the metrics should be, it looks like we just define the metrics and measure them. For Level 2, an improvement plan must be created and implemented.
Although there should be measurements for design time and deployment time performance, here we focus on the run time performance metrics for policy execution in the PDPs, the most critical run time metrics for Policy. We propose to measure these metrics in a simulated environment and in a full ONAP deployment.
| No. | Metric | Description |
|---|---|---|
| 1 | Single Threaded Response Time | Measure the execution time for onset and abatement in each use case with only a single policy executing |
| 2 | Single Threaded CPU Usage | CPU Usage for each use case when executing alone |
| 3 | Single Threaded Memory Usage | Memory Usage for each use case when executing alone |
| 4 | Maximum Simultaneous Executions | Measure the maximum number of simultaneous policy executions that can be achieved whilst maintaining system stability and resource utilization |
| 5 | Installation Footprint | The requirements of the ONAP Policy Framework installation in terms of VM resources including the amount of disk space required for the database and for logs. |
5 Comments
Liam Fallon
Pamela Dragosh Jorge Hernandez Waqas Ikram Ram Krishna Verma I made a strawman type stab at defining some performance metrics for the PDPs as a start. Please feel free to edit at will.
Pamela Dragosh
Since Drools is single-threaded, it may not make sense to measure multi-threaded. Maybe the responsiveness for a drools VM that has multiple controllers loaded and running can/should be measured?
The XACML PDP sits behind a webapp so multi requests for that can be measured in that type of environment - not really multi-threaded within the PDP itself.
Liam Fallon
OK. let's remove the multi threaded ones.
Pamela Dragosh
Maybe it would be good to measure what is the level of performance right now under the default ONAP installation? If its acceptable and can be used as a guide.
Jim Hahn
Adjusting the meaning of "multi-threaded" to mean that multiple threads are injecting ONSETs simultaneously, not that the PDPD is necessarily running multiple threads simultaneously. That can be measured, if a real DMaaP is used for injections and notifications. Will continue to use the direct (i.e., non-DMaaP) approach for single-threaded measurements, as that provides focus on the PDPD without as much noise from the communication mechanism.