| Table of Contents |
|---|
Summary
Stability Performance test plan was triggered for 72 hours injecting vCPE, vFirewall, vDNS, VOLTE onset & abatement (where applicable) messages through REST interface of Drools PDP installed in windriver lab (PFPP).
The test environment was set-up using simulated components i.e. A&AI, VFC and SO using "features enable controlloop-utils". However the performance test subscribes to the POLICY-CL-MGT topic, which is not simulated, to determine when various steps have completed.
The performance test runs 10 simultaneous threads injecting ONSET messages. In each thread, each of the different types of ONSETs is injected and processed in a serial fashion, first vCPE is injected and its associated APPC requests are answered, then vFirewall is injected, and so on. Each thread repeats this process 1000 times.
APPC response messages were injected in test plan wherever needed.
...
...
| Use case | Average time taken to execute one ONSET message | Sample taken for average |
|---|---|---|
| vCPE | 3 341 ms | Last 10,000 |
| vFirewall | 2 184 ms | Last 10,000 |
| vDNS | 8 18 ms | Last 10,000 |
| VOLTE | 7 16 ms | Last 10,000 |
CPU Utilization
...
Time taken by APP-C
Each thread in the performance test script waits for notifications, via the POLICY-CL-MGT topic, to indicate that an APP-C request has been issued, and then injects the APP-C response via the REST interface. Thus the elapsed time reported above includes time spent from the publish to POLICY-CL-MGT until the response is received on the REST interface. This was calculated by examining all network.log files corresponding to the time when the test was executed using the below script
| Average time taken to publish request to POLICY-CL-MGT and receive response on REST interface | Sample taken for average |
|---|---|
| 142 ms | 30,000 |
The above time applies twice for vCPE, to handle the APPC-reponse and the ABATEMENT, it applies once for vFirewall, to handle the APPC-response, and it applies zero times for vDNS and VOLTE. Eliminating these times results in the following elapsed times for Drools PDP.
| Use case | Average time taken to execute one ONSET message without DMaaP time |
|---|---|
| vCPE | 57 ms |
| vFirewall | 42 ms |
| vDNS | 18 ms |
| VOLTE | 16 ms |
CPU Utilization
Total CPU used by the PDP was measured before and after the test, using "ps -l".
| Initial CPU time | Final CPU time | Total CPU used during test | Average CPU per ONSET |
|---|---|---|---|
| 00:36:55 | 00:43:28 | 393 ms | 9.8 ms |
Memory Utilization
| Code Block |
|---|
Number of young garbage collection: 67111789 Avg. Young garbage collection time: ~7~5.099 ms per collection Number of Full garbage collection: 43 Avg. Full garbage collection time: ~197~112 ms per collection S0 S1 E O M CCS YGC YGCT FGC FGCT GCT 0.00 90.621789 4510.71564 21.36 97.74 93.42 6711 47.578 3 4 0.788337 48.366 10.901 |
Performance Metrics
No. | Metric | Description | Result | Comments | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 4 | Maximum Simultaneous Executions | Measure the maximum number of simultaneous policy executions that can be achieved whilst maintaining system stability and resource utilization | 10 | DMaaP connection limitations prevented the test from running more than 10 simultaneous threads/ONSETs | |||||||
| 5 | Multi- | 1 | SingleThreaded Response Time | Measure the execution time for onset and abatement in each use case | with only a single policy executingwhen multiple threads are injecting ONSET events simultaneously | vCPE - 3 57 ms vFirewall - 2 42 ms vDNS - 8 18 ms VOLTE - 7 16 ms | The average time taken is calculated based on last 10,000 samples in audit.log file. | ||||
| 6 | Multi | 2 | Single Threaded CPU Usage | CPU Usage for each use case when executing alone | ~ 1% | multiple threads are injecting ONSET events simultaneously | 9.8 ms | ||||
| 7 | Multi | 3 | Single Threaded Memory Usage | Memory Usage Memory Usage for each use case when executing alonemultiple threads are injecting ONSET events simultaneously | ~ 100 MB | 4 | Maximum Simultaneous Executions | Measure the maximum number of simultaneous policy executions that can be achieved whilst maintaining system stability and resource utilization | 2 | The current size of source/sink topics is 10. And 1 onset message results to 5 messages in sink topic. Hence we can only run 2 simaltaneous thread and assert the values coming in sink topics. |