...
The performance test runs 10 simultaneous threads injecting ONSET messages. In each thread, each of the different types of ONSETs is injected and processed in a serial fashion, first vCPE is injected and its associated APPC requests are answered, then vFirewall is injected, and so on. Each thread repeats this process 1000 times. APPC response messages were injected in the test plan wherever needed.
Average processing time was calculated using the last 10000 records from audit.log for each use case, as well as the matching transaction time extracted from the network.log, using the below script:
The 72 hour stability test was executed on a VM running the policy docker containers in an openstack cloud instance. The test execution resulted in ~135 million getDecision requests. Of those 99.74% returned the expected response, while the remaining 0.26% unexpectedly returned response code 401 "Unauthorised". CPU and memory usage can be seen in below graphs:
Result
Time taken by Drools PDP
| Code Block |
|---|
Elapsed time for vCPE :
matched 8870 samples, average 18 ms
unmatched 1130 samples, average 339 ms
Elapsed time for vFirewall :
matched 8871 samples, average 55 ms
unmatched 1129 samples, average 177 ms
Elapsed time for vDNS :
matched 8869 samples, average 9 ms
unmatched 1131 samples, average 16 ms
Elapsed time for VOLTE :
matched 8868 samples, average 9 ms
unmatched 1132 samples, average 15 ms
|
Note: the “unmatched samples” are requests for which no corresponding network time could be identified by the reporting tool. Hence those numbers represent the total elapsed time, with nothing subtracted out.
calling Policy Engine RESTful APIs to create and push policies to the PDP-X for Guard, Guard Min Max, and Config Base policy types, each thread repeats this process 10 times. Once the first thread group is complete, the performance test runs 10 simultaneous threads calling Policy Engine RESTful APIs to get decisions for Guard and Guard Min/Max policies on the PDP-X, each thread repeats this process 1000 times.
The test execution resulted in 20000 getDecision PERMIT requests resulting in 99.87% returning the expected response and 20000 getDecision DENY requests resulting in 99.79% returning the expected response with throughput averaging about 2.3 transactions per second. CPU and memory usage are provided below with JMeter results.
Result
CPU Utilization
Total CPU used by the PDP was measured before and after the test, using "ps -l".
| Initial CPU time | Final CPU time | Total CPU used during test | Average CPU per ONSETrequest | Initial CPU% | Initial Memory% | Final CPU% | Final Memory% |
|---|---|---|---|---|---|---|---|
| 00:3602:553300 | 01:4356:2808 | 393 6815 ms | 9.8 ms | 9.4% | 4.4% | 92.3% | 8.9% |
Memory Utilization
| Code Block |
|---|
Number of young garbage collections used during the test: 14681506 Avg. Young garbage collection time: ~5~34.817 ms per collection Total number of Full garbage collection: 35 Avg. Full garbage collection time: ~112~98.4 ms per collection S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT GCT 204852736.0 204849664.0 17713.7 0.0 704246784.0 6809672845.07 59110.3 432640699392.0 8611661080.68 7334478336.0 7184976078.43 83209472.0 78019004.24 32124 21.033559 3 0.337584 2.370143 256027648.0 153627648.0 24017.9 0.0 1088293888.0 93184.0 8289062859.4 432640699392.0 143366168780.10 7334482944.0 7188280562.95 83209472.0 78049027.86 17891530 1053.564013 35 0.3371.076 1054.901088 |
Performance Metrics
...
No.
...
Metric
...
Description
...
vCPE - 18 ms
vFirewall - 55 ms
vDNS - 9 ms
VOLTE - 9 ms
Jmeter Results Summary
...